@adminide-stack/yantra-help-browser 12.0.16-alpha.27 → 12.0.16-alpha.29

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (163) hide show
  1. package/lib/components/HelpCenterFooter.d.ts.map +1 -1
  2. package/lib/components/HelpCenterFooter.js +43 -88
  3. package/lib/components/HelpCenterFooter.js.map +1 -1
  4. package/lib/components/HelpCenterHeader.d.ts.map +1 -1
  5. package/lib/components/HelpCenterHeader.js +3 -8
  6. package/lib/components/HelpCenterHeader.js.map +1 -1
  7. package/lib/components/Icons.d.ts +55 -0
  8. package/lib/components/Icons.d.ts.map +1 -0
  9. package/lib/{pages/LandingPage/components → components}/Icons.js +39 -2
  10. package/lib/components/Icons.js.map +1 -0
  11. package/lib/components/Logo.d.ts +1 -2
  12. package/lib/components/Logo.d.ts.map +1 -1
  13. package/lib/components/Logo.js +2 -9
  14. package/lib/components/Logo.js.map +1 -1
  15. package/lib/components/PageHero.d.ts +9 -0
  16. package/lib/components/PageHero.d.ts.map +1 -0
  17. package/lib/components/PageHero.js +47 -0
  18. package/lib/components/PageHero.js.map +1 -0
  19. package/lib/components/SearchBar.d.ts.map +1 -1
  20. package/lib/components/SearchBar.js +9 -45
  21. package/lib/components/SearchBar.js.map +1 -1
  22. package/lib/components/SidebarSearch.js +2 -2
  23. package/lib/components/SidebarSearch.js.map +1 -1
  24. package/lib/components/navbar/index.d.ts.map +1 -1
  25. package/lib/components/navbar/index.js +5 -16
  26. package/lib/components/navbar/index.js.map +1 -1
  27. package/lib/compute.d.ts +15 -0
  28. package/lib/compute.d.ts.map +1 -1
  29. package/lib/compute.js +74 -2
  30. package/lib/compute.js.map +1 -1
  31. package/lib/pages/About/index.d.ts +3 -0
  32. package/lib/pages/About/index.d.ts.map +1 -0
  33. package/lib/pages/About/index.js +69 -0
  34. package/lib/pages/About/index.js.map +1 -0
  35. package/lib/pages/Careers/index.d.ts +3 -0
  36. package/lib/pages/Careers/index.d.ts.map +1 -0
  37. package/lib/pages/Careers/index.js +78 -0
  38. package/lib/pages/Careers/index.js.map +1 -0
  39. package/lib/pages/CategoryCollection/index.d.ts.map +1 -1
  40. package/lib/pages/CategoryCollection/index.js +15 -78
  41. package/lib/pages/CategoryCollection/index.js.map +1 -1
  42. package/lib/pages/Community/index.d.ts +3 -0
  43. package/lib/pages/Community/index.d.ts.map +1 -0
  44. package/lib/pages/Community/index.js +72 -0
  45. package/lib/pages/Community/index.js.map +1 -0
  46. package/lib/pages/Contact/index.d.ts +3 -0
  47. package/lib/pages/Contact/index.d.ts.map +1 -0
  48. package/lib/pages/Contact/index.js +128 -0
  49. package/lib/pages/Contact/index.js.map +1 -0
  50. package/lib/pages/GetStarted/components/ExampleCard.d.ts.map +1 -1
  51. package/lib/pages/GetStarted/index.d.ts.map +1 -1
  52. package/lib/pages/GetStarted/index.js +9 -45
  53. package/lib/pages/GetStarted/index.js.map +1 -1
  54. package/lib/pages/HelpCenter/components/HelpCategoryCard.d.ts.map +1 -1
  55. package/lib/pages/HelpCenter/components/HelpCategoryCard.js +7 -34
  56. package/lib/pages/HelpCenter/components/HelpCategoryCard.js.map +1 -1
  57. package/lib/pages/HelpCenter/components/PopularArticle.d.ts.map +1 -1
  58. package/lib/pages/HelpCenter/components/PopularArticle.js +3 -12
  59. package/lib/pages/HelpCenter/components/PopularArticle.js.map +1 -1
  60. package/lib/pages/HelpCenter/index.d.ts.map +1 -1
  61. package/lib/pages/HelpCenter/index.js +15 -74
  62. package/lib/pages/HelpCenter/index.js.map +1 -1
  63. package/lib/pages/LandingPage/components/ArticleCard.d.ts.map +1 -1
  64. package/lib/pages/LandingPage/components/ArticleCard.js +3 -12
  65. package/lib/pages/LandingPage/components/ArticleCard.js.map +1 -1
  66. package/lib/pages/LandingPage/components/CategoryCard.d.ts.map +1 -1
  67. package/lib/pages/LandingPage/components/CategoryCard.js +3 -12
  68. package/lib/pages/LandingPage/components/CategoryCard.js.map +1 -1
  69. package/lib/pages/LandingPage/components/HeroSection.d.ts.map +1 -1
  70. package/lib/pages/LandingPage/components/HeroSection.js +3 -8
  71. package/lib/pages/LandingPage/components/HeroSection.js.map +1 -1
  72. package/lib/pages/LandingPage/components/ResourceCard.d.ts.map +1 -1
  73. package/lib/pages/LandingPage/components/ResourceCard.js +3 -12
  74. package/lib/pages/LandingPage/components/ResourceCard.js.map +1 -1
  75. package/lib/pages/LandingPage/index.d.ts.map +1 -1
  76. package/lib/pages/LandingPage/index.js +62 -33
  77. package/lib/pages/LandingPage/index.js.map +1 -1
  78. package/lib/pages/Markdown/MarkdownPageLayout.d.ts.map +1 -1
  79. package/lib/pages/Markdown/MarkdownPageLayout.js +11 -56
  80. package/lib/pages/Markdown/MarkdownPageLayout.js.map +1 -1
  81. package/lib/pages/Privacy/index.d.ts +3 -0
  82. package/lib/pages/Privacy/index.d.ts.map +1 -0
  83. package/lib/pages/Privacy/index.js +449 -0
  84. package/lib/pages/Privacy/index.js.map +1 -0
  85. package/lib/pages/ReleaseNotes/index.d.ts +3 -0
  86. package/lib/pages/ReleaseNotes/index.d.ts.map +1 -0
  87. package/lib/pages/ReleaseNotes/index.js +61 -0
  88. package/lib/pages/ReleaseNotes/index.js.map +1 -0
  89. package/lib/pages/StatusPage/index.d.ts +3 -0
  90. package/lib/pages/StatusPage/index.d.ts.map +1 -0
  91. package/lib/pages/StatusPage/index.js +85 -0
  92. package/lib/pages/StatusPage/index.js.map +1 -0
  93. package/lib/pages/Terms/index.d.ts +3 -0
  94. package/lib/pages/Terms/index.d.ts.map +1 -0
  95. package/lib/pages/Terms/index.js +60 -0
  96. package/lib/pages/Terms/index.js.map +1 -0
  97. package/lib/routes.json +88 -0
  98. package/lib/templates/content/account-management/account-setup.md +56 -57
  99. package/lib/templates/content/account-management/delete-account.md +55 -58
  100. package/lib/templates/content/account-management/preferences.md +70 -55
  101. package/lib/templates/content/account-management/privacy-settings.md +67 -56
  102. package/lib/templates/content/account-management/profile-settings.md +56 -45
  103. package/lib/templates/content/browser-extension/browser-extension-overview.md +60 -0
  104. package/lib/templates/content/browser-extension/extension-security.md +126 -0
  105. package/lib/templates/content/browser-extension/getting-started-extension.md +96 -0
  106. package/lib/templates/content/browser-extension/how-it-works.md +112 -0
  107. package/lib/templates/content/browser-extension/troubleshooting-extension.md +138 -0
  108. package/lib/templates/content/browser-extension/use-cases-workflows.md +123 -0
  109. package/lib/templates/content/content-manifest.json +328 -0
  110. package/lib/templates/content/data-privacy/data-collection.md +55 -51
  111. package/lib/templates/content/data-privacy/privacy-policy.md +74 -57
  112. package/lib/templates/content/data-subject-privacy/data-access.md +59 -69
  113. package/lib/templates/content/data-subject-privacy/data-portability.md +67 -93
  114. package/lib/templates/content/data-subject-privacy/privacy-requests.md +66 -62
  115. package/lib/templates/content/file-uploads/file-upload-overview.md +69 -50
  116. package/lib/templates/content/getting-started/getting-started-guide.md +65 -51
  117. package/lib/templates/content/product-features/ai-models.md +62 -57
  118. package/lib/templates/content/product-features/collaboration-tools.md +66 -45
  119. package/lib/templates/content/product-features/conversation-features.md +59 -34
  120. package/lib/templates/content/product-features/export-features.md +65 -45
  121. package/lib/templates/content/product-features/follow-up-questions.md +61 -46
  122. package/lib/templates/content/product-features/real-time-search.md +63 -46
  123. package/lib/templates/content/product-features/saved-searches.md +53 -45
  124. package/lib/templates/content/product-features/search-features.md +61 -43
  125. package/lib/templates/content/product-features/search-history.md +64 -45
  126. package/lib/templates/content/product-features/source-citations.md +77 -44
  127. package/lib/templates/content/scope-api/api-overview.md +83 -46
  128. package/lib/templates/content/search-modes/deep-research.md +87 -46
  129. package/lib/templates/content/search-modes/labs-features.md +54 -56
  130. package/lib/templates/content/search-modes/pro-search.md +53 -45
  131. package/lib/templates/content/search-modes/regular-search.md +64 -46
  132. package/lib/templates/content/spaces-library/spaces-overview.md +68 -47
  133. package/lib/templates/content/student-hub/academic-research.md +85 -48
  134. package/lib/templates/content/student-hub/student-discounts.md +51 -57
  135. package/lib/templates/content/student-hub/student-overview.md +64 -45
  136. package/lib/templates/content/student-hub/study-tools.md +99 -45
  137. package/lib/templates/content/subscription-billing/billing-cycle.md +50 -48
  138. package/lib/templates/content/subscription-billing/billing-overview.md +55 -34
  139. package/lib/templates/content/subscription-billing/billing-support.md +52 -45
  140. package/lib/templates/content/subscription-billing/currency-support.md +36 -62
  141. package/lib/templates/content/subscription-billing/enterprise-pricing.md +52 -46
  142. package/lib/templates/content/subscription-billing/invoice-management.md +64 -45
  143. package/lib/templates/content/subscription-billing/payment-methods.md +53 -52
  144. package/lib/templates/content/subscription-billing/promotional-offers.md +58 -46
  145. package/lib/templates/content/subscription-billing/refund-policy.md +53 -58
  146. package/lib/templates/content/subscription-billing/student-discounts.md +61 -57
  147. package/lib/templates/content/subscription-billing/tax-information.md +64 -45
  148. package/lib/templates/content/technical-questions/ai-models-technical.md +60 -56
  149. package/lib/templates/content/technical-questions/api-technical.md +131 -56
  150. package/lib/templates/content/technical-questions/data-processing.md +74 -55
  151. package/lib/templates/content/technical-questions/database-architecture.md +91 -54
  152. package/lib/templates/content/technical-questions/infrastructure.md +104 -55
  153. package/lib/templates/content/technical-questions/performance-optimization.md +77 -55
  154. package/lib/templates/content/technical-questions/search-algorithms.md +78 -56
  155. package/lib/templates/content/technical-questions/technical-overview.md +76 -52
  156. package/lib/templates/content/threads/conversation-management.md +81 -45
  157. package/lib/templates/content/threads/threads-overview.md +69 -45
  158. package/lib/templates/content/troubleshooting/common-issues.md +91 -43
  159. package/lib/templates/content/what-is-yantra/getting-started-yantra.md +58 -57
  160. package/package.json +2 -2
  161. package/lib/pages/LandingPage/components/Icons.d.ts +0 -13
  162. package/lib/pages/LandingPage/components/Icons.d.ts.map +0 -1
  163. package/lib/pages/LandingPage/components/Icons.js.map +0 -1
@@ -1,83 +1,105 @@
1
1
  # Performance Optimization
2
2
 
3
- Technical optimizations for speed and efficiency.
3
+ How Yantra achieves sub-second response times at scale — and how you can get the most out of the platform.
4
4
 
5
- ## System Performance
5
+ ---
6
6
 
7
- ### Response Time Optimization
7
+ ## System Performance Targets
8
8
 
9
- - **Query Optimization**: Database query optimization
10
- - **Caching Strategies**: Intelligent caching
11
- - **CDN Integration**: Content delivery network
12
- - **Load Balancing**: Advanced load balancing
9
+ | Metric | Target | Current |
10
+ | ------------------------- | -------------- | -------------- |
11
+ | Search latency (P50) | < 200 ms | 145 ms |
12
+ | Search latency (P99) | < 1 s | 780 ms |
13
+ | AI response latency (P50) | < 800 ms | 620 ms |
14
+ | API uptime | 99.95% | 99.98% |
15
+ | Throughput | 10,000 req/min | 14,200 req/min |
13
16
 
14
- ### Throughput Optimization
17
+ ---
15
18
 
16
- - **Parallel Processing**: Multi-threaded processing
17
- - **Async Operations**: Asynchronous operations
18
- - **Connection Pooling**: Database connection pooling
19
- - **Resource Management**: Efficient resource management
19
+ ## Caching Strategy
20
+
21
+ Yantra uses a **multi-layer caching architecture** to minimize redundant computation:
22
+
23
+ ### Layer 1 — Edge cache (CDN)
24
+
25
+ Static assets and frequently requested API responses are cached at 200+ edge locations worldwide. Cache hit rates typically exceed 85%.
26
+
27
+ ### Layer 2 — Application cache (Redis)
28
+
29
+ - **Query result cache** — Identical queries return cached results within 5 ms.
30
+ - **Session cache** — User sessions and preferences are stored in Redis for instant access.
31
+ - **Embedding cache** — Pre-computed embeddings for popular documents avoid redundant inference.
32
+
33
+ ### Layer 3 — Database cache
34
+
35
+ - **Connection pooling** — PgBouncer maintains a pool of warm database connections, eliminating connection overhead.
36
+ - **Materialized views** — Complex aggregation queries use pre-computed materialized views refreshed every 5 minutes.
37
+ - **Query plan caching** — PostgreSQL's prepared statements reuse optimized query plans.
38
+
39
+ ---
20
40
 
21
41
  ## Database Optimization
22
42
 
23
- ### Query Performance
43
+ ### Indexing strategy
24
44
 
25
- - **Index Optimization**: Database index optimization
26
- - **Query Analysis**: Query performance analysis
27
- - **Execution Plans**: Query execution optimization
28
- - **Connection Optimization**: Database connection optimization
45
+ - **B-tree indexes** on primary keys and frequently filtered columns.
46
+ - **GIN indexes** on JSONB fields for flexible metadata queries.
47
+ - **Partial indexes** on hot data subsets (e.g., active users, recent content).
48
+ - **Covering indexes** that include all queried columns to avoid table lookups.
29
49
 
30
- ### Storage Optimization
50
+ ### Query optimization
31
51
 
32
- - **Data Partitioning**: Database partitioning
33
- - **Compression**: Data compression techniques
34
- - **Archiving**: Data archiving strategies
35
- - **Cleanup**: Automated data cleanup
52
+ Every query is analyzed for:
36
53
 
37
- ## Application Optimization
54
+ 1. **Execution plan review** — EXPLAIN ANALYZE is run on all new queries during code review.
55
+ 2. **N+1 detection** — Automated detection of N+1 query patterns in development.
56
+ 3. **Slow query logging** — Queries exceeding 100 ms are logged and reviewed weekly.
57
+ 4. **Connection management** — Read replicas handle analytics queries to keep the primary database responsive.
38
58
 
39
- ### Code Optimization
59
+ ---
40
60
 
41
- - **Algorithm Optimization**: Algorithm efficiency
42
- - **Memory Management**: Memory optimization
43
- - **CPU Optimization**: CPU usage optimization
44
- - **I/O Optimization**: Input/output optimization
61
+ ## Application-Level Optimization
45
62
 
46
- ### Framework Optimization
63
+ ### Frontend performance
47
64
 
48
- - **Framework Tuning**: Framework configuration
49
- - **Middleware Optimization**: Middleware performance
50
- - **Library Optimization**: Third-party library optimization
51
- - **Dependency Management**: Dependency optimization
65
+ - **Code splitting** Routes are lazy-loaded so users only download the JavaScript they need.
66
+ - **Image optimization** All images are served in WebP/AVIF format with responsive srcsets.
67
+ - **Prefetching** The most likely next navigation target is prefetched in the background.
68
+ - **Bundle analysis** We maintain strict bundle size budgets (< 200 KB initial JS, < 50 KB CSS).
52
69
 
53
- ## Infrastructure Optimization
70
+ ### Backend performance
54
71
 
55
- ### Server Optimization
72
+ - **Async I/O** — All I/O-bound operations (database queries, API calls, file reads) are non-blocking.
73
+ - **Worker pools** — CPU-intensive tasks (embedding generation, document parsing) run in dedicated worker processes.
74
+ - **Streaming responses** — AI-generated responses stream token-by-token for perceived instant feedback.
75
+ - **Graceful degradation** — If a dependency is slow, the system returns partial results rather than timing out entirely.
56
76
 
57
- - **Hardware Optimization**: Hardware configuration
58
- - **OS Tuning**: Operating system tuning
59
- - **Network Optimization**: Network performance
60
- - **Storage Optimization**: Storage performance
77
+ ---
78
+
79
+ ## Infrastructure Optimization
61
80
 
62
- ### Cloud Optimization
81
+ ### Auto-scaling
63
82
 
64
- - **Resource Allocation**: Optimal resource allocation
65
- - **Auto-scaling**: Intelligent auto-scaling
66
- - **Cost Optimization**: Cloud cost optimization
67
- - **Performance Monitoring**: Performance monitoring
83
+ | Component | Scaling metric | Scale range |
84
+ | ---------------------- | ----------------------------- | ------------------ |
85
+ | API servers | Request rate + latency | 4 to 64 pods |
86
+ | AI workers | GPU utilization + queue depth | 2 to 16 nodes |
87
+ | Kafka consumers | Consumer lag | 3 to 24 partitions |
88
+ | Database read replicas | Connection count | 1 to 4 replicas |
68
89
 
69
- ## Monitoring and Profiling
90
+ ### Cost optimization
70
91
 
71
- ### Performance Monitoring
92
+ - **Spot instances** for batch processing workloads (60% cost savings).
93
+ - **Reserved instances** for always-on services (40% cost savings).
94
+ - **Right-sizing** — Monthly reviews of resource utilization with automated recommendations.
95
+ - **Tiered storage** — Infrequently accessed data is automatically moved to cheaper storage tiers.
72
96
 
73
- - **Real-time Monitoring**: Real-time performance monitoring
74
- - **Metrics Collection**: Performance metrics collection
75
- - **Alerting**: Performance alerting
76
- - **Dashboards**: Performance dashboards
97
+ ---
77
98
 
78
- ### Profiling Tools
99
+ ## Tips for Optimizing Your Experience
79
100
 
80
- - **Application Profiling**: Application performance profiling
81
- - **Database Profiling**: Database performance profiling
82
- - **Network Profiling**: Network performance profiling
83
- - **System Profiling**: System performance profiling
101
+ 1. **Use specific queries** More precise queries return faster, more relevant results.
102
+ 2. **Apply filters** Narrowing by date, source, or content type reduces the search space.
103
+ 3. **Enable caching** For API integrations, respect cache headers to avoid unnecessary requests.
104
+ 4. **Use batch endpoints** Bundle multiple operations into a single batch request.
105
+ 5. **Monitor your usage** — The Dashboard shows your API usage patterns and suggests optimizations.
@@ -1,83 +1,105 @@
1
- # Search Algorithms
1
+ # Search Algorithms & Ranking
2
2
 
3
- Technical details about Yantra's search algorithms.
3
+ How Yantra finds the most relevant results in milliseconds — the algorithms, ranking signals, and optimizations behind the search experience.
4
+
5
+ ---
4
6
 
5
7
  ## Search Architecture
6
8
 
7
- ### Core Algorithms
9
+ Yantra's search system combines **three complementary approaches** to deliver highly relevant results:
8
10
 
9
- - **Vector Search**: Semantic vector search
10
- - **Keyword Search**: Traditional keyword matching
11
- - **Hybrid Search**: Combined vector and keyword search
12
- - **Ranking Algorithms**: Advanced ranking algorithms
11
+ ### 1. Keyword search (BM25)
13
12
 
14
- ### Search Pipeline
13
+ Traditional lexical matching using the BM25 algorithm. This excels at exact-match queries where the user knows specific terms.
15
14
 
16
- - **Query Processing**: Query analysis and processing
17
- - **Index Lookup**: Search index lookup
18
- - **Result Ranking**: Result ranking and scoring
19
- - **Result Filtering**: Result filtering and refinement
15
+ - **How it works** Documents are scored based on term frequency (TF), inverse document frequency (IDF), and document length normalization.
16
+ - **Best for** Error codes, product names, exact phrases, technical identifiers.
20
17
 
21
- ## Ranking Algorithms
18
+ ### 2. Semantic search (vector similarity)
22
19
 
23
- ### Relevance Scoring
20
+ Neural embedding-based search that understands meaning beyond exact words.
24
21
 
25
- - **TF-IDF**: Term frequency-inverse document frequency
26
- - **BM25**: Best Matching 25 algorithm
27
- - **Semantic Similarity**: Semantic similarity scoring
28
- - **User Behavior**: User behavior-based ranking
22
+ - **How it works** — Both the query and documents are converted to high-dimensional vectors. Results are ranked by cosine similarity.
23
+ - **Best for** Natural language questions, conceptual queries, "how do I..." questions.
29
24
 
30
- ### Ranking Factors
25
+ ### 3. Hybrid search (default)
31
26
 
32
- - **Content Quality**: Content quality assessment
33
- - **Source Authority**: Source authority scoring
34
- - **Recency**: Content recency scoring
35
- - **User Preferences**: Personalized ranking
27
+ Combines keyword and semantic results using **Reciprocal Rank Fusion (RRF)**, producing results that are both lexically precise and semantically relevant.
36
28
 
37
- ## Indexing System
29
+ ```
30
+ RRF_score = sum(1 / (k + rank_i)) for each ranking system i
31
+ ```
38
32
 
39
- ### Search Index
33
+ This is Yantra's default search mode and performs best for most queries.
40
34
 
41
- - **Inverted Index**: Traditional inverted index
42
- - **Vector Index**: Semantic vector index
43
- - **Metadata Index**: Metadata indexing
44
- - **Full-text Index**: Full-text search index
35
+ ---
45
36
 
46
- ### Index Management
37
+ ## Ranking Signals
47
38
 
48
- - **Index Updates**: Real-time index updates
49
- - **Index Optimization**: Index optimization
50
- - **Index Partitioning**: Index partitioning
51
- - **Index Backup**: Index backup and recovery
39
+ After initial retrieval, results are re-ranked using a learned ranking model that weighs multiple signals:
52
40
 
53
- ## Query Processing
41
+ | Signal | Weight | Description |
42
+ | ----------------- | ------ | ---------------------------------------------------- |
43
+ | Text relevance | 35% | BM25 + semantic similarity combined score |
44
+ | Source authority | 20% | Domain reputation, citation count, editorial quality |
45
+ | Content freshness | 15% | Recency of creation or last update |
46
+ | User engagement | 15% | Click-through rate, time-on-page, bookmarks |
47
+ | Personalization | 10% | User's search history, preferred sources, role |
48
+ | Content quality | 5% | Readability score, completeness, structure |
49
+
50
+ ### Boosting and suppression
54
51
 
55
- ### Query Analysis
52
+ - **Recency boost** — Content published in the last 30 days receives a configurable relevance boost.
53
+ - **Source pinning** — Admins can pin trusted sources to always appear in the top results.
54
+ - **Duplicate suppression** — Near-duplicate documents (> 85% similarity) are collapsed, showing only the highest-quality version.
55
+
56
+ ---
57
+
58
+ ## Indexing System
56
59
 
57
- - **Query Parsing**: Natural language query parsing
58
- - **Query Expansion**: Query expansion techniques
59
- - **Query Optimization**: Query optimization
60
- - **Query Caching**: Query result caching
60
+ ### How content gets indexed
61
+
62
+ 1. **Document intake** New or updated content enters the indexing queue via the data pipeline.
63
+ 2. **Analysis** The document is processed through language-specific analyzers (tokenization, stemming, synonym expansion).
64
+ 3. **Embedding** — A vector representation is generated using the configured embedding model.
65
+ 4. **Storage** — The analyzed text, metadata, and vector are written to Elasticsearch shards.
66
+ 5. **Availability** — The document becomes searchable within 1-2 seconds (near real-time).
67
+
68
+ ### Index management
69
+
70
+ - **Shard allocation** — Each index is distributed across multiple shards for parallel query execution.
71
+ - **Replica configuration** — Each shard has at least one replica for fault tolerance and read throughput.
72
+ - **Index lifecycle** — Older indexes are automatically merged, force-merged, or moved to warm storage based on access patterns.
73
+
74
+ ---
75
+
76
+ ## Query Processing
61
77
 
62
- ### Intent Recognition
78
+ ### What happens when you search
63
79
 
64
- - **Intent Classification**: User intent classification
65
- - **Entity Recognition**: Named entity recognition
66
- - **Context Understanding**: Context analysis
67
- - **Query Disambiguation**: Query disambiguation
80
+ 1. **Query parsing** The raw input is tokenized and analyzed using the same pipeline as indexed documents.
81
+ 2. **Query expansion** Synonyms and related terms are added automatically (e.g., "k8s" expands to "kubernetes").
82
+ 3. **Spell correction** Common typos are detected and corrected using edit-distance algorithms.
83
+ 4. **Intent detection** The system classifies whether the user wants a factual answer, a list, a comparison, or a how-to guide.
84
+ 5. **Execution** — The query runs against both keyword and vector indexes in parallel.
85
+ 6. **Fusion + re-ranking** — Results are merged via RRF, then re-ranked by the learned ranking model.
86
+ 7. **Response assembly** — Top results are formatted with highlighted snippets and source metadata.
68
87
 
69
- ## Performance Optimization
88
+ ---
70
89
 
71
- ### Search Performance
90
+ ## Performance
72
91
 
73
- - **Response Time**: Sub-second response times
74
- - **Throughput**: High query throughput
75
- - **Scalability**: Horizontal scaling
76
- - **Caching**: Multi-layer caching
92
+ | Metric | Value |
93
+ | -------------------------- | ----------------------------- |
94
+ | Average search latency | 145 ms |
95
+ | P99 search latency | < 780 ms |
96
+ | Index refresh interval | 1 second |
97
+ | Maximum concurrent queries | 12,000/sec |
98
+ | Index size | 500M+ documents across 8.7 TB |
77
99
 
78
- ### Optimization Techniques
100
+ ### Optimization techniques
79
101
 
80
- - **Index Optimization**: Search index optimization
81
- - **Query Optimization**: Query performance optimization
82
- - **Caching Strategies**: Intelligent caching
83
- - **Load Balancing**: Search load balancing
102
+ - **Query caching** Identical queries are served from Redis cache (5-minute TTL).
103
+ - **Shard routing** Queries are routed to the shard most likely to contain relevant results.
104
+ - **Early termination** If the top results are already confident, remaining shards stop processing.
105
+ - **Parallel execution** Keyword and vector searches run concurrently, not sequentially.
@@ -1,83 +1,107 @@
1
- # Technical Overview
1
+ # Technical Architecture Overview
2
2
 
3
- Technical architecture and implementation details.
3
+ A high-level look at Yantra's system design, technology choices, and engineering principles.
4
+
5
+ ---
4
6
 
5
7
  ## System Architecture
6
8
 
7
- ### Core Components
9
+ Yantra is built as a collection of loosely coupled **microservices** communicating over gRPC and async message queues. This design enables independent scaling, deployment, and fault isolation.
8
10
 
9
- - **Search Engine**: Advanced search processing engine
10
- - **AI Integration**: Multiple AI model integration
11
- - **Data Processing**: Real-time data processing pipeline
12
- - **API Layer**: RESTful API architecture
11
+ ### Core components
13
12
 
14
- ### Infrastructure
13
+ | Component | Responsibility | Technology |
14
+ | -------------------- | ----------------------------------------------------- | -------------------------------- |
15
+ | API Gateway | Request routing, auth, rate limiting | Kong / custom middleware |
16
+ | Search Service | Query processing, ranking, retrieval | Elasticsearch + custom vector DB |
17
+ | AI Service | Model inference, prompt assembly, response generation | Python (FastAPI) + GPU clusters |
18
+ | Data Pipeline | Content ingestion, processing, indexing | Apache Kafka + Kafka Streams |
19
+ | User Service | Authentication, profiles, preferences | Node.js (Express) + PostgreSQL |
20
+ | Notification Service | Real-time alerts, webhooks, emails | Go (Gin) + Redis Pub/Sub |
21
+
22
+ ### Communication patterns
15
23
 
16
- - **Cloud Infrastructure**: Scalable cloud infrastructure
17
- - **Microservices**: Microservices architecture
18
- - **Load Balancing**: Advanced load balancing
19
- - **Caching**: Multi-layer caching system
24
+ - **Synchronous** REST/gRPC for user-facing requests where latency matters.
25
+ - **Asynchronous** Kafka events for background processing (indexing, analytics, notifications).
26
+ - **Pub/Sub** Redis for real-time features (live search updates, collaborative cursors).
27
+
28
+ ---
20
29
 
21
30
  ## Technology Stack
22
31
 
23
- ### Backend Technologies
32
+ ### Backend
24
33
 
25
- - **Programming Languages**: Python, Node.js, Go
26
- - **Frameworks**: FastAPI, Express.js, Gin
27
- - **Databases**: PostgreSQL, Redis, Elasticsearch
28
- - **Message Queues**: Apache Kafka, RabbitMQ
34
+ | Layer | Technologies |
35
+ | ------------- | ----------------------------------------- |
36
+ | Languages | Python 3.12, Node.js 20, Go 1.22 |
37
+ | Frameworks | FastAPI, Express.js, Gin |
38
+ | Databases | PostgreSQL 16, Redis 7, Elasticsearch 8.x |
39
+ | Message Queue | Apache Kafka 3.7 |
40
+ | ML/AI | PyTorch, vLLM, LangChain |
29
41
 
30
- ### Frontend Technologies
42
+ ### Frontend
31
43
 
32
- - **Web Framework**: React, TypeScript
33
- - **Mobile**: React Native, Flutter
34
- - **Build Tools**: Webpack, Vite, Rollup
35
- - **Testing**: Jest, Cypress, Playwright
44
+ | Layer | Technologies |
45
+ | --------- | ---------------------------- |
46
+ | Framework | React 18 + TypeScript 5 |
47
+ | Routing | Remix (SSR) |
48
+ | Styling | Tailwind CSS + shadcn/ui |
49
+ | State | Zustand + React Query |
50
+ | Mobile | React Native (iOS & Android) |
51
+
52
+ ### Infrastructure
53
+
54
+ | Layer | Technologies |
55
+ | ----------------- | ----------------------------------- |
56
+ | Container runtime | Docker + Kubernetes (EKS) |
57
+ | CI/CD | GitHub Actions + ArgoCD |
58
+ | Monitoring | Prometheus, Grafana, Sentry |
59
+ | CDN | CloudFront + edge caching |
60
+ | DNS | Route 53 with latency-based routing |
61
+
62
+ ---
36
63
 
37
64
  ## AI Integration
38
65
 
39
- ### Model Architecture
66
+ ### Multi-model approach
40
67
 
41
- - **GPT Integration**: OpenAI GPT model integration
42
- - **Claude Integration**: Anthropic Claude integration
43
- - **Custom Models**: Proprietary model development
44
- - **Model Selection**: Intelligent model routing
68
+ Rather than being locked into a single AI provider, Yantra maintains integrations with multiple model providers:
45
69
 
46
- ### Processing Pipeline
70
+ - **OpenAI** (GPT-4o, text-embedding-3-large)
71
+ - **Anthropic** (Claude 3.5 Sonnet, Claude 3 Haiku)
72
+ - **Open-source** (Llama 3, Mistral) for on-premise deployments
47
73
 
48
- - **Query Analysis**: Natural language query analysis
49
- - **Intent Recognition**: User intent recognition
50
- - **Context Management**: Conversation context management
51
- - **Response Generation**: AI response generation
74
+ The **Intelligent Router** selects the best model for each request based on task type, latency requirements, and cost constraints.
52
75
 
53
- ## Data Architecture
76
+ ### RAG (Retrieval-Augmented Generation)
54
77
 
55
- ### Data Storage
78
+ Yantra's core AI workflow follows the RAG pattern:
56
79
 
57
- - **Primary Database**: PostgreSQL for structured data
58
- - **Search Index**: Elasticsearch for search functionality
59
- - **Cache Layer**: Redis for high-speed caching
60
- - **File Storage**: S3-compatible object storage
80
+ 1. **Retrieve** relevant documents from the vector index based on the user's query.
81
+ 2. **Augment** the prompt with retrieved context, conversation history, and system instructions.
82
+ 3. **Generate** a grounded response that cites specific sources.
61
83
 
62
- ### Data Processing
84
+ This approach ensures responses are factually anchored to your data rather than relying solely on the model's training data.
63
85
 
64
- - **ETL Pipelines**: Extract, transform, load processes
65
- - **Real-time Processing**: Stream processing with Kafka
66
- - **Data Validation**: Comprehensive data validation
67
- - **Data Quality**: Data quality monitoring
86
+ ---
68
87
 
69
88
  ## Security Architecture
70
89
 
71
- ### Security Measures
90
+ ### Defense in depth
72
91
 
73
- - **Authentication**: Multi-factor authentication
74
- - **Authorization**: Role-based access control
75
- - **Encryption**: End-to-end encryption
76
- - **Network Security**: Network-level security
92
+ | Layer | Measures |
93
+ | ----------- | ---------------------------------------------------------- |
94
+ | Network | VPC isolation, WAF, DDoS protection, private subnets |
95
+ | Application | Input validation, CSRF protection, Content Security Policy |
96
+ | Data | AES-256 encryption at rest, TLS 1.3 in transit |
97
+ | Access | RBAC, MFA, API key rotation, session management |
98
+ | Audit | Immutable audit logs, real-time anomaly detection |
77
99
 
78
100
  ### Compliance
79
101
 
80
- - **GDPR Compliance**: European data protection compliance
81
- - **SOC 2**: Security and compliance certification
82
- - **ISO 27001**: Information security management
83
- - **Privacy by Design**: Privacy-first architecture
102
+ Yantra maintains the following certifications and compliance standards:
103
+
104
+ - **SOC 2 Type II** — Independently audited annually
105
+ - **GDPR** Full compliance with EU data protection regulations
106
+ - **ISO 27001** — Information security management certification
107
+ - **HIPAA** — Available for healthcare customers on Enterprise plans
@@ -1,67 +1,103 @@
1
1
  # Conversation Management
2
2
 
3
- Managing and organizing your conversation threads.
3
+ Tips and tools for keeping your Yantra conversations organized, productive, and easy to find.
4
4
 
5
- ## Thread Management
5
+ ---
6
6
 
7
- ### Creating Conversations
7
+ ## Organizing Conversations
8
8
 
9
- - **New Thread**: Start new conversation thread
10
- - **Topic Definition**: Define conversation topic
11
- - **Context Setting**: Set conversation context
12
- - **Participant Invitation**: Invite conversation participants
9
+ ### Naming threads
13
10
 
14
- ### Organizing Threads
11
+ By default, threads are named after your first query. Rename them for better organization:
15
12
 
16
- - **Topic Grouping**: Group threads by topic
17
- - **Priority Setting**: Set thread priorities
18
- - **Status Tracking**: Track thread status
19
- - **Archive Management**: Archive completed threads
13
+ 1. Click the thread title in the sidebar or at the top of the conversation.
14
+ 2. Type a descriptive name (e.g., "React performance optimization research").
15
+ 3. Press Enter.
20
16
 
21
- ## Conversation Features
17
+ ### Pinning important threads
22
18
 
23
- ### Context Preservation
19
+ Pin threads you reference frequently:
24
20
 
25
- - **History Tracking**: Track conversation history
26
- - **Reference Linking**: Link to previous discussions
27
- - **Context Awareness**: Maintain context across messages
28
- - **Follow-up Management**: Manage follow-up questions
21
+ - Click **⋯** > **Pin** on any thread.
22
+ - Pinned threads stay at the top of the sidebar, regardless of age.
23
+ - Unpin anytime via **⋯** > **Unpin**.
29
24
 
30
- ### Collaboration Tools
25
+ ### Tagging
31
26
 
32
- - **Member Roles**: Assign conversation roles
33
- - **Permission Control**: Control conversation permissions
34
- - **Activity Monitoring**: Monitor conversation activity
35
- - **Communication Tools**: Use built-in communication tools
27
+ Add tags to threads for categorization:
36
28
 
37
- ## Thread Organization
29
+ - Click the tag icon in the thread header.
30
+ - Add tags like "work," "personal," "research," "client: Acme Corp."
31
+ - Filter the thread list by tag.
38
32
 
39
- ### Structure Management
33
+ ---
40
34
 
41
- - **Hierarchical Organization**: Organize threads hierarchically
42
- - **Category System**: Use category system
43
- - **Tagging System**: Tag conversations for easy finding
44
- - **Search Functionality**: Search within conversations
35
+ ## Thread Actions
45
36
 
46
- ### Content Management
37
+ | Action | Shortcut (Mac) | Shortcut (Windows) | Description |
38
+ | -------------- | -------------- | ------------------ | ----------------------------------------- |
39
+ | New thread | ⌘ + N | Ctrl + N | Start a fresh conversation |
40
+ | Search threads | ⌘ + F | Ctrl + F | Find threads by keyword |
41
+ | Pin/unpin | — | — | Keep threads accessible |
42
+ | Archive | — | — | Move to archive (not deleted) |
43
+ | Delete | — | — | Permanently remove (30-day recovery) |
44
+ | Export | — | — | Download as Markdown, PDF, or text |
45
+ | Branch | — | — | Fork the conversation at a specific point |
47
46
 
48
- - **Message Organization**: Organize messages within threads
49
- - **Attachment Management**: Manage file attachments
50
- - **Link Management**: Manage external links
51
- - **Citation Tracking**: Track source citations
47
+ ---
52
48
 
53
- ## Advanced Features
49
+ ## Branching Conversations
54
50
 
55
- ### Thread Analytics
51
+ When a conversation takes you in an unexpected direction, branch it instead of cluttering the original thread:
56
52
 
57
- - **Participation Metrics**: Track participation metrics
58
- - **Activity Analysis**: Analyze conversation activity
59
- - **Engagement Tracking**: Track engagement levels
60
- - **Productivity Metrics**: Measure conversation productivity
53
+ 1. Hover over any message in the thread.
54
+ 2. Click the **Branch** icon.
55
+ 3. A new thread is created with the conversation history up to that point.
56
+ 4. Continue the original thread and the branch independently.
61
57
 
62
- ### Integration Features
58
+ This is especially useful for:
63
59
 
64
- - **External Tools**: Integrate with external tools
65
- - **API Access**: Programmatic access to threads
66
- - **Export Options**: Export conversation data
67
- - **Sync Capabilities**: Sync across devices
60
+ - Exploring "what if" scenarios
61
+ - Separating sub-topics into their own threads
62
+ - Trying different approaches without losing the original context
63
+
64
+ ---
65
+
66
+ ## Search and Filter
67
+
68
+ ### Full-text search
69
+
70
+ Search across all your conversations:
71
+
72
+ - Search by **keywords** in your messages or Yantra's responses.
73
+ - Search by **date range** to find conversations from a specific period.
74
+ - Search by **search mode** (Regular, Pro, Deep Research).
75
+
76
+ ### Filters
77
+
78
+ | Filter | Options |
79
+ | ------ | ------------------------------------------ |
80
+ | Date | Today, this week, this month, custom range |
81
+ | Mode | Regular, Pro Search, Deep Research |
82
+ | Tags | Your custom tags |
83
+ | Shared | Only shared threads, only private |
84
+
85
+ ---
86
+
87
+ ## Auto-Cleanup
88
+
89
+ Configure automatic thread management:
90
+
91
+ - **Auto-archive** — Threads with no activity for X days are archived (default: 90 days).
92
+ - **Auto-delete** — Archived threads are deleted after X days (default: never).
93
+ - **Configurable** — Adjust at **Settings > Preferences > Threads**.
94
+
95
+ ---
96
+
97
+ ## Best Practices
98
+
99
+ 1. **One topic per thread** — Keeps context clean and makes threads easier to find later.
100
+ 2. **Name your threads** — A good name saves time when searching later.
101
+ 3. **Use branches** — Don't let a tangent derail a focused research thread.
102
+ 4. **Pin active projects** — Keep your current focus areas at the top.
103
+ 5. **Archive regularly** — A clean sidebar helps you focus on what matters now.