@adminide-stack/yantra-help-browser 12.0.16-alpha.27 → 12.0.16-alpha.29
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/lib/components/HelpCenterFooter.d.ts.map +1 -1
- package/lib/components/HelpCenterFooter.js +43 -88
- package/lib/components/HelpCenterFooter.js.map +1 -1
- package/lib/components/HelpCenterHeader.d.ts.map +1 -1
- package/lib/components/HelpCenterHeader.js +3 -8
- package/lib/components/HelpCenterHeader.js.map +1 -1
- package/lib/components/Icons.d.ts +55 -0
- package/lib/components/Icons.d.ts.map +1 -0
- package/lib/{pages/LandingPage/components → components}/Icons.js +39 -2
- package/lib/components/Icons.js.map +1 -0
- package/lib/components/Logo.d.ts +1 -2
- package/lib/components/Logo.d.ts.map +1 -1
- package/lib/components/Logo.js +2 -9
- package/lib/components/Logo.js.map +1 -1
- package/lib/components/PageHero.d.ts +9 -0
- package/lib/components/PageHero.d.ts.map +1 -0
- package/lib/components/PageHero.js +47 -0
- package/lib/components/PageHero.js.map +1 -0
- package/lib/components/SearchBar.d.ts.map +1 -1
- package/lib/components/SearchBar.js +9 -45
- package/lib/components/SearchBar.js.map +1 -1
- package/lib/components/SidebarSearch.js +2 -2
- package/lib/components/SidebarSearch.js.map +1 -1
- package/lib/components/navbar/index.d.ts.map +1 -1
- package/lib/components/navbar/index.js +5 -16
- package/lib/components/navbar/index.js.map +1 -1
- package/lib/compute.d.ts +15 -0
- package/lib/compute.d.ts.map +1 -1
- package/lib/compute.js +74 -2
- package/lib/compute.js.map +1 -1
- package/lib/pages/About/index.d.ts +3 -0
- package/lib/pages/About/index.d.ts.map +1 -0
- package/lib/pages/About/index.js +69 -0
- package/lib/pages/About/index.js.map +1 -0
- package/lib/pages/Careers/index.d.ts +3 -0
- package/lib/pages/Careers/index.d.ts.map +1 -0
- package/lib/pages/Careers/index.js +78 -0
- package/lib/pages/Careers/index.js.map +1 -0
- package/lib/pages/CategoryCollection/index.d.ts.map +1 -1
- package/lib/pages/CategoryCollection/index.js +15 -78
- package/lib/pages/CategoryCollection/index.js.map +1 -1
- package/lib/pages/Community/index.d.ts +3 -0
- package/lib/pages/Community/index.d.ts.map +1 -0
- package/lib/pages/Community/index.js +72 -0
- package/lib/pages/Community/index.js.map +1 -0
- package/lib/pages/Contact/index.d.ts +3 -0
- package/lib/pages/Contact/index.d.ts.map +1 -0
- package/lib/pages/Contact/index.js +128 -0
- package/lib/pages/Contact/index.js.map +1 -0
- package/lib/pages/GetStarted/components/ExampleCard.d.ts.map +1 -1
- package/lib/pages/GetStarted/index.d.ts.map +1 -1
- package/lib/pages/GetStarted/index.js +9 -45
- package/lib/pages/GetStarted/index.js.map +1 -1
- package/lib/pages/HelpCenter/components/HelpCategoryCard.d.ts.map +1 -1
- package/lib/pages/HelpCenter/components/HelpCategoryCard.js +7 -34
- package/lib/pages/HelpCenter/components/HelpCategoryCard.js.map +1 -1
- package/lib/pages/HelpCenter/components/PopularArticle.d.ts.map +1 -1
- package/lib/pages/HelpCenter/components/PopularArticle.js +3 -12
- package/lib/pages/HelpCenter/components/PopularArticle.js.map +1 -1
- package/lib/pages/HelpCenter/index.d.ts.map +1 -1
- package/lib/pages/HelpCenter/index.js +15 -74
- package/lib/pages/HelpCenter/index.js.map +1 -1
- package/lib/pages/LandingPage/components/ArticleCard.d.ts.map +1 -1
- package/lib/pages/LandingPage/components/ArticleCard.js +3 -12
- package/lib/pages/LandingPage/components/ArticleCard.js.map +1 -1
- package/lib/pages/LandingPage/components/CategoryCard.d.ts.map +1 -1
- package/lib/pages/LandingPage/components/CategoryCard.js +3 -12
- package/lib/pages/LandingPage/components/CategoryCard.js.map +1 -1
- package/lib/pages/LandingPage/components/HeroSection.d.ts.map +1 -1
- package/lib/pages/LandingPage/components/HeroSection.js +3 -8
- package/lib/pages/LandingPage/components/HeroSection.js.map +1 -1
- package/lib/pages/LandingPage/components/ResourceCard.d.ts.map +1 -1
- package/lib/pages/LandingPage/components/ResourceCard.js +3 -12
- package/lib/pages/LandingPage/components/ResourceCard.js.map +1 -1
- package/lib/pages/LandingPage/index.d.ts.map +1 -1
- package/lib/pages/LandingPage/index.js +62 -33
- package/lib/pages/LandingPage/index.js.map +1 -1
- package/lib/pages/Markdown/MarkdownPageLayout.d.ts.map +1 -1
- package/lib/pages/Markdown/MarkdownPageLayout.js +11 -56
- package/lib/pages/Markdown/MarkdownPageLayout.js.map +1 -1
- package/lib/pages/Privacy/index.d.ts +3 -0
- package/lib/pages/Privacy/index.d.ts.map +1 -0
- package/lib/pages/Privacy/index.js +449 -0
- package/lib/pages/Privacy/index.js.map +1 -0
- package/lib/pages/ReleaseNotes/index.d.ts +3 -0
- package/lib/pages/ReleaseNotes/index.d.ts.map +1 -0
- package/lib/pages/ReleaseNotes/index.js +61 -0
- package/lib/pages/ReleaseNotes/index.js.map +1 -0
- package/lib/pages/StatusPage/index.d.ts +3 -0
- package/lib/pages/StatusPage/index.d.ts.map +1 -0
- package/lib/pages/StatusPage/index.js +85 -0
- package/lib/pages/StatusPage/index.js.map +1 -0
- package/lib/pages/Terms/index.d.ts +3 -0
- package/lib/pages/Terms/index.d.ts.map +1 -0
- package/lib/pages/Terms/index.js +60 -0
- package/lib/pages/Terms/index.js.map +1 -0
- package/lib/routes.json +88 -0
- package/lib/templates/content/account-management/account-setup.md +56 -57
- package/lib/templates/content/account-management/delete-account.md +55 -58
- package/lib/templates/content/account-management/preferences.md +70 -55
- package/lib/templates/content/account-management/privacy-settings.md +67 -56
- package/lib/templates/content/account-management/profile-settings.md +56 -45
- package/lib/templates/content/browser-extension/browser-extension-overview.md +60 -0
- package/lib/templates/content/browser-extension/extension-security.md +126 -0
- package/lib/templates/content/browser-extension/getting-started-extension.md +96 -0
- package/lib/templates/content/browser-extension/how-it-works.md +112 -0
- package/lib/templates/content/browser-extension/troubleshooting-extension.md +138 -0
- package/lib/templates/content/browser-extension/use-cases-workflows.md +123 -0
- package/lib/templates/content/content-manifest.json +328 -0
- package/lib/templates/content/data-privacy/data-collection.md +55 -51
- package/lib/templates/content/data-privacy/privacy-policy.md +74 -57
- package/lib/templates/content/data-subject-privacy/data-access.md +59 -69
- package/lib/templates/content/data-subject-privacy/data-portability.md +67 -93
- package/lib/templates/content/data-subject-privacy/privacy-requests.md +66 -62
- package/lib/templates/content/file-uploads/file-upload-overview.md +69 -50
- package/lib/templates/content/getting-started/getting-started-guide.md +65 -51
- package/lib/templates/content/product-features/ai-models.md +62 -57
- package/lib/templates/content/product-features/collaboration-tools.md +66 -45
- package/lib/templates/content/product-features/conversation-features.md +59 -34
- package/lib/templates/content/product-features/export-features.md +65 -45
- package/lib/templates/content/product-features/follow-up-questions.md +61 -46
- package/lib/templates/content/product-features/real-time-search.md +63 -46
- package/lib/templates/content/product-features/saved-searches.md +53 -45
- package/lib/templates/content/product-features/search-features.md +61 -43
- package/lib/templates/content/product-features/search-history.md +64 -45
- package/lib/templates/content/product-features/source-citations.md +77 -44
- package/lib/templates/content/scope-api/api-overview.md +83 -46
- package/lib/templates/content/search-modes/deep-research.md +87 -46
- package/lib/templates/content/search-modes/labs-features.md +54 -56
- package/lib/templates/content/search-modes/pro-search.md +53 -45
- package/lib/templates/content/search-modes/regular-search.md +64 -46
- package/lib/templates/content/spaces-library/spaces-overview.md +68 -47
- package/lib/templates/content/student-hub/academic-research.md +85 -48
- package/lib/templates/content/student-hub/student-discounts.md +51 -57
- package/lib/templates/content/student-hub/student-overview.md +64 -45
- package/lib/templates/content/student-hub/study-tools.md +99 -45
- package/lib/templates/content/subscription-billing/billing-cycle.md +50 -48
- package/lib/templates/content/subscription-billing/billing-overview.md +55 -34
- package/lib/templates/content/subscription-billing/billing-support.md +52 -45
- package/lib/templates/content/subscription-billing/currency-support.md +36 -62
- package/lib/templates/content/subscription-billing/enterprise-pricing.md +52 -46
- package/lib/templates/content/subscription-billing/invoice-management.md +64 -45
- package/lib/templates/content/subscription-billing/payment-methods.md +53 -52
- package/lib/templates/content/subscription-billing/promotional-offers.md +58 -46
- package/lib/templates/content/subscription-billing/refund-policy.md +53 -58
- package/lib/templates/content/subscription-billing/student-discounts.md +61 -57
- package/lib/templates/content/subscription-billing/tax-information.md +64 -45
- package/lib/templates/content/technical-questions/ai-models-technical.md +60 -56
- package/lib/templates/content/technical-questions/api-technical.md +131 -56
- package/lib/templates/content/technical-questions/data-processing.md +74 -55
- package/lib/templates/content/technical-questions/database-architecture.md +91 -54
- package/lib/templates/content/technical-questions/infrastructure.md +104 -55
- package/lib/templates/content/technical-questions/performance-optimization.md +77 -55
- package/lib/templates/content/technical-questions/search-algorithms.md +78 -56
- package/lib/templates/content/technical-questions/technical-overview.md +76 -52
- package/lib/templates/content/threads/conversation-management.md +81 -45
- package/lib/templates/content/threads/threads-overview.md +69 -45
- package/lib/templates/content/troubleshooting/common-issues.md +91 -43
- package/lib/templates/content/what-is-yantra/getting-started-yantra.md +58 -57
- package/package.json +2 -2
- package/lib/pages/LandingPage/components/Icons.d.ts +0 -13
- package/lib/pages/LandingPage/components/Icons.d.ts.map +0 -1
- package/lib/pages/LandingPage/components/Icons.js.map +0 -1
|
@@ -1,83 +1,105 @@
|
|
|
1
1
|
# Performance Optimization
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
How Yantra achieves sub-second response times at scale — and how you can get the most out of the platform.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
---
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## System Performance Targets
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
9
|
+
| Metric | Target | Current |
|
|
10
|
+
| ------------------------- | -------------- | -------------- |
|
|
11
|
+
| Search latency (P50) | < 200 ms | 145 ms |
|
|
12
|
+
| Search latency (P99) | < 1 s | 780 ms |
|
|
13
|
+
| AI response latency (P50) | < 800 ms | 620 ms |
|
|
14
|
+
| API uptime | 99.95% | 99.98% |
|
|
15
|
+
| Throughput | 10,000 req/min | 14,200 req/min |
|
|
13
16
|
|
|
14
|
-
|
|
17
|
+
---
|
|
15
18
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
- **
|
|
19
|
-
|
|
19
|
+
## Caching Strategy
|
|
20
|
+
|
|
21
|
+
Yantra uses a **multi-layer caching architecture** to minimize redundant computation:
|
|
22
|
+
|
|
23
|
+
### Layer 1 — Edge cache (CDN)
|
|
24
|
+
|
|
25
|
+
Static assets and frequently requested API responses are cached at 200+ edge locations worldwide. Cache hit rates typically exceed 85%.
|
|
26
|
+
|
|
27
|
+
### Layer 2 — Application cache (Redis)
|
|
28
|
+
|
|
29
|
+
- **Query result cache** — Identical queries return cached results within 5 ms.
|
|
30
|
+
- **Session cache** — User sessions and preferences are stored in Redis for instant access.
|
|
31
|
+
- **Embedding cache** — Pre-computed embeddings for popular documents avoid redundant inference.
|
|
32
|
+
|
|
33
|
+
### Layer 3 — Database cache
|
|
34
|
+
|
|
35
|
+
- **Connection pooling** — PgBouncer maintains a pool of warm database connections, eliminating connection overhead.
|
|
36
|
+
- **Materialized views** — Complex aggregation queries use pre-computed materialized views refreshed every 5 minutes.
|
|
37
|
+
- **Query plan caching** — PostgreSQL's prepared statements reuse optimized query plans.
|
|
38
|
+
|
|
39
|
+
---
|
|
20
40
|
|
|
21
41
|
## Database Optimization
|
|
22
42
|
|
|
23
|
-
###
|
|
43
|
+
### Indexing strategy
|
|
24
44
|
|
|
25
|
-
- **
|
|
26
|
-
- **
|
|
27
|
-
- **
|
|
28
|
-
- **
|
|
45
|
+
- **B-tree indexes** on primary keys and frequently filtered columns.
|
|
46
|
+
- **GIN indexes** on JSONB fields for flexible metadata queries.
|
|
47
|
+
- **Partial indexes** on hot data subsets (e.g., active users, recent content).
|
|
48
|
+
- **Covering indexes** that include all queried columns to avoid table lookups.
|
|
29
49
|
|
|
30
|
-
###
|
|
50
|
+
### Query optimization
|
|
31
51
|
|
|
32
|
-
|
|
33
|
-
- **Compression**: Data compression techniques
|
|
34
|
-
- **Archiving**: Data archiving strategies
|
|
35
|
-
- **Cleanup**: Automated data cleanup
|
|
52
|
+
Every query is analyzed for:
|
|
36
53
|
|
|
37
|
-
|
|
54
|
+
1. **Execution plan review** — EXPLAIN ANALYZE is run on all new queries during code review.
|
|
55
|
+
2. **N+1 detection** — Automated detection of N+1 query patterns in development.
|
|
56
|
+
3. **Slow query logging** — Queries exceeding 100 ms are logged and reviewed weekly.
|
|
57
|
+
4. **Connection management** — Read replicas handle analytics queries to keep the primary database responsive.
|
|
38
58
|
|
|
39
|
-
|
|
59
|
+
---
|
|
40
60
|
|
|
41
|
-
-
|
|
42
|
-
- **Memory Management**: Memory optimization
|
|
43
|
-
- **CPU Optimization**: CPU usage optimization
|
|
44
|
-
- **I/O Optimization**: Input/output optimization
|
|
61
|
+
## Application-Level Optimization
|
|
45
62
|
|
|
46
|
-
###
|
|
63
|
+
### Frontend performance
|
|
47
64
|
|
|
48
|
-
- **
|
|
49
|
-
- **
|
|
50
|
-
- **
|
|
51
|
-
- **
|
|
65
|
+
- **Code splitting** — Routes are lazy-loaded so users only download the JavaScript they need.
|
|
66
|
+
- **Image optimization** — All images are served in WebP/AVIF format with responsive srcsets.
|
|
67
|
+
- **Prefetching** — The most likely next navigation target is prefetched in the background.
|
|
68
|
+
- **Bundle analysis** — We maintain strict bundle size budgets (< 200 KB initial JS, < 50 KB CSS).
|
|
52
69
|
|
|
53
|
-
|
|
70
|
+
### Backend performance
|
|
54
71
|
|
|
55
|
-
|
|
72
|
+
- **Async I/O** — All I/O-bound operations (database queries, API calls, file reads) are non-blocking.
|
|
73
|
+
- **Worker pools** — CPU-intensive tasks (embedding generation, document parsing) run in dedicated worker processes.
|
|
74
|
+
- **Streaming responses** — AI-generated responses stream token-by-token for perceived instant feedback.
|
|
75
|
+
- **Graceful degradation** — If a dependency is slow, the system returns partial results rather than timing out entirely.
|
|
56
76
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
- **Storage Optimization**: Storage performance
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## Infrastructure Optimization
|
|
61
80
|
|
|
62
|
-
###
|
|
81
|
+
### Auto-scaling
|
|
63
82
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
83
|
+
| Component | Scaling metric | Scale range |
|
|
84
|
+
| ---------------------- | ----------------------------- | ------------------ |
|
|
85
|
+
| API servers | Request rate + latency | 4 to 64 pods |
|
|
86
|
+
| AI workers | GPU utilization + queue depth | 2 to 16 nodes |
|
|
87
|
+
| Kafka consumers | Consumer lag | 3 to 24 partitions |
|
|
88
|
+
| Database read replicas | Connection count | 1 to 4 replicas |
|
|
68
89
|
|
|
69
|
-
|
|
90
|
+
### Cost optimization
|
|
70
91
|
|
|
71
|
-
|
|
92
|
+
- **Spot instances** for batch processing workloads (60% cost savings).
|
|
93
|
+
- **Reserved instances** for always-on services (40% cost savings).
|
|
94
|
+
- **Right-sizing** — Monthly reviews of resource utilization with automated recommendations.
|
|
95
|
+
- **Tiered storage** — Infrequently accessed data is automatically moved to cheaper storage tiers.
|
|
72
96
|
|
|
73
|
-
|
|
74
|
-
- **Metrics Collection**: Performance metrics collection
|
|
75
|
-
- **Alerting**: Performance alerting
|
|
76
|
-
- **Dashboards**: Performance dashboards
|
|
97
|
+
---
|
|
77
98
|
|
|
78
|
-
|
|
99
|
+
## Tips for Optimizing Your Experience
|
|
79
100
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
101
|
+
1. **Use specific queries** — More precise queries return faster, more relevant results.
|
|
102
|
+
2. **Apply filters** — Narrowing by date, source, or content type reduces the search space.
|
|
103
|
+
3. **Enable caching** — For API integrations, respect cache headers to avoid unnecessary requests.
|
|
104
|
+
4. **Use batch endpoints** — Bundle multiple operations into a single batch request.
|
|
105
|
+
5. **Monitor your usage** — The Dashboard shows your API usage patterns and suggests optimizations.
|
|
@@ -1,83 +1,105 @@
|
|
|
1
|
-
# Search Algorithms
|
|
1
|
+
# Search Algorithms & Ranking
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
How Yantra finds the most relevant results in milliseconds — the algorithms, ranking signals, and optimizations behind the search experience.
|
|
4
|
+
|
|
5
|
+
---
|
|
4
6
|
|
|
5
7
|
## Search Architecture
|
|
6
8
|
|
|
7
|
-
|
|
9
|
+
Yantra's search system combines **three complementary approaches** to deliver highly relevant results:
|
|
8
10
|
|
|
9
|
-
|
|
10
|
-
- **Keyword Search**: Traditional keyword matching
|
|
11
|
-
- **Hybrid Search**: Combined vector and keyword search
|
|
12
|
-
- **Ranking Algorithms**: Advanced ranking algorithms
|
|
11
|
+
### 1. Keyword search (BM25)
|
|
13
12
|
|
|
14
|
-
|
|
13
|
+
Traditional lexical matching using the BM25 algorithm. This excels at exact-match queries where the user knows specific terms.
|
|
15
14
|
|
|
16
|
-
- **
|
|
17
|
-
- **
|
|
18
|
-
- **Result Ranking**: Result ranking and scoring
|
|
19
|
-
- **Result Filtering**: Result filtering and refinement
|
|
15
|
+
- **How it works** — Documents are scored based on term frequency (TF), inverse document frequency (IDF), and document length normalization.
|
|
16
|
+
- **Best for** — Error codes, product names, exact phrases, technical identifiers.
|
|
20
17
|
|
|
21
|
-
|
|
18
|
+
### 2. Semantic search (vector similarity)
|
|
22
19
|
|
|
23
|
-
|
|
20
|
+
Neural embedding-based search that understands meaning beyond exact words.
|
|
24
21
|
|
|
25
|
-
- **
|
|
26
|
-
- **
|
|
27
|
-
- **Semantic Similarity**: Semantic similarity scoring
|
|
28
|
-
- **User Behavior**: User behavior-based ranking
|
|
22
|
+
- **How it works** — Both the query and documents are converted to high-dimensional vectors. Results are ranked by cosine similarity.
|
|
23
|
+
- **Best for** — Natural language questions, conceptual queries, "how do I..." questions.
|
|
29
24
|
|
|
30
|
-
###
|
|
25
|
+
### 3. Hybrid search (default)
|
|
31
26
|
|
|
32
|
-
|
|
33
|
-
- **Source Authority**: Source authority scoring
|
|
34
|
-
- **Recency**: Content recency scoring
|
|
35
|
-
- **User Preferences**: Personalized ranking
|
|
27
|
+
Combines keyword and semantic results using **Reciprocal Rank Fusion (RRF)**, producing results that are both lexically precise and semantically relevant.
|
|
36
28
|
|
|
37
|
-
|
|
29
|
+
```
|
|
30
|
+
RRF_score = sum(1 / (k + rank_i)) for each ranking system i
|
|
31
|
+
```
|
|
38
32
|
|
|
39
|
-
|
|
33
|
+
This is Yantra's default search mode and performs best for most queries.
|
|
40
34
|
|
|
41
|
-
|
|
42
|
-
- **Vector Index**: Semantic vector index
|
|
43
|
-
- **Metadata Index**: Metadata indexing
|
|
44
|
-
- **Full-text Index**: Full-text search index
|
|
35
|
+
---
|
|
45
36
|
|
|
46
|
-
|
|
37
|
+
## Ranking Signals
|
|
47
38
|
|
|
48
|
-
|
|
49
|
-
- **Index Optimization**: Index optimization
|
|
50
|
-
- **Index Partitioning**: Index partitioning
|
|
51
|
-
- **Index Backup**: Index backup and recovery
|
|
39
|
+
After initial retrieval, results are re-ranked using a learned ranking model that weighs multiple signals:
|
|
52
40
|
|
|
53
|
-
|
|
41
|
+
| Signal | Weight | Description |
|
|
42
|
+
| ----------------- | ------ | ---------------------------------------------------- |
|
|
43
|
+
| Text relevance | 35% | BM25 + semantic similarity combined score |
|
|
44
|
+
| Source authority | 20% | Domain reputation, citation count, editorial quality |
|
|
45
|
+
| Content freshness | 15% | Recency of creation or last update |
|
|
46
|
+
| User engagement | 15% | Click-through rate, time-on-page, bookmarks |
|
|
47
|
+
| Personalization | 10% | User's search history, preferred sources, role |
|
|
48
|
+
| Content quality | 5% | Readability score, completeness, structure |
|
|
49
|
+
|
|
50
|
+
### Boosting and suppression
|
|
54
51
|
|
|
55
|
-
|
|
52
|
+
- **Recency boost** — Content published in the last 30 days receives a configurable relevance boost.
|
|
53
|
+
- **Source pinning** — Admins can pin trusted sources to always appear in the top results.
|
|
54
|
+
- **Duplicate suppression** — Near-duplicate documents (> 85% similarity) are collapsed, showing only the highest-quality version.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Indexing System
|
|
56
59
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
60
|
+
### How content gets indexed
|
|
61
|
+
|
|
62
|
+
1. **Document intake** — New or updated content enters the indexing queue via the data pipeline.
|
|
63
|
+
2. **Analysis** — The document is processed through language-specific analyzers (tokenization, stemming, synonym expansion).
|
|
64
|
+
3. **Embedding** — A vector representation is generated using the configured embedding model.
|
|
65
|
+
4. **Storage** — The analyzed text, metadata, and vector are written to Elasticsearch shards.
|
|
66
|
+
5. **Availability** — The document becomes searchable within 1-2 seconds (near real-time).
|
|
67
|
+
|
|
68
|
+
### Index management
|
|
69
|
+
|
|
70
|
+
- **Shard allocation** — Each index is distributed across multiple shards for parallel query execution.
|
|
71
|
+
- **Replica configuration** — Each shard has at least one replica for fault tolerance and read throughput.
|
|
72
|
+
- **Index lifecycle** — Older indexes are automatically merged, force-merged, or moved to warm storage based on access patterns.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Query Processing
|
|
61
77
|
|
|
62
|
-
###
|
|
78
|
+
### What happens when you search
|
|
63
79
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
80
|
+
1. **Query parsing** — The raw input is tokenized and analyzed using the same pipeline as indexed documents.
|
|
81
|
+
2. **Query expansion** — Synonyms and related terms are added automatically (e.g., "k8s" expands to "kubernetes").
|
|
82
|
+
3. **Spell correction** — Common typos are detected and corrected using edit-distance algorithms.
|
|
83
|
+
4. **Intent detection** — The system classifies whether the user wants a factual answer, a list, a comparison, or a how-to guide.
|
|
84
|
+
5. **Execution** — The query runs against both keyword and vector indexes in parallel.
|
|
85
|
+
6. **Fusion + re-ranking** — Results are merged via RRF, then re-ranked by the learned ranking model.
|
|
86
|
+
7. **Response assembly** — Top results are formatted with highlighted snippets and source metadata.
|
|
68
87
|
|
|
69
|
-
|
|
88
|
+
---
|
|
70
89
|
|
|
71
|
-
|
|
90
|
+
## Performance
|
|
72
91
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
92
|
+
| Metric | Value |
|
|
93
|
+
| -------------------------- | ----------------------------- |
|
|
94
|
+
| Average search latency | 145 ms |
|
|
95
|
+
| P99 search latency | < 780 ms |
|
|
96
|
+
| Index refresh interval | 1 second |
|
|
97
|
+
| Maximum concurrent queries | 12,000/sec |
|
|
98
|
+
| Index size | 500M+ documents across 8.7 TB |
|
|
77
99
|
|
|
78
|
-
### Optimization
|
|
100
|
+
### Optimization techniques
|
|
79
101
|
|
|
80
|
-
- **
|
|
81
|
-
- **
|
|
82
|
-
- **
|
|
83
|
-
- **
|
|
102
|
+
- **Query caching** — Identical queries are served from Redis cache (5-minute TTL).
|
|
103
|
+
- **Shard routing** — Queries are routed to the shard most likely to contain relevant results.
|
|
104
|
+
- **Early termination** — If the top results are already confident, remaining shards stop processing.
|
|
105
|
+
- **Parallel execution** — Keyword and vector searches run concurrently, not sequentially.
|
|
@@ -1,83 +1,107 @@
|
|
|
1
|
-
# Technical Overview
|
|
1
|
+
# Technical Architecture Overview
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A high-level look at Yantra's system design, technology choices, and engineering principles.
|
|
4
|
+
|
|
5
|
+
---
|
|
4
6
|
|
|
5
7
|
## System Architecture
|
|
6
8
|
|
|
7
|
-
|
|
9
|
+
Yantra is built as a collection of loosely coupled **microservices** communicating over gRPC and async message queues. This design enables independent scaling, deployment, and fault isolation.
|
|
8
10
|
|
|
9
|
-
|
|
10
|
-
- **AI Integration**: Multiple AI model integration
|
|
11
|
-
- **Data Processing**: Real-time data processing pipeline
|
|
12
|
-
- **API Layer**: RESTful API architecture
|
|
11
|
+
### Core components
|
|
13
12
|
|
|
14
|
-
|
|
13
|
+
| Component | Responsibility | Technology |
|
|
14
|
+
| -------------------- | ----------------------------------------------------- | -------------------------------- |
|
|
15
|
+
| API Gateway | Request routing, auth, rate limiting | Kong / custom middleware |
|
|
16
|
+
| Search Service | Query processing, ranking, retrieval | Elasticsearch + custom vector DB |
|
|
17
|
+
| AI Service | Model inference, prompt assembly, response generation | Python (FastAPI) + GPU clusters |
|
|
18
|
+
| Data Pipeline | Content ingestion, processing, indexing | Apache Kafka + Kafka Streams |
|
|
19
|
+
| User Service | Authentication, profiles, preferences | Node.js (Express) + PostgreSQL |
|
|
20
|
+
| Notification Service | Real-time alerts, webhooks, emails | Go (Gin) + Redis Pub/Sub |
|
|
21
|
+
|
|
22
|
+
### Communication patterns
|
|
15
23
|
|
|
16
|
-
- **
|
|
17
|
-
- **
|
|
18
|
-
- **
|
|
19
|
-
|
|
24
|
+
- **Synchronous** — REST/gRPC for user-facing requests where latency matters.
|
|
25
|
+
- **Asynchronous** — Kafka events for background processing (indexing, analytics, notifications).
|
|
26
|
+
- **Pub/Sub** — Redis for real-time features (live search updates, collaborative cursors).
|
|
27
|
+
|
|
28
|
+
---
|
|
20
29
|
|
|
21
30
|
## Technology Stack
|
|
22
31
|
|
|
23
|
-
### Backend
|
|
32
|
+
### Backend
|
|
24
33
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
34
|
+
| Layer | Technologies |
|
|
35
|
+
| ------------- | ----------------------------------------- |
|
|
36
|
+
| Languages | Python 3.12, Node.js 20, Go 1.22 |
|
|
37
|
+
| Frameworks | FastAPI, Express.js, Gin |
|
|
38
|
+
| Databases | PostgreSQL 16, Redis 7, Elasticsearch 8.x |
|
|
39
|
+
| Message Queue | Apache Kafka 3.7 |
|
|
40
|
+
| ML/AI | PyTorch, vLLM, LangChain |
|
|
29
41
|
|
|
30
|
-
### Frontend
|
|
42
|
+
### Frontend
|
|
31
43
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
44
|
+
| Layer | Technologies |
|
|
45
|
+
| --------- | ---------------------------- |
|
|
46
|
+
| Framework | React 18 + TypeScript 5 |
|
|
47
|
+
| Routing | Remix (SSR) |
|
|
48
|
+
| Styling | Tailwind CSS + shadcn/ui |
|
|
49
|
+
| State | Zustand + React Query |
|
|
50
|
+
| Mobile | React Native (iOS & Android) |
|
|
51
|
+
|
|
52
|
+
### Infrastructure
|
|
53
|
+
|
|
54
|
+
| Layer | Technologies |
|
|
55
|
+
| ----------------- | ----------------------------------- |
|
|
56
|
+
| Container runtime | Docker + Kubernetes (EKS) |
|
|
57
|
+
| CI/CD | GitHub Actions + ArgoCD |
|
|
58
|
+
| Monitoring | Prometheus, Grafana, Sentry |
|
|
59
|
+
| CDN | CloudFront + edge caching |
|
|
60
|
+
| DNS | Route 53 with latency-based routing |
|
|
61
|
+
|
|
62
|
+
---
|
|
36
63
|
|
|
37
64
|
## AI Integration
|
|
38
65
|
|
|
39
|
-
###
|
|
66
|
+
### Multi-model approach
|
|
40
67
|
|
|
41
|
-
|
|
42
|
-
- **Claude Integration**: Anthropic Claude integration
|
|
43
|
-
- **Custom Models**: Proprietary model development
|
|
44
|
-
- **Model Selection**: Intelligent model routing
|
|
68
|
+
Rather than being locked into a single AI provider, Yantra maintains integrations with multiple model providers:
|
|
45
69
|
|
|
46
|
-
|
|
70
|
+
- **OpenAI** (GPT-4o, text-embedding-3-large)
|
|
71
|
+
- **Anthropic** (Claude 3.5 Sonnet, Claude 3 Haiku)
|
|
72
|
+
- **Open-source** (Llama 3, Mistral) for on-premise deployments
|
|
47
73
|
|
|
48
|
-
|
|
49
|
-
- **Intent Recognition**: User intent recognition
|
|
50
|
-
- **Context Management**: Conversation context management
|
|
51
|
-
- **Response Generation**: AI response generation
|
|
74
|
+
The **Intelligent Router** selects the best model for each request based on task type, latency requirements, and cost constraints.
|
|
52
75
|
|
|
53
|
-
|
|
76
|
+
### RAG (Retrieval-Augmented Generation)
|
|
54
77
|
|
|
55
|
-
|
|
78
|
+
Yantra's core AI workflow follows the RAG pattern:
|
|
56
79
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
- **File Storage**: S3-compatible object storage
|
|
80
|
+
1. **Retrieve** relevant documents from the vector index based on the user's query.
|
|
81
|
+
2. **Augment** the prompt with retrieved context, conversation history, and system instructions.
|
|
82
|
+
3. **Generate** a grounded response that cites specific sources.
|
|
61
83
|
|
|
62
|
-
|
|
84
|
+
This approach ensures responses are factually anchored to your data rather than relying solely on the model's training data.
|
|
63
85
|
|
|
64
|
-
|
|
65
|
-
- **Real-time Processing**: Stream processing with Kafka
|
|
66
|
-
- **Data Validation**: Comprehensive data validation
|
|
67
|
-
- **Data Quality**: Data quality monitoring
|
|
86
|
+
---
|
|
68
87
|
|
|
69
88
|
## Security Architecture
|
|
70
89
|
|
|
71
|
-
###
|
|
90
|
+
### Defense in depth
|
|
72
91
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
92
|
+
| Layer | Measures |
|
|
93
|
+
| ----------- | ---------------------------------------------------------- |
|
|
94
|
+
| Network | VPC isolation, WAF, DDoS protection, private subnets |
|
|
95
|
+
| Application | Input validation, CSRF protection, Content Security Policy |
|
|
96
|
+
| Data | AES-256 encryption at rest, TLS 1.3 in transit |
|
|
97
|
+
| Access | RBAC, MFA, API key rotation, session management |
|
|
98
|
+
| Audit | Immutable audit logs, real-time anomaly detection |
|
|
77
99
|
|
|
78
100
|
### Compliance
|
|
79
101
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
- **
|
|
83
|
-
- **
|
|
102
|
+
Yantra maintains the following certifications and compliance standards:
|
|
103
|
+
|
|
104
|
+
- **SOC 2 Type II** — Independently audited annually
|
|
105
|
+
- **GDPR** — Full compliance with EU data protection regulations
|
|
106
|
+
- **ISO 27001** — Information security management certification
|
|
107
|
+
- **HIPAA** — Available for healthcare customers on Enterprise plans
|
|
@@ -1,67 +1,103 @@
|
|
|
1
1
|
# Conversation Management
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Tips and tools for keeping your Yantra conversations organized, productive, and easy to find.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
---
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## Organizing Conversations
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
- **Topic Definition**: Define conversation topic
|
|
11
|
-
- **Context Setting**: Set conversation context
|
|
12
|
-
- **Participant Invitation**: Invite conversation participants
|
|
9
|
+
### Naming threads
|
|
13
10
|
|
|
14
|
-
|
|
11
|
+
By default, threads are named after your first query. Rename them for better organization:
|
|
15
12
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
- **Archive Management**: Archive completed threads
|
|
13
|
+
1. Click the thread title in the sidebar or at the top of the conversation.
|
|
14
|
+
2. Type a descriptive name (e.g., "React performance optimization research").
|
|
15
|
+
3. Press Enter.
|
|
20
16
|
|
|
21
|
-
|
|
17
|
+
### Pinning important threads
|
|
22
18
|
|
|
23
|
-
|
|
19
|
+
Pin threads you reference frequently:
|
|
24
20
|
|
|
25
|
-
- **
|
|
26
|
-
-
|
|
27
|
-
-
|
|
28
|
-
- **Follow-up Management**: Manage follow-up questions
|
|
21
|
+
- Click **⋯** > **Pin** on any thread.
|
|
22
|
+
- Pinned threads stay at the top of the sidebar, regardless of age.
|
|
23
|
+
- Unpin anytime via **⋯** > **Unpin**.
|
|
29
24
|
|
|
30
|
-
###
|
|
25
|
+
### Tagging
|
|
31
26
|
|
|
32
|
-
|
|
33
|
-
- **Permission Control**: Control conversation permissions
|
|
34
|
-
- **Activity Monitoring**: Monitor conversation activity
|
|
35
|
-
- **Communication Tools**: Use built-in communication tools
|
|
27
|
+
Add tags to threads for categorization:
|
|
36
28
|
|
|
37
|
-
|
|
29
|
+
- Click the tag icon in the thread header.
|
|
30
|
+
- Add tags like "work," "personal," "research," "client: Acme Corp."
|
|
31
|
+
- Filter the thread list by tag.
|
|
38
32
|
|
|
39
|
-
|
|
33
|
+
---
|
|
40
34
|
|
|
41
|
-
|
|
42
|
-
- **Category System**: Use category system
|
|
43
|
-
- **Tagging System**: Tag conversations for easy finding
|
|
44
|
-
- **Search Functionality**: Search within conversations
|
|
35
|
+
## Thread Actions
|
|
45
36
|
|
|
46
|
-
|
|
37
|
+
| Action | Shortcut (Mac) | Shortcut (Windows) | Description |
|
|
38
|
+
| -------------- | -------------- | ------------------ | ----------------------------------------- |
|
|
39
|
+
| New thread | ⌘ + N | Ctrl + N | Start a fresh conversation |
|
|
40
|
+
| Search threads | ⌘ + F | Ctrl + F | Find threads by keyword |
|
|
41
|
+
| Pin/unpin | — | — | Keep threads accessible |
|
|
42
|
+
| Archive | — | — | Move to archive (not deleted) |
|
|
43
|
+
| Delete | — | — | Permanently remove (30-day recovery) |
|
|
44
|
+
| Export | — | — | Download as Markdown, PDF, or text |
|
|
45
|
+
| Branch | — | — | Fork the conversation at a specific point |
|
|
47
46
|
|
|
48
|
-
|
|
49
|
-
- **Attachment Management**: Manage file attachments
|
|
50
|
-
- **Link Management**: Manage external links
|
|
51
|
-
- **Citation Tracking**: Track source citations
|
|
47
|
+
---
|
|
52
48
|
|
|
53
|
-
##
|
|
49
|
+
## Branching Conversations
|
|
54
50
|
|
|
55
|
-
|
|
51
|
+
When a conversation takes you in an unexpected direction, branch it instead of cluttering the original thread:
|
|
56
52
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
53
|
+
1. Hover over any message in the thread.
|
|
54
|
+
2. Click the **Branch** icon.
|
|
55
|
+
3. A new thread is created with the conversation history up to that point.
|
|
56
|
+
4. Continue the original thread and the branch independently.
|
|
61
57
|
|
|
62
|
-
|
|
58
|
+
This is especially useful for:
|
|
63
59
|
|
|
64
|
-
-
|
|
65
|
-
-
|
|
66
|
-
-
|
|
67
|
-
|
|
60
|
+
- Exploring "what if" scenarios
|
|
61
|
+
- Separating sub-topics into their own threads
|
|
62
|
+
- Trying different approaches without losing the original context
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## Search and Filter
|
|
67
|
+
|
|
68
|
+
### Full-text search
|
|
69
|
+
|
|
70
|
+
Search across all your conversations:
|
|
71
|
+
|
|
72
|
+
- Search by **keywords** in your messages or Yantra's responses.
|
|
73
|
+
- Search by **date range** to find conversations from a specific period.
|
|
74
|
+
- Search by **search mode** (Regular, Pro, Deep Research).
|
|
75
|
+
|
|
76
|
+
### Filters
|
|
77
|
+
|
|
78
|
+
| Filter | Options |
|
|
79
|
+
| ------ | ------------------------------------------ |
|
|
80
|
+
| Date | Today, this week, this month, custom range |
|
|
81
|
+
| Mode | Regular, Pro Search, Deep Research |
|
|
82
|
+
| Tags | Your custom tags |
|
|
83
|
+
| Shared | Only shared threads, only private |
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## Auto-Cleanup
|
|
88
|
+
|
|
89
|
+
Configure automatic thread management:
|
|
90
|
+
|
|
91
|
+
- **Auto-archive** — Threads with no activity for X days are archived (default: 90 days).
|
|
92
|
+
- **Auto-delete** — Archived threads are deleted after X days (default: never).
|
|
93
|
+
- **Configurable** — Adjust at **Settings > Preferences > Threads**.
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Best Practices
|
|
98
|
+
|
|
99
|
+
1. **One topic per thread** — Keeps context clean and makes threads easier to find later.
|
|
100
|
+
2. **Name your threads** — A good name saves time when searching later.
|
|
101
|
+
3. **Use branches** — Don't let a tangent derail a focused research thread.
|
|
102
|
+
4. **Pin active projects** — Keep your current focus areas at the top.
|
|
103
|
+
5. **Archive regularly** — A clean sidebar helps you focus on what matters now.
|