@devilsdev/rag-pipeline-utils 2.1.7 → 2.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (148) hide show
  1. package/LICENSE +674 -674
  2. package/README.md +395 -186
  3. package/bin/cli.js +14 -62
  4. package/contracts/embedder-contract.json +52 -0
  5. package/contracts/llm-contract.json +36 -0
  6. package/contracts/loader-contract.json +43 -0
  7. package/contracts/retriever-contract.json +68 -0
  8. package/dist/index.cjs +67 -0
  9. package/dist/index.mjs +51 -0
  10. package/package.json +107 -21
  11. package/.codecov.yml +0 -21
  12. package/.dockerignore +0 -14
  13. package/.eslintignore +0 -5
  14. package/.eslintrc.json +0 -28
  15. package/.github/PROJECT_ROADMAP.md +0 -102
  16. package/.github/PULL_REQUEST_TEMPLATE.md +0 -8
  17. package/.github/release-template.md +0 -20
  18. package/.github/workflows/auto-release-note.yml +0 -73
  19. package/.github/workflows/ci.yml +0 -55
  20. package/.github/workflows/docs-ci.yml +0 -46
  21. package/.github/workflows/docs-deploy.yml +0 -64
  22. package/.github/workflows/enforce-release-review.yml +0 -50
  23. package/.github/workflows/generate-blog-post.yml +0 -44
  24. package/.github/workflows/post-release-generate-blog.yml +0 -71
  25. package/.github/workflows/publish-release-blog.yml +0 -47
  26. package/.github/workflows/release-sync.yml +0 -50
  27. package/.github/workflows/roadmap-label-sync.yml +0 -35
  28. package/.github/workflows/roadmap-labels.yml +0 -67
  29. package/.github/workflows/roadmap-sync.yml +0 -44
  30. package/.github/workflows/sync-roadmap.yml +0 -62
  31. package/.github/workflows/verify-blog-badge.yml +0 -50
  32. package/.husky/pre-commit +0 -6
  33. package/CHANGELOG.md +0 -131
  34. package/Dockerfile +0 -22
  35. package/Use-Cases.md +0 -134
  36. package/__tests__/fixtures/.ragrc.json +0 -8
  37. package/__tests__/fixtures/invalid-ragrc.json +0 -4
  38. package/__tests__/fixtures/sample-eval-dataset.json +0 -15
  39. package/__tests__/fixtures/sample.pdf +0 -1
  40. package/__tests__/fixtures/src/mocks/openai-embedder.js +0 -16
  41. package/__tests__/fixtures/src/mocks/openai-llm.js +0 -9
  42. package/__tests__/fixtures/src/mocks/pdf-loader.js +0 -17
  43. package/__tests__/fixtures/src/mocks/pinecone-retriever.js +0 -26
  44. package/__tests__/fixtures/valid-ragrc.json +0 -16
  45. package/__tests__/integration/cli/config-flow.test.js +0 -98
  46. package/__tests__/integration/config/load-config.test.js +0 -67
  47. package/__tests__/scripts/ensure-roadmap-labels.test.js +0 -64
  48. package/__tests__/unit/config/validate-schema.test.js +0 -121
  49. package/__tests__/unit/core/plugin-registry.test.js +0 -55
  50. package/__tests__/unit/reranker/__snapshots__/reranker.snapshot.test.js.snap +0 -15
  51. package/__tests__/unit/reranker/llm-reranker.test.js +0 -18
  52. package/__tests__/unit/reranker/reranker.enriched.test.js +0 -19
  53. package/__tests__/unit/reranker/reranker.fallback.test.js +0 -15
  54. package/__tests__/unit/reranker/reranker.snapshot.test.js +0 -15
  55. package/__tests__/unit/reranker/reranker.structured-output.test.js +0 -17
  56. package/codecov +0 -0
  57. package/codecov.SHA256SUM +0 -1
  58. package/codecov.SHA256SUM.sig +0 -16
  59. package/docs-site/README.md +0 -41
  60. package/docs-site/blog/2025-04-18-architecture-overview.md +0 -16
  61. package/docs-site/blog/2025-04-18-release-v2.1.1.md +0 -13
  62. package/docs-site/blog/2025-04-19-prompt-engineering-best-practices.md +0 -16
  63. package/docs-site/blog/2025-04-20-image-showcase.md +0 -14
  64. package/docs-site/blog/2025-04-20-release-2.1.1.md +0 -21
  65. package/docs-site/blog/2025-04-21-release-v2.1.3.md +0 -13
  66. package/docs-site/blog/2025-04-21-release-v2.1.4.md +0 -13
  67. package/docs-site/blog/2025-04-22-release-v2.1.5.md +0 -13
  68. package/docs-site/blog/2025-04-24-v2.1.6.md +0 -21
  69. package/docs-site/blog/authors.yml +0 -5
  70. package/docs-site/blog/tags.yml +0 -60
  71. package/docs-site/docs/Architecture.md +0 -76
  72. package/docs-site/docs/CLI.md +0 -69
  73. package/docs-site/docs/Evaluation.md +0 -61
  74. package/docs-site/docs/FAQ.md +0 -50
  75. package/docs-site/docs/Introduction.md +0 -58
  76. package/docs-site/docs/Plugins.md +0 -61
  77. package/docs-site/docs/Tutorials.md +0 -106
  78. package/docs-site/docs/Usage.md +0 -72
  79. package/docs-site/docs/Use-Cases.md +0 -77
  80. package/docs-site/docs/intro.mdx +0 -6
  81. package/docs-site/docusaurus.config.js +0 -88
  82. package/docs-site/package-lock.json +0 -17914
  83. package/docs-site/package.json +0 -47
  84. package/docs-site/sidebars.js +0 -39
  85. package/docs-site/src/components/HomepageFeatures/index.js +0 -56
  86. package/docs-site/src/components/HomepageFeatures/index.module.css +0 -57
  87. package/docs-site/src/components/HomepageHeader/index.js +0 -46
  88. package/docs-site/src/components/HomepageHeader/index.module.css +0 -77
  89. package/docs-site/src/css/custom.css +0 -104
  90. package/docs-site/src/pages/index.js +0 -20
  91. package/docs-site/src/pages/index.module.css +0 -23
  92. package/docs-site/src/pages/markdown-page.md +0 -7
  93. package/docs-site/static/.nojekyll +0 -0
  94. package/docs-site/static/img/authors/ali.jpg +0 -0
  95. package/docs-site/static/img/hero.jpg +0 -0
  96. package/docs-site/static/img/logo.png +0 -0
  97. package/docs-site/static/img/logo.svg +0 -6
  98. package/docs-site/static/img/social-card.jpg +0 -0
  99. package/jest.config.js +0 -24
  100. package/plugin.config.json +0 -14
  101. package/public/index.html +0 -0
  102. package/public/results/sample-eval.json +0 -0
  103. package/public/src/App.jsx +0 -57
  104. package/release.config.js +0 -36
  105. package/scripts/banner-injector.js +0 -102
  106. package/scripts/ci-runner.js +0 -51
  107. package/scripts/close-done-roadmap-issues.js +0 -65
  108. package/scripts/create-roadmap-issues.js +0 -47
  109. package/scripts/ensure-roadmap-labels.js +0 -73
  110. package/scripts/fix-mdx-blog-imports.js +0 -50
  111. package/scripts/fix-mdx-image-imports.js +0 -27
  112. package/scripts/generate-release-note.js +0 -110
  113. package/scripts/get-previous-tag.js +0 -40
  114. package/scripts/label-roadmap-issues.js +0 -125
  115. package/scripts/manual-test-checklist.md +0 -90
  116. package/scripts/repair-fixtures.js +0 -30
  117. package/scripts/setup.js +0 -31
  118. package/scripts/sync-labels.js +0 -71
  119. package/scripts/sync-roadmap-labels.js +0 -21
  120. package/scripts/tag-release.js +0 -64
  121. package/scripts/test-global-setup.js +0 -18
  122. package/scripts/validate-changelog-version.js +0 -48
  123. package/scripts/verify-fixture-mocks.js +0 -45
  124. package/scripts/verify-fixtures.js +0 -36
  125. package/server.js +0 -31
  126. package/src/config/load-config.js +0 -33
  127. package/src/config/load-plugin-config.js +0 -56
  128. package/src/config/validate-plugin-schema.js +0 -28
  129. package/src/config/validate-schema.js +0 -107
  130. package/src/core/create-pipeline.js +0 -96
  131. package/src/core/plugin-contracts.js +0 -21
  132. package/src/core/plugin-registry.js +0 -48
  133. package/src/dag/dag-engine.js +0 -70
  134. package/src/evaluate/evaluator.js +0 -44
  135. package/src/evaluate/scoring.js +0 -74
  136. package/src/loader/csv-loader.js +0 -30
  137. package/src/loader/directory-loader.js +0 -38
  138. package/src/loader/html-loader.js +0 -57
  139. package/src/loader/markdown-loader.js +0 -58
  140. package/src/mocks/openai-embedder.js +0 -30
  141. package/src/mocks/openai-llm.js +0 -18
  142. package/src/mocks/pdf-loader.js +0 -17
  143. package/src/mocks/pinecone-retriever.js +0 -25
  144. package/src/reranker/llm-reranker.js +0 -34
  145. package/src/utils/ci/diagnostic-reporter.js +0 -36
  146. package/src/utils/logger.js +0 -29
  147. package/src/utils/retry.js +0 -42
  148. package/src/utils/validate-plugin-contract.js +0 -36
@@ -1,98 +0,0 @@
1
- /**
2
- * Version: 1.1.0
3
- * Description: Final CLI config fallback integration test with fixture mock copying and validation
4
- * Author: Ali Kahwaji
5
- */
6
-
7
- import { execSync } from 'child_process';
8
- import {
9
- mkdirSync,
10
- rmSync,
11
- writeFileSync,
12
- copyFileSync,
13
- existsSync,
14
- readdirSync,
15
- readFileSync
16
- } from 'fs';
17
- import { join, resolve } from 'path';
18
- import { fileURLToPath } from 'url';
19
-
20
- const __filename = fileURLToPath(import.meta.url);
21
- const __dirname = resolve(__filename, '..');
22
-
23
- const CLI_PATH = resolve('bin/cli.js');
24
- const TEMP_TEST_DIR = resolve('__tests__/__temp__/cli-config-fallback');
25
- const FIXTURE_CONFIG = join(TEMP_TEST_DIR, '.ragrc.json');
26
- const FIXTURE_PDF = resolve('__tests__/fixtures/sample.pdf');
27
-
28
- // Fixture sources
29
- const FIXTURE_SRC = resolve('__tests__/fixtures/src/mocks');
30
- const PDF_LOADER_SRC = join(FIXTURE_SRC, 'pdf-loader.js');
31
- const EMBEDDER_SRC = join(FIXTURE_SRC, 'openai-embedder.js');
32
- const RETRIEVER_SRC = join(FIXTURE_SRC, 'pinecone-retriever.js');
33
- const LLM_SRC = join(FIXTURE_SRC, 'openai-llm.js');
34
-
35
- const FIXTURE_CONFIG_OBJECT = {
36
- loader: { pdf: './src/mocks/pdf-loader.js' },
37
- embedder: { openai: './src/mocks/openai-embedder.js' },
38
- retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
39
- llm: { openai: './src/mocks/openai-llm.js' },
40
- namespace: 'cli-config-test',
41
- pipeline: ['loader', 'embedder', 'retriever']
42
- };
43
-
44
- describe('CLI integration with .ragrc.json config fallback', () => {
45
- beforeAll(() => {
46
- rmSync(TEMP_TEST_DIR, { recursive: true, force: true });
47
- mkdirSync(join(TEMP_TEST_DIR, 'src/mocks'), { recursive: true });
48
-
49
- // Assert existence
50
- [FIXTURE_PDF, PDF_LOADER_SRC, EMBEDDER_SRC, RETRIEVER_SRC, LLM_SRC].forEach(file => {
51
- if (!existsSync(file)) throw new Error(`[FATAL] Missing required fixture: ${file}`);
52
- });
53
-
54
- // Copy fixture PDF
55
- copyFileSync(FIXTURE_PDF, join(TEMP_TEST_DIR, 'sample.pdf'));
56
-
57
- // Copy mock plugins into temp dir
58
- copyFileSync(PDF_LOADER_SRC, join(TEMP_TEST_DIR, 'src/mocks/pdf-loader.js'));
59
- copyFileSync(EMBEDDER_SRC, join(TEMP_TEST_DIR, 'src/mocks/openai-embedder.js'));
60
- copyFileSync(RETRIEVER_SRC, join(TEMP_TEST_DIR, 'src/mocks/pinecone-retriever.js'));
61
- copyFileSync(LLM_SRC, join(TEMP_TEST_DIR, 'src/mocks/openai-llm.js'));
62
-
63
- // Write valid .ragrc.json
64
- writeFileSync(FIXTURE_CONFIG, JSON.stringify(FIXTURE_CONFIG_OBJECT, null, 2));
65
-
66
- console.debug('[DEBUG] .ragrc.json contents:\n', JSON.stringify(FIXTURE_CONFIG_OBJECT, null, 2));
67
- console.debug('[SETUP] Temp dir:', TEMP_TEST_DIR);
68
- console.debug('[SETUP] Contents:', readdirSync(TEMP_TEST_DIR));
69
- });
70
-
71
- afterAll(() => {
72
- rmSync(TEMP_TEST_DIR, { recursive: true, force: true });
73
- });
74
-
75
- test('validates plugin sections contain resolvable names', () => {
76
- const config = JSON.parse(readFileSync(FIXTURE_CONFIG, 'utf-8'));
77
- ['loader', 'embedder', 'retriever', 'llm'].forEach(type => {
78
- expect(typeof config[type]).toBe('object');
79
- expect(Object.keys(config[type]).length).toBeGreaterThan(0);
80
- });
81
- });
82
-
83
- test('executes CLI ingest using config fallback', () => {
84
- const result = execSync(`node ${CLI_PATH} ingest sample.pdf`, {
85
- cwd: TEMP_TEST_DIR,
86
- encoding: 'utf-8'
87
- });
88
- expect(result).toMatch(/Ingestion complete/);
89
- });
90
-
91
- test('executes CLI query using config fallback', () => {
92
- const result = execSync(`node ${CLI_PATH} query "What is this test about?"`, {
93
- cwd: TEMP_TEST_DIR,
94
- encoding: 'utf-8'
95
- });
96
- expect(result).toMatch(/Answer:/);
97
- });
98
- });
@@ -1,67 +0,0 @@
1
- /**
2
- * Version: 1.1.1
3
- * Description: Integration tests for loading and validating .ragrc.json via loadRagConfig()
4
- * Author: Ali Kahwaji
5
- * File: __tests__/integration/config/load-config.test.js
6
- */
7
-
8
- import fs from 'fs';
9
- import { copyFileSync, mkdirSync, rmSync, existsSync } from 'fs';
10
- import { join, resolve } from 'path';
11
- import { fileURLToPath } from 'url';
12
- import { loadRagConfig } from '../../../src/config/load-config.js';
13
-
14
- const __filename = fileURLToPath(import.meta.url);
15
- const __dirname = resolve(__filename, '..');
16
-
17
- // Fixture paths
18
- const FIXTURE_DIR = resolve(__dirname, '../../fixtures');
19
- const VALID_FIXTURE = join(FIXTURE_DIR, 'valid-ragrc.json');
20
- const INVALID_FIXTURE = join(FIXTURE_DIR, 'invalid-ragrc.json');
21
-
22
- // Temp directory used during tests
23
- const TEMP_DIR = resolve('__tests__', '__temp__', 'load-config');
24
- const TEMP_CONFIG = join(TEMP_DIR, '.ragrc.json');
25
-
26
- // Ensure fixtures are available before copying
27
- function assertFixturesExist() {
28
- const required = [VALID_FIXTURE, INVALID_FIXTURE];
29
- required.forEach(file => {
30
- if (!existsSync(file)) {
31
- throw new Error(`❌ Required test fixture missing: ${file}`);
32
- }
33
- });
34
- }
35
-
36
- describe('loadRagConfig()', () => {
37
- beforeAll(() => {
38
- rmSync(TEMP_DIR, { recursive: true, force: true });
39
- mkdirSync(TEMP_DIR, { recursive: true });
40
-
41
- assertFixturesExist();
42
- });
43
-
44
- afterAll(() => {
45
- rmSync(TEMP_DIR, { recursive: true, force: true });
46
- });
47
-
48
- test('loads a valid .ragrc.json config file successfully', () => {
49
- copyFileSync(VALID_FIXTURE, TEMP_CONFIG);
50
-
51
- const config = loadRagConfig(TEMP_CONFIG);
52
- expect(config).toBeDefined();
53
- expect(config.namespace).toBe('cli-config-test');
54
- expect(config.loader?.pdf).toBe('./src/mocks/pdf-loader.js');
55
- expect(config.pipeline).toEqual(expect.arrayContaining(['loader', 'embedder', 'retriever']));
56
- });
57
-
58
- test('throws validation error for invalid .ragrc.json config', () => {
59
- copyFileSync(INVALID_FIXTURE, TEMP_CONFIG);
60
- expect(() => loadRagConfig(TEMP_CONFIG)).toThrow(/Config validation failed/);
61
- });
62
-
63
- test('throws error if config file does not exist', () => {
64
- const NON_EXISTENT = join(TEMP_DIR, 'missing.json');
65
- expect(() => loadRagConfig(NON_EXISTENT)).toThrow(/Config file not found/);
66
- });
67
- });
@@ -1,64 +0,0 @@
1
- /**
2
- * Version: 2.1.2
3
- * Path: __tests__/scripts/ensure-roadmap-labels.test.js
4
- * Description: Unit tests for roadmap label creation using ESM-compatible Jest syntax.
5
- * Author: Ali Kahwaji
6
- */
7
-
8
- import { jest } from '@jest/globals';
9
-
10
- jest.unstable_mockModule('octokit', () => ({
11
- Octokit: jest.fn().mockImplementation(() => ({
12
- rest: {
13
- issues: {
14
- listLabelsForRepo: mockListLabelsForRepo,
15
- createLabel: mockCreateLabel
16
- }
17
- }
18
- }))
19
- }));
20
-
21
- // ✅ Shared mocks outside import context
22
- const mockCreateLabel = jest.fn();
23
- const mockListLabelsForRepo = jest.fn();
24
-
25
- let ensureRoadmapLabels;
26
- let roadmapLabels;
27
-
28
- beforeAll(async () => {
29
- const imported = await import('../../scripts/ensure-roadmap-labels.js');
30
- ensureRoadmapLabels = imported.ensureRoadmapLabels;
31
- roadmapLabels = imported.roadmapLabels;
32
- });
33
-
34
- beforeEach(() => {
35
- mockCreateLabel.mockReset();
36
- mockListLabelsForRepo.mockReset();
37
- });
38
-
39
- describe('ensureRoadmapLabels', () => {
40
- it('creates all labels if none exist', async () => {
41
- mockListLabelsForRepo.mockResolvedValue({ data: [] });
42
-
43
- await ensureRoadmapLabels({ token: 'test-token', owner: 'ali', repo: 'rag-pipeline-utils' });
44
-
45
- expect(mockCreateLabel).toHaveBeenCalledTimes(roadmapLabels.length);
46
- for (const label of roadmapLabels) {
47
- expect(mockCreateLabel).toHaveBeenCalledWith({
48
- owner: 'ali',
49
- repo: 'rag-pipeline-utils',
50
- ...label
51
- });
52
- }
53
- });
54
-
55
- it('skips already existing labels', async () => {
56
- const existing = [{ name: 'priority: high' }, { name: 'group: docs' }];
57
- mockListLabelsForRepo.mockResolvedValue({ data: existing });
58
-
59
- await ensureRoadmapLabels({ token: 'test-token', owner: 'ali', repo: 'rag-pipeline-utils' });
60
-
61
- const expected = roadmapLabels.filter(l => !existing.some(e => e.name === l.name));
62
- expect(mockCreateLabel).toHaveBeenCalledTimes(expected.length);
63
- });
64
- });
@@ -1,121 +0,0 @@
1
- /**
2
- * Version: 1.1.0
3
- * Description: Schema validation tests for .ragrc.json using AJV and external fixtures
4
- * Author: Ali Kahwaji
5
- * File: __tests__/unit/config/validate-schema.test.js
6
- */
7
-
8
- import { readFileSync } from 'fs';
9
- import { resolve } from 'path';
10
- import { validateRagrcSchema, validatePluginSchema } from '../../../src/config/validate-schema.js';
11
-
12
- const readFixture = (name) => {
13
- const filePath = resolve('__tests__/fixtures',name.replace('.ragrc', '-ragrc'));
14
- return JSON.parse(readFileSync(filePath, 'utf-8'));
15
- };
16
-
17
- describe('validateRagrcSchema', () => {
18
- test('passes with valid structured plugin schema', () => {
19
- const config = readFixture('valid-ragrc.json');
20
- const result = validateRagrcSchema(config);
21
- expect(result.valid).toBe(true);
22
- expect(result.errors).toBeUndefined(); // updated: null => undefined
23
- });
24
-
25
- test('fails with missing pipeline', () => {
26
- const config = {
27
- loader: { pdf: './src/mocks/pdf-loader.js' },
28
- namespace: 'no-pipeline'
29
- };
30
- const result = validateRagrcSchema(config);
31
- expect(result.valid).toBe(false);
32
- expect(result.errors).toEqual(
33
- expect.arrayContaining([
34
- expect.objectContaining({
35
- keyword: 'required',
36
- params: expect.objectContaining({ missingProperty: 'pipeline' })
37
- })
38
- ])
39
- );
40
- });
41
-
42
- test('fails with non-object plugin sections', () => {
43
- const config = {
44
- loader: './src/invalid.js', // should be object
45
- embedder: { openai: './src/mocks/openai-embedder.js' },
46
- retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
47
- llm: { openai: './src/mocks/openai-llm.js' },
48
- pipeline: ['loader'],
49
- namespace: 'invalid-section'
50
- };
51
- const result = validateRagrcSchema(config);
52
- expect(result.valid).toBe(false);
53
- expect(result.errors).toEqual(
54
- expect.arrayContaining([
55
- expect.objectContaining({ instancePath: '/loader' })
56
- ])
57
- );
58
- });
59
-
60
- test('fails with non-string namespace', () => {
61
- const config = {
62
- loader: { pdf: './src/mocks/pdf-loader.js' },
63
- embedder: { openai: './src/mocks/openai-embedder.js' },
64
- retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
65
- llm: { openai: './src/mocks/openai-llm.js' },
66
- pipeline: ['loader'],
67
- namespace: 12345
68
- };
69
- const result = validateRagrcSchema(config);
70
- expect(result.valid).toBe(false);
71
- expect(result.errors).toEqual(
72
- expect.arrayContaining([
73
- expect.objectContaining({ instancePath: '/namespace' })
74
- ])
75
- );
76
- });
77
- });
78
-
79
- describe('validatePluginSchema (subset)', () => {
80
- test('passes with plugin-only valid structure', () => {
81
- const config = {
82
- loader: { pdf: './src/mocks/pdf-loader.js' },
83
- embedder: { openai: './src/mocks/openai-embedder.js' },
84
- retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
85
- llm: { openai: './src/mocks/openai-llm.js' }
86
- };
87
- const result = validatePluginSchema(config);
88
- expect(result.valid).toBe(true);
89
- });
90
-
91
- test('fails when plugin section is missing', () => {
92
- const config = {
93
- embedder: { openai: './mock.js' },
94
- retriever: { pinecone: './mock.js' },
95
- llm: { openai: './mock.js' }
96
- };
97
- const result = validatePluginSchema(config);
98
- expect(result.valid).toBe(false);
99
- expect(result.errors).toEqual(
100
- expect.arrayContaining([
101
- expect.objectContaining({ instancePath: '', message: expect.stringContaining('must have required property') })
102
- ])
103
- );
104
- });
105
-
106
- test('fails when plugin paths are not strings', () => {
107
- const config = {
108
- loader: { pdf: 12345 },
109
- embedder: { openai: './mock.js' },
110
- retriever: { pinecone: './mock.js' },
111
- llm: { openai: './mock.js' }
112
- };
113
- const result = validatePluginSchema(config);
114
- expect(result.valid).toBe(false);
115
- expect(result.errors).toEqual(
116
- expect.arrayContaining([
117
- expect.objectContaining({ instancePath: '/loader/pdf' })
118
- ])
119
- );
120
- });
121
- });
@@ -1,55 +0,0 @@
1
- /**
2
- * Version: 0.1.0
3
- * Path: /__tests__/unit/core/plugin-registry.test.js
4
- * Description: Unit tests for the PluginRegistry class
5
- * Author: Ali Kahwaji
6
- */
7
-
8
- import { PluginRegistry } from '../../../src/core/plugin-registry.js';
9
-
10
- describe('PluginRegistry', () => {
11
- let registry;
12
-
13
- beforeEach(() => {
14
- registry = new PluginRegistry();
15
- });
16
-
17
- test('registers and retrieves a plugin successfully', () => {
18
- const mockPlugin = { name: 'test-loader' };
19
- registry.register('loader', 'test', mockPlugin);
20
- const result = registry.get('loader', 'test');
21
- expect(result).toBe(mockPlugin);
22
- });
23
-
24
- test('overwrites existing plugin under the same name', () => {
25
- const firstPlugin = { name: 'first' };
26
- const secondPlugin = { name: 'second' };
27
- registry.register('embedder', 'dup', firstPlugin);
28
- registry.register('embedder', 'dup', secondPlugin);
29
- const result = registry.get('embedder', 'dup');
30
- expect(result).toBe(secondPlugin);
31
- });
32
-
33
- test('throws error for unknown plugin type on registration', () => {
34
- expect(() => registry.register('unknown-type', 'x', {})).toThrow(/Unknown plugin type/);
35
- });
36
-
37
- test('throws error when retrieving a nonexistent plugin', () => {
38
- expect(() => registry.get('retriever', 'missing')).toThrow(/Plugin not found/);
39
- });
40
-
41
- test('lists all registered plugin names for a type', () => {
42
- registry.register('llm', 'gpt-4', {});
43
- registry.register('llm', 'gpt-3.5', {});
44
- const list = registry.list('llm');
45
- expect(list).toContain('gpt-4');
46
- expect(list).toContain('gpt-3.5');
47
- expect(list.length).toBe(2);
48
- });
49
-
50
- test('throws error when listing plugins for invalid type', () => {
51
- expect(() => registry.list('invalid')).toThrow(/Unknown plugin type/);
52
- });
53
- });
54
-
55
-
@@ -1,15 +0,0 @@
1
- // Jest Snapshot v1, https://goo.gl/fbAQLP
2
-
3
- exports[`LLMReranker output snapshot 1`] = `
4
- [
5
- {
6
- "text": "B",
7
- },
8
- {
9
- "text": "A",
10
- },
11
- {
12
- "text": "C",
13
- },
14
- ]
15
- `;
@@ -1,18 +0,0 @@
1
- /**
2
- * File: __tests__/unit/reranker/llm-reranker.test.js
3
- * Version: 1.0.0
4
- * Description: Validates core behavior of the LLMReranker module
5
- * Author: @devilsdev
6
- */
7
-
8
- import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
9
-
10
- describe('LLMReranker', () => {
11
- test('returns reranked documents based on mock LLM JSON output', async () => {
12
- const mockLLM = { async generate() { return '[1, 0, 2]'; } };
13
- const docs = [{ text: 'A' }, { text: 'B' }, { text: 'C' }];
14
- const reranker = new LLMReranker({ llm: mockLLM });
15
- const result = await reranker.rerank('What is A?', docs);
16
- expect(result.map(d => d.text)).toEqual(['B', 'A', 'C']);
17
- });
18
- });
@@ -1,19 +0,0 @@
1
- /**
2
- * File: __tests__/unit/reranker/reranker.enriched.test.js
3
- * Version: 1.0.0
4
- * Description: Tests support for extended document structures
5
- * Author: @devilsdev
6
- */
7
- import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
8
-
9
- test('supports documents with metadata', async () => {
10
- const mockLLM = { async generate() { return '[2, 0, 1]'; } };
11
- const docs = [
12
- { text: 'One', source: 'alpha' },
13
- { text: 'Two', source: 'beta' },
14
- { text: 'Three', source: 'gamma' }
15
- ];
16
- const reranker = new LLMReranker({ llm: mockLLM });
17
- const result = await reranker.rerank('Choose the most complete', docs);
18
- expect(result[0]).toHaveProperty('source', 'gamma');
19
- });
@@ -1,15 +0,0 @@
1
- /**
2
- * File: __tests__/unit/reranker/reranker.fallback.test.js
3
- * Version: 1.0.0
4
- * Description: Validates fallback handling for invalid LLM output
5
- * Author: @devilsdev
6
- */
7
- import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
8
-
9
- test('gracefully handles invalid LLM output', async () => {
10
- const mockLLM = { async generate() { return 'INVALID_JSON'; } };
11
- const docs = [{ text: 'X' }, { text: 'Y' }, { text: 'Z' }];
12
- const reranker = new LLMReranker({ llm: mockLLM });
13
- const result = await reranker.rerank('Prompt?', docs);
14
- expect(result.map(d => d.text)).toEqual(['X', 'Y', 'Z']);
15
- });
@@ -1,15 +0,0 @@
1
- /**
2
- * File: __tests__/unit/reranker/reranker.snapshot.test.js
3
- * Version: 1.0.0
4
- * Description: Captures a snapshot baseline of reranked results
5
- * Author: @devilsdev
6
- */
7
- import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
8
-
9
- test('LLMReranker output snapshot', async () => {
10
- const mockLLM = { async generate() { return '[1, 0, 2]'; } };
11
- const mockDocs = [{ text: 'A' }, { text: 'B' }, { text: 'C' }];
12
- const reranker = new LLMReranker({ llm: mockLLM });
13
- const result = await reranker.rerank('What is A?', mockDocs);
14
- expect(result).toMatchSnapshot();
15
- });
@@ -1,17 +0,0 @@
1
- /**
2
- * File: __tests__/unit/reranker/reranker.structured-output.test.js
3
- * Version: 1.0.0
4
- * Description: Validates structured LLM output parsing
5
- * Author: @devilsdev
6
- */
7
- import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
8
-
9
- test('parses structured LLM JSON result', async () => {
10
- const mockLLM = { async generate() {
11
- return JSON.stringify({ ranking: [0, 2, 1] });
12
- }};
13
- const docs = [{ text: 'Alpha' }, { text: 'Beta' }, { text: 'Gamma' }];
14
- const reranker = new LLMReranker({ llm: mockLLM });
15
- const result = await reranker.rerank('Which is alpha?', docs);
16
- expect(result[0].text).toBe('Alpha');
17
- });
package/codecov DELETED
Binary file
package/codecov.SHA256SUM DELETED
@@ -1 +0,0 @@
1
- 0f7aadde579ebde1443ad2f977beada703f562997fdda603f213faf2a8559868 codecov
@@ -1,16 +0,0 @@
1
- -----BEGIN PGP SIGNATURE-----
2
-
3
- iQIzBAABCgAdFiEEJwNOf9uFDgu8LGL/gGuyiu13mGkFAmf1418ACgkQgGuyiu13
4
- mGke3w/8DpgS1CDxKdV50CKy81LmiA995bEw2ePBRc90JzNrGX7m2JcFQr/GUITs
5
- JofhfQ6sxnvLLxTLAyiMI8Tcbr2yN/BIjBERIhN8b20zL3KGY6kVS85czQdPs0a7
6
- BlY3xp7VgCv24E/SWc0m367GvBjVnanhMg+hIoaK4HQpWOx+93GzaGZWCnISl6yh
7
- Zn13zn/b/oXwvd3iekh3qb918kPwq3F3Ufd8gHLf71gH0usDoRIUwVIMYMjuoMeI
8
- ceIjZYe1BLG+hNAUkqboi1GK8IJQH66hwyB82KQxUZG0zEOTwhbvFrIijR30BSk6
9
- KROFxxe8s1kuEZGI/zhKPXkYkjiIdXQQvMP7emsQeLQljRb2k3prUhHw7LyFTPla
10
- KRK9MX1BoLr1Iol0Py/y8luSBCNvAmeCmbh6uGWS/1SJ/mj6CiZibWIO40qH9NJI
11
- wDY/JHrTHA2W2EsvREegny7znurxk1ORwT2scEJobu/NiLQEvpz6fvHaKiBcX45z
12
- 53pv1v0xlTFCEF6Vfg2geywIk4Dfi4l7dxyG9hyHtb1lQ0zwcz9KaAPhMjiJCUPa
13
- zGh0nN/xgkD9u7TNhOW3vG01R/wNW5BF+E8+04AO+RroV230m+a/e958I6pCd9M+
14
- N6FtNND7ZyIVoHiVA/v/tWc+ODZ5b9K8Q1Y6a0jkCbFZ3ZeCrPI=
15
- =onSD
16
- -----END PGP SIGNATURE-----
@@ -1,41 +0,0 @@
1
- # Website
2
-
3
- This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
4
-
5
- ### Installation
6
-
7
- ```
8
- $ yarn
9
- ```
10
-
11
- ### Local Development
12
-
13
- ```
14
- $ yarn start
15
- ```
16
-
17
- This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
18
-
19
- ### Build
20
-
21
- ```
22
- $ yarn build
23
- ```
24
-
25
- This command generates static content into the `build` directory and can be served using any static contents hosting service.
26
-
27
- ### Deployment
28
-
29
- Using SSH:
30
-
31
- ```
32
- $ USE_SSH=true yarn deploy
33
- ```
34
-
35
- Not using SSH:
36
-
37
- ```
38
- $ GIT_USER=<Your GitHub username> yarn deploy
39
- ```
40
-
41
- If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
@@ -1,16 +0,0 @@
1
- ---
2
- slug: architecture-overview
3
- title: RAG Architecture Overview
4
- authors: [ali]
5
- tags: [architecture, modular]
6
- ---
7
-
8
- Explore the modular RAG pipeline architecture. Components like loaders, embedders, retrievers, and generators are pluggable and testable.
9
-
10
- <!-- truncate -->
11
-
12
- ## Key Concepts
13
-
14
- - Plugin registry for clean composition
15
- - Evaluation layer for metrics like BLEU and recall
16
- - CLI integration for dev-first workflows
@@ -1,13 +0,0 @@
1
- ---
2
- slug: release-v2.1.1
3
- title: ✨ Version v2.1.1 Released
4
- authors: [ali]
5
- tags: [release, changelog]
6
- ---
7
-
8
- RAG Pipeline Utils vv2.1.1 is now available on NPM!
9
-
10
- #### Highlights
11
- - 🚀 Auto-published blog from GitHub release
12
- - 📦 Version bump: v2.1.1
13
- - 📘 See CHANGELOG.md for full details
@@ -1,16 +0,0 @@
1
- ---
2
- slug: prompt-engineering-best-practices
3
- title: Prompt Engineering Best Practices
4
- authors: [ali]
5
- tags: [prompt-engineering, tutorials]
6
- ---
7
-
8
- Design better prompts for your LLM-driven RAG workflows using templating, in-context examples, and task-specific embeddings.
9
-
10
- <!-- truncate -->
11
-
12
- ## Tips
13
-
14
- - Use instructions + context
15
- - Dynamically inject retrieved passages
16
- - Validate through CLI or dashboard
@@ -1,14 +0,0 @@
1
- ---
2
- slug: image-showcase
3
- title: Image Showcase with LQIP
4
- authors: [ali]
5
- tags: [image, tutorials]
6
- ---
7
-
8
- Demonstration of how low-quality image placeholders (LQIP) improve blog UX and optimize web performance.
9
-
10
- <!-- truncate -->
11
-
12
- ## Details
13
-
14
- This post showcases `BlogImage` and `BlogCard` with LQIP integration via Plaiceholder and SSR-safe fallbacks.
@@ -1,21 +0,0 @@
1
- <!-- .github/release-template.md -->
2
-
3
- # 🔖 Release v2.1.1
4
-
5
- **Published:** `2025-04-20`
6
- **Compare changes:** [View diff](https://github.com/DevilsDev/rag-pipeline-utils/compare/v2.0.0...v2.1.1)
7
- **Contributors:** `3`
8
- **Commits:** `17`
9
-
10
- ---
11
-
12
- ## 📦 What's New
13
-
14
- - Placeholder for changelog details or highlights.
15
-
16
- ---
17
-
18
- ## 📘 Changelog
19
-
20
- See full details in [CHANGELOG.md](/CHANGELOG.md)
21
-