@devilsdev/rag-pipeline-utils 2.1.7 → 2.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +674 -674
- package/README.md +395 -186
- package/bin/cli.js +14 -62
- package/contracts/embedder-contract.json +52 -0
- package/contracts/llm-contract.json +36 -0
- package/contracts/loader-contract.json +43 -0
- package/contracts/retriever-contract.json +68 -0
- package/dist/index.cjs +67 -0
- package/dist/index.mjs +51 -0
- package/package.json +107 -21
- package/.codecov.yml +0 -21
- package/.dockerignore +0 -14
- package/.eslintignore +0 -5
- package/.eslintrc.json +0 -28
- package/.github/PROJECT_ROADMAP.md +0 -102
- package/.github/PULL_REQUEST_TEMPLATE.md +0 -8
- package/.github/release-template.md +0 -20
- package/.github/workflows/auto-release-note.yml +0 -73
- package/.github/workflows/ci.yml +0 -55
- package/.github/workflows/docs-ci.yml +0 -46
- package/.github/workflows/docs-deploy.yml +0 -64
- package/.github/workflows/enforce-release-review.yml +0 -50
- package/.github/workflows/generate-blog-post.yml +0 -44
- package/.github/workflows/post-release-generate-blog.yml +0 -71
- package/.github/workflows/publish-release-blog.yml +0 -47
- package/.github/workflows/release-sync.yml +0 -50
- package/.github/workflows/roadmap-label-sync.yml +0 -35
- package/.github/workflows/roadmap-labels.yml +0 -67
- package/.github/workflows/roadmap-sync.yml +0 -44
- package/.github/workflows/sync-roadmap.yml +0 -62
- package/.github/workflows/verify-blog-badge.yml +0 -50
- package/.husky/pre-commit +0 -6
- package/CHANGELOG.md +0 -131
- package/Dockerfile +0 -22
- package/Use-Cases.md +0 -134
- package/__tests__/fixtures/.ragrc.json +0 -8
- package/__tests__/fixtures/invalid-ragrc.json +0 -4
- package/__tests__/fixtures/sample-eval-dataset.json +0 -15
- package/__tests__/fixtures/sample.pdf +0 -1
- package/__tests__/fixtures/src/mocks/openai-embedder.js +0 -16
- package/__tests__/fixtures/src/mocks/openai-llm.js +0 -9
- package/__tests__/fixtures/src/mocks/pdf-loader.js +0 -17
- package/__tests__/fixtures/src/mocks/pinecone-retriever.js +0 -26
- package/__tests__/fixtures/valid-ragrc.json +0 -16
- package/__tests__/integration/cli/config-flow.test.js +0 -98
- package/__tests__/integration/config/load-config.test.js +0 -67
- package/__tests__/scripts/ensure-roadmap-labels.test.js +0 -64
- package/__tests__/unit/config/validate-schema.test.js +0 -121
- package/__tests__/unit/core/plugin-registry.test.js +0 -55
- package/__tests__/unit/reranker/__snapshots__/reranker.snapshot.test.js.snap +0 -15
- package/__tests__/unit/reranker/llm-reranker.test.js +0 -18
- package/__tests__/unit/reranker/reranker.enriched.test.js +0 -19
- package/__tests__/unit/reranker/reranker.fallback.test.js +0 -15
- package/__tests__/unit/reranker/reranker.snapshot.test.js +0 -15
- package/__tests__/unit/reranker/reranker.structured-output.test.js +0 -17
- package/codecov +0 -0
- package/codecov.SHA256SUM +0 -1
- package/codecov.SHA256SUM.sig +0 -16
- package/docs-site/README.md +0 -41
- package/docs-site/blog/2025-04-18-architecture-overview.md +0 -16
- package/docs-site/blog/2025-04-18-release-v2.1.1.md +0 -13
- package/docs-site/blog/2025-04-19-prompt-engineering-best-practices.md +0 -16
- package/docs-site/blog/2025-04-20-image-showcase.md +0 -14
- package/docs-site/blog/2025-04-20-release-2.1.1.md +0 -21
- package/docs-site/blog/2025-04-21-release-v2.1.3.md +0 -13
- package/docs-site/blog/2025-04-21-release-v2.1.4.md +0 -13
- package/docs-site/blog/2025-04-22-release-v2.1.5.md +0 -13
- package/docs-site/blog/2025-04-24-v2.1.6.md +0 -21
- package/docs-site/blog/authors.yml +0 -5
- package/docs-site/blog/tags.yml +0 -60
- package/docs-site/docs/Architecture.md +0 -76
- package/docs-site/docs/CLI.md +0 -69
- package/docs-site/docs/Evaluation.md +0 -61
- package/docs-site/docs/FAQ.md +0 -50
- package/docs-site/docs/Introduction.md +0 -58
- package/docs-site/docs/Plugins.md +0 -61
- package/docs-site/docs/Tutorials.md +0 -106
- package/docs-site/docs/Usage.md +0 -72
- package/docs-site/docs/Use-Cases.md +0 -77
- package/docs-site/docs/intro.mdx +0 -6
- package/docs-site/docusaurus.config.js +0 -88
- package/docs-site/package-lock.json +0 -17914
- package/docs-site/package.json +0 -47
- package/docs-site/sidebars.js +0 -39
- package/docs-site/src/components/HomepageFeatures/index.js +0 -56
- package/docs-site/src/components/HomepageFeatures/index.module.css +0 -57
- package/docs-site/src/components/HomepageHeader/index.js +0 -46
- package/docs-site/src/components/HomepageHeader/index.module.css +0 -77
- package/docs-site/src/css/custom.css +0 -104
- package/docs-site/src/pages/index.js +0 -20
- package/docs-site/src/pages/index.module.css +0 -23
- package/docs-site/src/pages/markdown-page.md +0 -7
- package/docs-site/static/.nojekyll +0 -0
- package/docs-site/static/img/authors/ali.jpg +0 -0
- package/docs-site/static/img/hero.jpg +0 -0
- package/docs-site/static/img/logo.png +0 -0
- package/docs-site/static/img/logo.svg +0 -6
- package/docs-site/static/img/social-card.jpg +0 -0
- package/jest.config.js +0 -24
- package/plugin.config.json +0 -14
- package/public/index.html +0 -0
- package/public/results/sample-eval.json +0 -0
- package/public/src/App.jsx +0 -57
- package/release.config.js +0 -36
- package/scripts/banner-injector.js +0 -102
- package/scripts/ci-runner.js +0 -51
- package/scripts/close-done-roadmap-issues.js +0 -65
- package/scripts/create-roadmap-issues.js +0 -47
- package/scripts/ensure-roadmap-labels.js +0 -73
- package/scripts/fix-mdx-blog-imports.js +0 -50
- package/scripts/fix-mdx-image-imports.js +0 -27
- package/scripts/generate-release-note.js +0 -110
- package/scripts/get-previous-tag.js +0 -40
- package/scripts/label-roadmap-issues.js +0 -125
- package/scripts/manual-test-checklist.md +0 -90
- package/scripts/repair-fixtures.js +0 -30
- package/scripts/setup.js +0 -31
- package/scripts/sync-labels.js +0 -71
- package/scripts/sync-roadmap-labels.js +0 -21
- package/scripts/tag-release.js +0 -64
- package/scripts/test-global-setup.js +0 -18
- package/scripts/validate-changelog-version.js +0 -48
- package/scripts/verify-fixture-mocks.js +0 -45
- package/scripts/verify-fixtures.js +0 -36
- package/server.js +0 -31
- package/src/config/load-config.js +0 -33
- package/src/config/load-plugin-config.js +0 -56
- package/src/config/validate-plugin-schema.js +0 -28
- package/src/config/validate-schema.js +0 -107
- package/src/core/create-pipeline.js +0 -96
- package/src/core/plugin-contracts.js +0 -21
- package/src/core/plugin-registry.js +0 -48
- package/src/dag/dag-engine.js +0 -70
- package/src/evaluate/evaluator.js +0 -44
- package/src/evaluate/scoring.js +0 -74
- package/src/loader/csv-loader.js +0 -30
- package/src/loader/directory-loader.js +0 -38
- package/src/loader/html-loader.js +0 -57
- package/src/loader/markdown-loader.js +0 -58
- package/src/mocks/openai-embedder.js +0 -30
- package/src/mocks/openai-llm.js +0 -18
- package/src/mocks/pdf-loader.js +0 -17
- package/src/mocks/pinecone-retriever.js +0 -25
- package/src/reranker/llm-reranker.js +0 -34
- package/src/utils/ci/diagnostic-reporter.js +0 -36
- package/src/utils/logger.js +0 -29
- package/src/utils/retry.js +0 -42
- package/src/utils/validate-plugin-contract.js +0 -36
|
@@ -1,98 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Version: 1.1.0
|
|
3
|
-
* Description: Final CLI config fallback integration test with fixture mock copying and validation
|
|
4
|
-
* Author: Ali Kahwaji
|
|
5
|
-
*/
|
|
6
|
-
|
|
7
|
-
import { execSync } from 'child_process';
|
|
8
|
-
import {
|
|
9
|
-
mkdirSync,
|
|
10
|
-
rmSync,
|
|
11
|
-
writeFileSync,
|
|
12
|
-
copyFileSync,
|
|
13
|
-
existsSync,
|
|
14
|
-
readdirSync,
|
|
15
|
-
readFileSync
|
|
16
|
-
} from 'fs';
|
|
17
|
-
import { join, resolve } from 'path';
|
|
18
|
-
import { fileURLToPath } from 'url';
|
|
19
|
-
|
|
20
|
-
const __filename = fileURLToPath(import.meta.url);
|
|
21
|
-
const __dirname = resolve(__filename, '..');
|
|
22
|
-
|
|
23
|
-
const CLI_PATH = resolve('bin/cli.js');
|
|
24
|
-
const TEMP_TEST_DIR = resolve('__tests__/__temp__/cli-config-fallback');
|
|
25
|
-
const FIXTURE_CONFIG = join(TEMP_TEST_DIR, '.ragrc.json');
|
|
26
|
-
const FIXTURE_PDF = resolve('__tests__/fixtures/sample.pdf');
|
|
27
|
-
|
|
28
|
-
// Fixture sources
|
|
29
|
-
const FIXTURE_SRC = resolve('__tests__/fixtures/src/mocks');
|
|
30
|
-
const PDF_LOADER_SRC = join(FIXTURE_SRC, 'pdf-loader.js');
|
|
31
|
-
const EMBEDDER_SRC = join(FIXTURE_SRC, 'openai-embedder.js');
|
|
32
|
-
const RETRIEVER_SRC = join(FIXTURE_SRC, 'pinecone-retriever.js');
|
|
33
|
-
const LLM_SRC = join(FIXTURE_SRC, 'openai-llm.js');
|
|
34
|
-
|
|
35
|
-
const FIXTURE_CONFIG_OBJECT = {
|
|
36
|
-
loader: { pdf: './src/mocks/pdf-loader.js' },
|
|
37
|
-
embedder: { openai: './src/mocks/openai-embedder.js' },
|
|
38
|
-
retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
|
|
39
|
-
llm: { openai: './src/mocks/openai-llm.js' },
|
|
40
|
-
namespace: 'cli-config-test',
|
|
41
|
-
pipeline: ['loader', 'embedder', 'retriever']
|
|
42
|
-
};
|
|
43
|
-
|
|
44
|
-
describe('CLI integration with .ragrc.json config fallback', () => {
|
|
45
|
-
beforeAll(() => {
|
|
46
|
-
rmSync(TEMP_TEST_DIR, { recursive: true, force: true });
|
|
47
|
-
mkdirSync(join(TEMP_TEST_DIR, 'src/mocks'), { recursive: true });
|
|
48
|
-
|
|
49
|
-
// Assert existence
|
|
50
|
-
[FIXTURE_PDF, PDF_LOADER_SRC, EMBEDDER_SRC, RETRIEVER_SRC, LLM_SRC].forEach(file => {
|
|
51
|
-
if (!existsSync(file)) throw new Error(`[FATAL] Missing required fixture: ${file}`);
|
|
52
|
-
});
|
|
53
|
-
|
|
54
|
-
// Copy fixture PDF
|
|
55
|
-
copyFileSync(FIXTURE_PDF, join(TEMP_TEST_DIR, 'sample.pdf'));
|
|
56
|
-
|
|
57
|
-
// Copy mock plugins into temp dir
|
|
58
|
-
copyFileSync(PDF_LOADER_SRC, join(TEMP_TEST_DIR, 'src/mocks/pdf-loader.js'));
|
|
59
|
-
copyFileSync(EMBEDDER_SRC, join(TEMP_TEST_DIR, 'src/mocks/openai-embedder.js'));
|
|
60
|
-
copyFileSync(RETRIEVER_SRC, join(TEMP_TEST_DIR, 'src/mocks/pinecone-retriever.js'));
|
|
61
|
-
copyFileSync(LLM_SRC, join(TEMP_TEST_DIR, 'src/mocks/openai-llm.js'));
|
|
62
|
-
|
|
63
|
-
// Write valid .ragrc.json
|
|
64
|
-
writeFileSync(FIXTURE_CONFIG, JSON.stringify(FIXTURE_CONFIG_OBJECT, null, 2));
|
|
65
|
-
|
|
66
|
-
console.debug('[DEBUG] .ragrc.json contents:\n', JSON.stringify(FIXTURE_CONFIG_OBJECT, null, 2));
|
|
67
|
-
console.debug('[SETUP] Temp dir:', TEMP_TEST_DIR);
|
|
68
|
-
console.debug('[SETUP] Contents:', readdirSync(TEMP_TEST_DIR));
|
|
69
|
-
});
|
|
70
|
-
|
|
71
|
-
afterAll(() => {
|
|
72
|
-
rmSync(TEMP_TEST_DIR, { recursive: true, force: true });
|
|
73
|
-
});
|
|
74
|
-
|
|
75
|
-
test('validates plugin sections contain resolvable names', () => {
|
|
76
|
-
const config = JSON.parse(readFileSync(FIXTURE_CONFIG, 'utf-8'));
|
|
77
|
-
['loader', 'embedder', 'retriever', 'llm'].forEach(type => {
|
|
78
|
-
expect(typeof config[type]).toBe('object');
|
|
79
|
-
expect(Object.keys(config[type]).length).toBeGreaterThan(0);
|
|
80
|
-
});
|
|
81
|
-
});
|
|
82
|
-
|
|
83
|
-
test('executes CLI ingest using config fallback', () => {
|
|
84
|
-
const result = execSync(`node ${CLI_PATH} ingest sample.pdf`, {
|
|
85
|
-
cwd: TEMP_TEST_DIR,
|
|
86
|
-
encoding: 'utf-8'
|
|
87
|
-
});
|
|
88
|
-
expect(result).toMatch(/Ingestion complete/);
|
|
89
|
-
});
|
|
90
|
-
|
|
91
|
-
test('executes CLI query using config fallback', () => {
|
|
92
|
-
const result = execSync(`node ${CLI_PATH} query "What is this test about?"`, {
|
|
93
|
-
cwd: TEMP_TEST_DIR,
|
|
94
|
-
encoding: 'utf-8'
|
|
95
|
-
});
|
|
96
|
-
expect(result).toMatch(/Answer:/);
|
|
97
|
-
});
|
|
98
|
-
});
|
|
@@ -1,67 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Version: 1.1.1
|
|
3
|
-
* Description: Integration tests for loading and validating .ragrc.json via loadRagConfig()
|
|
4
|
-
* Author: Ali Kahwaji
|
|
5
|
-
* File: __tests__/integration/config/load-config.test.js
|
|
6
|
-
*/
|
|
7
|
-
|
|
8
|
-
import fs from 'fs';
|
|
9
|
-
import { copyFileSync, mkdirSync, rmSync, existsSync } from 'fs';
|
|
10
|
-
import { join, resolve } from 'path';
|
|
11
|
-
import { fileURLToPath } from 'url';
|
|
12
|
-
import { loadRagConfig } from '../../../src/config/load-config.js';
|
|
13
|
-
|
|
14
|
-
const __filename = fileURLToPath(import.meta.url);
|
|
15
|
-
const __dirname = resolve(__filename, '..');
|
|
16
|
-
|
|
17
|
-
// Fixture paths
|
|
18
|
-
const FIXTURE_DIR = resolve(__dirname, '../../fixtures');
|
|
19
|
-
const VALID_FIXTURE = join(FIXTURE_DIR, 'valid-ragrc.json');
|
|
20
|
-
const INVALID_FIXTURE = join(FIXTURE_DIR, 'invalid-ragrc.json');
|
|
21
|
-
|
|
22
|
-
// Temp directory used during tests
|
|
23
|
-
const TEMP_DIR = resolve('__tests__', '__temp__', 'load-config');
|
|
24
|
-
const TEMP_CONFIG = join(TEMP_DIR, '.ragrc.json');
|
|
25
|
-
|
|
26
|
-
// Ensure fixtures are available before copying
|
|
27
|
-
function assertFixturesExist() {
|
|
28
|
-
const required = [VALID_FIXTURE, INVALID_FIXTURE];
|
|
29
|
-
required.forEach(file => {
|
|
30
|
-
if (!existsSync(file)) {
|
|
31
|
-
throw new Error(`❌ Required test fixture missing: ${file}`);
|
|
32
|
-
}
|
|
33
|
-
});
|
|
34
|
-
}
|
|
35
|
-
|
|
36
|
-
describe('loadRagConfig()', () => {
|
|
37
|
-
beforeAll(() => {
|
|
38
|
-
rmSync(TEMP_DIR, { recursive: true, force: true });
|
|
39
|
-
mkdirSync(TEMP_DIR, { recursive: true });
|
|
40
|
-
|
|
41
|
-
assertFixturesExist();
|
|
42
|
-
});
|
|
43
|
-
|
|
44
|
-
afterAll(() => {
|
|
45
|
-
rmSync(TEMP_DIR, { recursive: true, force: true });
|
|
46
|
-
});
|
|
47
|
-
|
|
48
|
-
test('loads a valid .ragrc.json config file successfully', () => {
|
|
49
|
-
copyFileSync(VALID_FIXTURE, TEMP_CONFIG);
|
|
50
|
-
|
|
51
|
-
const config = loadRagConfig(TEMP_CONFIG);
|
|
52
|
-
expect(config).toBeDefined();
|
|
53
|
-
expect(config.namespace).toBe('cli-config-test');
|
|
54
|
-
expect(config.loader?.pdf).toBe('./src/mocks/pdf-loader.js');
|
|
55
|
-
expect(config.pipeline).toEqual(expect.arrayContaining(['loader', 'embedder', 'retriever']));
|
|
56
|
-
});
|
|
57
|
-
|
|
58
|
-
test('throws validation error for invalid .ragrc.json config', () => {
|
|
59
|
-
copyFileSync(INVALID_FIXTURE, TEMP_CONFIG);
|
|
60
|
-
expect(() => loadRagConfig(TEMP_CONFIG)).toThrow(/Config validation failed/);
|
|
61
|
-
});
|
|
62
|
-
|
|
63
|
-
test('throws error if config file does not exist', () => {
|
|
64
|
-
const NON_EXISTENT = join(TEMP_DIR, 'missing.json');
|
|
65
|
-
expect(() => loadRagConfig(NON_EXISTENT)).toThrow(/Config file not found/);
|
|
66
|
-
});
|
|
67
|
-
});
|
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Version: 2.1.2
|
|
3
|
-
* Path: __tests__/scripts/ensure-roadmap-labels.test.js
|
|
4
|
-
* Description: Unit tests for roadmap label creation using ESM-compatible Jest syntax.
|
|
5
|
-
* Author: Ali Kahwaji
|
|
6
|
-
*/
|
|
7
|
-
|
|
8
|
-
import { jest } from '@jest/globals';
|
|
9
|
-
|
|
10
|
-
jest.unstable_mockModule('octokit', () => ({
|
|
11
|
-
Octokit: jest.fn().mockImplementation(() => ({
|
|
12
|
-
rest: {
|
|
13
|
-
issues: {
|
|
14
|
-
listLabelsForRepo: mockListLabelsForRepo,
|
|
15
|
-
createLabel: mockCreateLabel
|
|
16
|
-
}
|
|
17
|
-
}
|
|
18
|
-
}))
|
|
19
|
-
}));
|
|
20
|
-
|
|
21
|
-
// ✅ Shared mocks outside import context
|
|
22
|
-
const mockCreateLabel = jest.fn();
|
|
23
|
-
const mockListLabelsForRepo = jest.fn();
|
|
24
|
-
|
|
25
|
-
let ensureRoadmapLabels;
|
|
26
|
-
let roadmapLabels;
|
|
27
|
-
|
|
28
|
-
beforeAll(async () => {
|
|
29
|
-
const imported = await import('../../scripts/ensure-roadmap-labels.js');
|
|
30
|
-
ensureRoadmapLabels = imported.ensureRoadmapLabels;
|
|
31
|
-
roadmapLabels = imported.roadmapLabels;
|
|
32
|
-
});
|
|
33
|
-
|
|
34
|
-
beforeEach(() => {
|
|
35
|
-
mockCreateLabel.mockReset();
|
|
36
|
-
mockListLabelsForRepo.mockReset();
|
|
37
|
-
});
|
|
38
|
-
|
|
39
|
-
describe('ensureRoadmapLabels', () => {
|
|
40
|
-
it('creates all labels if none exist', async () => {
|
|
41
|
-
mockListLabelsForRepo.mockResolvedValue({ data: [] });
|
|
42
|
-
|
|
43
|
-
await ensureRoadmapLabels({ token: 'test-token', owner: 'ali', repo: 'rag-pipeline-utils' });
|
|
44
|
-
|
|
45
|
-
expect(mockCreateLabel).toHaveBeenCalledTimes(roadmapLabels.length);
|
|
46
|
-
for (const label of roadmapLabels) {
|
|
47
|
-
expect(mockCreateLabel).toHaveBeenCalledWith({
|
|
48
|
-
owner: 'ali',
|
|
49
|
-
repo: 'rag-pipeline-utils',
|
|
50
|
-
...label
|
|
51
|
-
});
|
|
52
|
-
}
|
|
53
|
-
});
|
|
54
|
-
|
|
55
|
-
it('skips already existing labels', async () => {
|
|
56
|
-
const existing = [{ name: 'priority: high' }, { name: 'group: docs' }];
|
|
57
|
-
mockListLabelsForRepo.mockResolvedValue({ data: existing });
|
|
58
|
-
|
|
59
|
-
await ensureRoadmapLabels({ token: 'test-token', owner: 'ali', repo: 'rag-pipeline-utils' });
|
|
60
|
-
|
|
61
|
-
const expected = roadmapLabels.filter(l => !existing.some(e => e.name === l.name));
|
|
62
|
-
expect(mockCreateLabel).toHaveBeenCalledTimes(expected.length);
|
|
63
|
-
});
|
|
64
|
-
});
|
|
@@ -1,121 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Version: 1.1.0
|
|
3
|
-
* Description: Schema validation tests for .ragrc.json using AJV and external fixtures
|
|
4
|
-
* Author: Ali Kahwaji
|
|
5
|
-
* File: __tests__/unit/config/validate-schema.test.js
|
|
6
|
-
*/
|
|
7
|
-
|
|
8
|
-
import { readFileSync } from 'fs';
|
|
9
|
-
import { resolve } from 'path';
|
|
10
|
-
import { validateRagrcSchema, validatePluginSchema } from '../../../src/config/validate-schema.js';
|
|
11
|
-
|
|
12
|
-
const readFixture = (name) => {
|
|
13
|
-
const filePath = resolve('__tests__/fixtures',name.replace('.ragrc', '-ragrc'));
|
|
14
|
-
return JSON.parse(readFileSync(filePath, 'utf-8'));
|
|
15
|
-
};
|
|
16
|
-
|
|
17
|
-
describe('validateRagrcSchema', () => {
|
|
18
|
-
test('passes with valid structured plugin schema', () => {
|
|
19
|
-
const config = readFixture('valid-ragrc.json');
|
|
20
|
-
const result = validateRagrcSchema(config);
|
|
21
|
-
expect(result.valid).toBe(true);
|
|
22
|
-
expect(result.errors).toBeUndefined(); // updated: null => undefined
|
|
23
|
-
});
|
|
24
|
-
|
|
25
|
-
test('fails with missing pipeline', () => {
|
|
26
|
-
const config = {
|
|
27
|
-
loader: { pdf: './src/mocks/pdf-loader.js' },
|
|
28
|
-
namespace: 'no-pipeline'
|
|
29
|
-
};
|
|
30
|
-
const result = validateRagrcSchema(config);
|
|
31
|
-
expect(result.valid).toBe(false);
|
|
32
|
-
expect(result.errors).toEqual(
|
|
33
|
-
expect.arrayContaining([
|
|
34
|
-
expect.objectContaining({
|
|
35
|
-
keyword: 'required',
|
|
36
|
-
params: expect.objectContaining({ missingProperty: 'pipeline' })
|
|
37
|
-
})
|
|
38
|
-
])
|
|
39
|
-
);
|
|
40
|
-
});
|
|
41
|
-
|
|
42
|
-
test('fails with non-object plugin sections', () => {
|
|
43
|
-
const config = {
|
|
44
|
-
loader: './src/invalid.js', // should be object
|
|
45
|
-
embedder: { openai: './src/mocks/openai-embedder.js' },
|
|
46
|
-
retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
|
|
47
|
-
llm: { openai: './src/mocks/openai-llm.js' },
|
|
48
|
-
pipeline: ['loader'],
|
|
49
|
-
namespace: 'invalid-section'
|
|
50
|
-
};
|
|
51
|
-
const result = validateRagrcSchema(config);
|
|
52
|
-
expect(result.valid).toBe(false);
|
|
53
|
-
expect(result.errors).toEqual(
|
|
54
|
-
expect.arrayContaining([
|
|
55
|
-
expect.objectContaining({ instancePath: '/loader' })
|
|
56
|
-
])
|
|
57
|
-
);
|
|
58
|
-
});
|
|
59
|
-
|
|
60
|
-
test('fails with non-string namespace', () => {
|
|
61
|
-
const config = {
|
|
62
|
-
loader: { pdf: './src/mocks/pdf-loader.js' },
|
|
63
|
-
embedder: { openai: './src/mocks/openai-embedder.js' },
|
|
64
|
-
retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
|
|
65
|
-
llm: { openai: './src/mocks/openai-llm.js' },
|
|
66
|
-
pipeline: ['loader'],
|
|
67
|
-
namespace: 12345
|
|
68
|
-
};
|
|
69
|
-
const result = validateRagrcSchema(config);
|
|
70
|
-
expect(result.valid).toBe(false);
|
|
71
|
-
expect(result.errors).toEqual(
|
|
72
|
-
expect.arrayContaining([
|
|
73
|
-
expect.objectContaining({ instancePath: '/namespace' })
|
|
74
|
-
])
|
|
75
|
-
);
|
|
76
|
-
});
|
|
77
|
-
});
|
|
78
|
-
|
|
79
|
-
describe('validatePluginSchema (subset)', () => {
|
|
80
|
-
test('passes with plugin-only valid structure', () => {
|
|
81
|
-
const config = {
|
|
82
|
-
loader: { pdf: './src/mocks/pdf-loader.js' },
|
|
83
|
-
embedder: { openai: './src/mocks/openai-embedder.js' },
|
|
84
|
-
retriever: { pinecone: './src/mocks/pinecone-retriever.js' },
|
|
85
|
-
llm: { openai: './src/mocks/openai-llm.js' }
|
|
86
|
-
};
|
|
87
|
-
const result = validatePluginSchema(config);
|
|
88
|
-
expect(result.valid).toBe(true);
|
|
89
|
-
});
|
|
90
|
-
|
|
91
|
-
test('fails when plugin section is missing', () => {
|
|
92
|
-
const config = {
|
|
93
|
-
embedder: { openai: './mock.js' },
|
|
94
|
-
retriever: { pinecone: './mock.js' },
|
|
95
|
-
llm: { openai: './mock.js' }
|
|
96
|
-
};
|
|
97
|
-
const result = validatePluginSchema(config);
|
|
98
|
-
expect(result.valid).toBe(false);
|
|
99
|
-
expect(result.errors).toEqual(
|
|
100
|
-
expect.arrayContaining([
|
|
101
|
-
expect.objectContaining({ instancePath: '', message: expect.stringContaining('must have required property') })
|
|
102
|
-
])
|
|
103
|
-
);
|
|
104
|
-
});
|
|
105
|
-
|
|
106
|
-
test('fails when plugin paths are not strings', () => {
|
|
107
|
-
const config = {
|
|
108
|
-
loader: { pdf: 12345 },
|
|
109
|
-
embedder: { openai: './mock.js' },
|
|
110
|
-
retriever: { pinecone: './mock.js' },
|
|
111
|
-
llm: { openai: './mock.js' }
|
|
112
|
-
};
|
|
113
|
-
const result = validatePluginSchema(config);
|
|
114
|
-
expect(result.valid).toBe(false);
|
|
115
|
-
expect(result.errors).toEqual(
|
|
116
|
-
expect.arrayContaining([
|
|
117
|
-
expect.objectContaining({ instancePath: '/loader/pdf' })
|
|
118
|
-
])
|
|
119
|
-
);
|
|
120
|
-
});
|
|
121
|
-
});
|
|
@@ -1,55 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Version: 0.1.0
|
|
3
|
-
* Path: /__tests__/unit/core/plugin-registry.test.js
|
|
4
|
-
* Description: Unit tests for the PluginRegistry class
|
|
5
|
-
* Author: Ali Kahwaji
|
|
6
|
-
*/
|
|
7
|
-
|
|
8
|
-
import { PluginRegistry } from '../../../src/core/plugin-registry.js';
|
|
9
|
-
|
|
10
|
-
describe('PluginRegistry', () => {
|
|
11
|
-
let registry;
|
|
12
|
-
|
|
13
|
-
beforeEach(() => {
|
|
14
|
-
registry = new PluginRegistry();
|
|
15
|
-
});
|
|
16
|
-
|
|
17
|
-
test('registers and retrieves a plugin successfully', () => {
|
|
18
|
-
const mockPlugin = { name: 'test-loader' };
|
|
19
|
-
registry.register('loader', 'test', mockPlugin);
|
|
20
|
-
const result = registry.get('loader', 'test');
|
|
21
|
-
expect(result).toBe(mockPlugin);
|
|
22
|
-
});
|
|
23
|
-
|
|
24
|
-
test('overwrites existing plugin under the same name', () => {
|
|
25
|
-
const firstPlugin = { name: 'first' };
|
|
26
|
-
const secondPlugin = { name: 'second' };
|
|
27
|
-
registry.register('embedder', 'dup', firstPlugin);
|
|
28
|
-
registry.register('embedder', 'dup', secondPlugin);
|
|
29
|
-
const result = registry.get('embedder', 'dup');
|
|
30
|
-
expect(result).toBe(secondPlugin);
|
|
31
|
-
});
|
|
32
|
-
|
|
33
|
-
test('throws error for unknown plugin type on registration', () => {
|
|
34
|
-
expect(() => registry.register('unknown-type', 'x', {})).toThrow(/Unknown plugin type/);
|
|
35
|
-
});
|
|
36
|
-
|
|
37
|
-
test('throws error when retrieving a nonexistent plugin', () => {
|
|
38
|
-
expect(() => registry.get('retriever', 'missing')).toThrow(/Plugin not found/);
|
|
39
|
-
});
|
|
40
|
-
|
|
41
|
-
test('lists all registered plugin names for a type', () => {
|
|
42
|
-
registry.register('llm', 'gpt-4', {});
|
|
43
|
-
registry.register('llm', 'gpt-3.5', {});
|
|
44
|
-
const list = registry.list('llm');
|
|
45
|
-
expect(list).toContain('gpt-4');
|
|
46
|
-
expect(list).toContain('gpt-3.5');
|
|
47
|
-
expect(list.length).toBe(2);
|
|
48
|
-
});
|
|
49
|
-
|
|
50
|
-
test('throws error when listing plugins for invalid type', () => {
|
|
51
|
-
expect(() => registry.list('invalid')).toThrow(/Unknown plugin type/);
|
|
52
|
-
});
|
|
53
|
-
});
|
|
54
|
-
|
|
55
|
-
|
|
@@ -1,18 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* File: __tests__/unit/reranker/llm-reranker.test.js
|
|
3
|
-
* Version: 1.0.0
|
|
4
|
-
* Description: Validates core behavior of the LLMReranker module
|
|
5
|
-
* Author: @devilsdev
|
|
6
|
-
*/
|
|
7
|
-
|
|
8
|
-
import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
|
|
9
|
-
|
|
10
|
-
describe('LLMReranker', () => {
|
|
11
|
-
test('returns reranked documents based on mock LLM JSON output', async () => {
|
|
12
|
-
const mockLLM = { async generate() { return '[1, 0, 2]'; } };
|
|
13
|
-
const docs = [{ text: 'A' }, { text: 'B' }, { text: 'C' }];
|
|
14
|
-
const reranker = new LLMReranker({ llm: mockLLM });
|
|
15
|
-
const result = await reranker.rerank('What is A?', docs);
|
|
16
|
-
expect(result.map(d => d.text)).toEqual(['B', 'A', 'C']);
|
|
17
|
-
});
|
|
18
|
-
});
|
|
@@ -1,19 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* File: __tests__/unit/reranker/reranker.enriched.test.js
|
|
3
|
-
* Version: 1.0.0
|
|
4
|
-
* Description: Tests support for extended document structures
|
|
5
|
-
* Author: @devilsdev
|
|
6
|
-
*/
|
|
7
|
-
import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
|
|
8
|
-
|
|
9
|
-
test('supports documents with metadata', async () => {
|
|
10
|
-
const mockLLM = { async generate() { return '[2, 0, 1]'; } };
|
|
11
|
-
const docs = [
|
|
12
|
-
{ text: 'One', source: 'alpha' },
|
|
13
|
-
{ text: 'Two', source: 'beta' },
|
|
14
|
-
{ text: 'Three', source: 'gamma' }
|
|
15
|
-
];
|
|
16
|
-
const reranker = new LLMReranker({ llm: mockLLM });
|
|
17
|
-
const result = await reranker.rerank('Choose the most complete', docs);
|
|
18
|
-
expect(result[0]).toHaveProperty('source', 'gamma');
|
|
19
|
-
});
|
|
@@ -1,15 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* File: __tests__/unit/reranker/reranker.fallback.test.js
|
|
3
|
-
* Version: 1.0.0
|
|
4
|
-
* Description: Validates fallback handling for invalid LLM output
|
|
5
|
-
* Author: @devilsdev
|
|
6
|
-
*/
|
|
7
|
-
import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
|
|
8
|
-
|
|
9
|
-
test('gracefully handles invalid LLM output', async () => {
|
|
10
|
-
const mockLLM = { async generate() { return 'INVALID_JSON'; } };
|
|
11
|
-
const docs = [{ text: 'X' }, { text: 'Y' }, { text: 'Z' }];
|
|
12
|
-
const reranker = new LLMReranker({ llm: mockLLM });
|
|
13
|
-
const result = await reranker.rerank('Prompt?', docs);
|
|
14
|
-
expect(result.map(d => d.text)).toEqual(['X', 'Y', 'Z']);
|
|
15
|
-
});
|
|
@@ -1,15 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* File: __tests__/unit/reranker/reranker.snapshot.test.js
|
|
3
|
-
* Version: 1.0.0
|
|
4
|
-
* Description: Captures a snapshot baseline of reranked results
|
|
5
|
-
* Author: @devilsdev
|
|
6
|
-
*/
|
|
7
|
-
import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
|
|
8
|
-
|
|
9
|
-
test('LLMReranker output snapshot', async () => {
|
|
10
|
-
const mockLLM = { async generate() { return '[1, 0, 2]'; } };
|
|
11
|
-
const mockDocs = [{ text: 'A' }, { text: 'B' }, { text: 'C' }];
|
|
12
|
-
const reranker = new LLMReranker({ llm: mockLLM });
|
|
13
|
-
const result = await reranker.rerank('What is A?', mockDocs);
|
|
14
|
-
expect(result).toMatchSnapshot();
|
|
15
|
-
});
|
|
@@ -1,17 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* File: __tests__/unit/reranker/reranker.structured-output.test.js
|
|
3
|
-
* Version: 1.0.0
|
|
4
|
-
* Description: Validates structured LLM output parsing
|
|
5
|
-
* Author: @devilsdev
|
|
6
|
-
*/
|
|
7
|
-
import { LLMReranker } from '../../../src/reranker/llm-reranker.js';
|
|
8
|
-
|
|
9
|
-
test('parses structured LLM JSON result', async () => {
|
|
10
|
-
const mockLLM = { async generate() {
|
|
11
|
-
return JSON.stringify({ ranking: [0, 2, 1] });
|
|
12
|
-
}};
|
|
13
|
-
const docs = [{ text: 'Alpha' }, { text: 'Beta' }, { text: 'Gamma' }];
|
|
14
|
-
const reranker = new LLMReranker({ llm: mockLLM });
|
|
15
|
-
const result = await reranker.rerank('Which is alpha?', docs);
|
|
16
|
-
expect(result[0].text).toBe('Alpha');
|
|
17
|
-
});
|
package/codecov
DELETED
|
Binary file
|
package/codecov.SHA256SUM
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
0f7aadde579ebde1443ad2f977beada703f562997fdda603f213faf2a8559868 codecov
|
package/codecov.SHA256SUM.sig
DELETED
|
@@ -1,16 +0,0 @@
|
|
|
1
|
-
-----BEGIN PGP SIGNATURE-----
|
|
2
|
-
|
|
3
|
-
iQIzBAABCgAdFiEEJwNOf9uFDgu8LGL/gGuyiu13mGkFAmf1418ACgkQgGuyiu13
|
|
4
|
-
mGke3w/8DpgS1CDxKdV50CKy81LmiA995bEw2ePBRc90JzNrGX7m2JcFQr/GUITs
|
|
5
|
-
JofhfQ6sxnvLLxTLAyiMI8Tcbr2yN/BIjBERIhN8b20zL3KGY6kVS85czQdPs0a7
|
|
6
|
-
BlY3xp7VgCv24E/SWc0m367GvBjVnanhMg+hIoaK4HQpWOx+93GzaGZWCnISl6yh
|
|
7
|
-
Zn13zn/b/oXwvd3iekh3qb918kPwq3F3Ufd8gHLf71gH0usDoRIUwVIMYMjuoMeI
|
|
8
|
-
ceIjZYe1BLG+hNAUkqboi1GK8IJQH66hwyB82KQxUZG0zEOTwhbvFrIijR30BSk6
|
|
9
|
-
KROFxxe8s1kuEZGI/zhKPXkYkjiIdXQQvMP7emsQeLQljRb2k3prUhHw7LyFTPla
|
|
10
|
-
KRK9MX1BoLr1Iol0Py/y8luSBCNvAmeCmbh6uGWS/1SJ/mj6CiZibWIO40qH9NJI
|
|
11
|
-
wDY/JHrTHA2W2EsvREegny7znurxk1ORwT2scEJobu/NiLQEvpz6fvHaKiBcX45z
|
|
12
|
-
53pv1v0xlTFCEF6Vfg2geywIk4Dfi4l7dxyG9hyHtb1lQ0zwcz9KaAPhMjiJCUPa
|
|
13
|
-
zGh0nN/xgkD9u7TNhOW3vG01R/wNW5BF+E8+04AO+RroV230m+a/e958I6pCd9M+
|
|
14
|
-
N6FtNND7ZyIVoHiVA/v/tWc+ODZ5b9K8Q1Y6a0jkCbFZ3ZeCrPI=
|
|
15
|
-
=onSD
|
|
16
|
-
-----END PGP SIGNATURE-----
|
package/docs-site/README.md
DELETED
|
@@ -1,41 +0,0 @@
|
|
|
1
|
-
# Website
|
|
2
|
-
|
|
3
|
-
This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
|
|
4
|
-
|
|
5
|
-
### Installation
|
|
6
|
-
|
|
7
|
-
```
|
|
8
|
-
$ yarn
|
|
9
|
-
```
|
|
10
|
-
|
|
11
|
-
### Local Development
|
|
12
|
-
|
|
13
|
-
```
|
|
14
|
-
$ yarn start
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
|
|
18
|
-
|
|
19
|
-
### Build
|
|
20
|
-
|
|
21
|
-
```
|
|
22
|
-
$ yarn build
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
This command generates static content into the `build` directory and can be served using any static contents hosting service.
|
|
26
|
-
|
|
27
|
-
### Deployment
|
|
28
|
-
|
|
29
|
-
Using SSH:
|
|
30
|
-
|
|
31
|
-
```
|
|
32
|
-
$ USE_SSH=true yarn deploy
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
Not using SSH:
|
|
36
|
-
|
|
37
|
-
```
|
|
38
|
-
$ GIT_USER=<Your GitHub username> yarn deploy
|
|
39
|
-
```
|
|
40
|
-
|
|
41
|
-
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
|
|
@@ -1,16 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
slug: architecture-overview
|
|
3
|
-
title: RAG Architecture Overview
|
|
4
|
-
authors: [ali]
|
|
5
|
-
tags: [architecture, modular]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
Explore the modular RAG pipeline architecture. Components like loaders, embedders, retrievers, and generators are pluggable and testable.
|
|
9
|
-
|
|
10
|
-
<!-- truncate -->
|
|
11
|
-
|
|
12
|
-
## Key Concepts
|
|
13
|
-
|
|
14
|
-
- Plugin registry for clean composition
|
|
15
|
-
- Evaluation layer for metrics like BLEU and recall
|
|
16
|
-
- CLI integration for dev-first workflows
|
|
@@ -1,13 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
slug: release-v2.1.1
|
|
3
|
-
title: ✨ Version v2.1.1 Released
|
|
4
|
-
authors: [ali]
|
|
5
|
-
tags: [release, changelog]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
RAG Pipeline Utils vv2.1.1 is now available on NPM!
|
|
9
|
-
|
|
10
|
-
#### Highlights
|
|
11
|
-
- 🚀 Auto-published blog from GitHub release
|
|
12
|
-
- 📦 Version bump: v2.1.1
|
|
13
|
-
- 📘 See CHANGELOG.md for full details
|
|
@@ -1,16 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
slug: prompt-engineering-best-practices
|
|
3
|
-
title: Prompt Engineering Best Practices
|
|
4
|
-
authors: [ali]
|
|
5
|
-
tags: [prompt-engineering, tutorials]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
Design better prompts for your LLM-driven RAG workflows using templating, in-context examples, and task-specific embeddings.
|
|
9
|
-
|
|
10
|
-
<!-- truncate -->
|
|
11
|
-
|
|
12
|
-
## Tips
|
|
13
|
-
|
|
14
|
-
- Use instructions + context
|
|
15
|
-
- Dynamically inject retrieved passages
|
|
16
|
-
- Validate through CLI or dashboard
|
|
@@ -1,14 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
slug: image-showcase
|
|
3
|
-
title: Image Showcase with LQIP
|
|
4
|
-
authors: [ali]
|
|
5
|
-
tags: [image, tutorials]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
Demonstration of how low-quality image placeholders (LQIP) improve blog UX and optimize web performance.
|
|
9
|
-
|
|
10
|
-
<!-- truncate -->
|
|
11
|
-
|
|
12
|
-
## Details
|
|
13
|
-
|
|
14
|
-
This post showcases `BlogImage` and `BlogCard` with LQIP integration via Plaiceholder and SSR-safe fallbacks.
|
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
<!-- .github/release-template.md -->
|
|
2
|
-
|
|
3
|
-
# 🔖 Release v2.1.1
|
|
4
|
-
|
|
5
|
-
**Published:** `2025-04-20`
|
|
6
|
-
**Compare changes:** [View diff](https://github.com/DevilsDev/rag-pipeline-utils/compare/v2.0.0...v2.1.1)
|
|
7
|
-
**Contributors:** `3`
|
|
8
|
-
**Commits:** `17`
|
|
9
|
-
|
|
10
|
-
---
|
|
11
|
-
|
|
12
|
-
## 📦 What's New
|
|
13
|
-
|
|
14
|
-
- Placeholder for changelog details or highlights.
|
|
15
|
-
|
|
16
|
-
---
|
|
17
|
-
|
|
18
|
-
## 📘 Changelog
|
|
19
|
-
|
|
20
|
-
See full details in [CHANGELOG.md](/CHANGELOG.md)
|
|
21
|
-
|