@ever-works/langfuse-plugin 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,78 @@
1
+ # @ever-works/langfuse-plugin
2
+
3
+ Langfuse Plugin - External prompt management with versioning, labels, and A/B testing
4
+
5
+ ## Plugin metadata
6
+
7
+ | Field | Value |
8
+ | ------------ | ----------------- |
9
+ | ID | `langfuse` |
10
+ | Category | `utility` |
11
+ | Capabilities | `prompt-provider` |
12
+ | Author | Ever Works Team |
13
+ | License | AGPL-3.0 |
14
+ | Built-in | yes |
15
+ | Auto-enable | yes |
16
+
17
+ ## What is the Langfuse plugin?
18
+
19
+ [Langfuse](https://langfuse.com) is an open-source prompt management and LLM observability platform. It provides a centralized place to author, version, label, and roll out prompt templates for LLM-powered applications, alongside tracing, evaluations, and analytics for every generation.
20
+
21
+ This plugin lets Ever Works pull its pipeline prompt templates directly from your Langfuse project. Once enabled, prompts that drive item extraction, categorization, comparisons, and other generation steps can be edited, versioned, and promoted between labels (e.g. `staging` → `production`) inside Langfuse — without redeploying the Ever Works platform.
22
+
23
+ ## Why use it?
24
+
25
+ - **Centralised prompt versioning** — every prompt change is captured as a new Langfuse version with full history
26
+ - **A/B-testable prompts** — use Langfuse labels and experiments to compare prompt variants on real generations
27
+ - **No-redeploy prompt edits** — update prompts in the Langfuse UI and have them picked up automatically by the platform
28
+ - **Observability of generations** — capture LLM calls, latency, token usage, and quality scores alongside the prompts that produced them
29
+ - **Traceable prompt iteration** — link outputs back to the exact prompt version, so regressions and improvements are easy to attribute
30
+
31
+ ## How it works in Ever Works
32
+
33
+ When Langfuse is enabled as the active prompt provider, Ever Works fetches named prompt templates (such as `standard-pipeline.generation`, `comparison.markdown`, `agent-pipeline.parent-system`, and others) from your Langfuse project at generation time and uses them in the corresponding pipeline steps. Outputs, token usage, and trace metadata can be reported back to Langfuse so each generation is observable end-to-end and tied to the prompt version that produced it.
34
+
35
+ ## Getting started
36
+
37
+ 1. Create a Langfuse project at [langfuse.com](https://langfuse.com) (or stand up a self-hosted instance).
38
+ 2. In [cloud.langfuse.com](https://cloud.langfuse.com) (or your self-hosted dashboard), generate an API key pair: a **public key** (`pk-lf-…`) and a **secret key** (`sk-lf-…`).
39
+ 3. Paste both keys into the Langfuse plugin settings in Ever Works. For self-hosted instances, also set the **Base URL** to your Langfuse host.
40
+ 4. Save the settings — Ever Works will validate the connection and start fetching prompts using the configured **Prompt Label** (defaults to `production`).
41
+
42
+ ## Settings
43
+
44
+ - **Secret Key** (`secretKey`, secret) — Langfuse secret key (`sk-lf-…`). Required. Also readable from `PLUGIN_LANGFUSE_SECRET_KEY`.
45
+ - **Public Key** (`publicKey`) — Langfuse public key (`pk-lf-…`). Required. Also readable from `PLUGIN_LANGFUSE_PUBLIC_KEY`.
46
+ - **Base URL** (`baseUrl`) — Langfuse base URL, only needed for self-hosted instances. Defaults to `https://cloud.langfuse.com`. Also readable from `PLUGIN_LANGFUSE_BASE_URL`.
47
+ - **Prompt Label** (`promptLabel`) — label used to fetch prompts (e.g. `production`, `staging`). Defaults to `production`.
48
+ - **Cache TTL (seconds)** (`cacheTtlSeconds`) — how long to cache fetched prompts locally. Defaults to 300, max 86400.
49
+
50
+ ## Troubleshooting
51
+
52
+ | Symptom | Likely cause | Fix |
53
+ | --------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
54
+ | `401 Unauthorized` from Langfuse | Public/secret key missing or wrong project | Re-issue keys at the Langfuse dashboard and re-enter both **Public Key** and **Secret Key**; or set `PLUGIN_LANGFUSE_PUBLIC_KEY` / `PLUGIN_LANGFUSE_SECRET_KEY` env vars |
55
+ | Prompts not appearing in Langfuse | Plugin not enabled OR `host` URL points to wrong region | Enable the plugin globally; verify **Host** matches the dashboard region (`https://cloud.langfuse.com` for US, `https://cloud.langfuse.eu` for EU) |
56
+ | `healthCheck` reports unhealthy | Credentials invalid OR Langfuse endpoint unreachable from the host | Verify the credentials with a `curl` against the documented `/api/public/health` endpoint and confirm outbound HTTPS is allowed |
57
+
58
+ ## Local development
59
+
60
+ This plugin ships built-in with the Ever Works platform. To work on it locally from the monorepo root:
61
+
62
+ ```bash
63
+ pnpm install
64
+ pnpm --filter @ever-works/langfuse-plugin build
65
+ pnpm --filter @ever-works/langfuse-plugin test
66
+ ```
67
+
68
+ ## Documentation
69
+
70
+ - [Ever Works documentation](https://docs.ever.works)
71
+ - [Ever Works repository](https://github.com/ever-works/ever-works)
72
+ - [Plugin system](../../plugin/README.md)
73
+ - [Langfuse homepage](https://langfuse.com)
74
+ - [Langfuse cloud console](https://cloud.langfuse.com)
75
+
76
+ ## License
77
+
78
+ AGPL-3.0
package/package.json ADDED
@@ -0,0 +1,92 @@
1
+ {
2
+ "name": "@ever-works/langfuse-plugin",
3
+ "version": "1.0.0",
4
+ "description": "Langfuse Plugin - External prompt management with versioning, labels, and A/B testing",
5
+ "author": {
6
+ "name": "Ever Co. LTD",
7
+ "email": "ever@ever.co",
8
+ "url": "https://ever.co"
9
+ },
10
+ "license": "AGPL-3.0",
11
+ "private": false,
12
+ "type": "module",
13
+ "main": "./dist/index.cjs",
14
+ "module": "./dist/index.js",
15
+ "types": "./dist/index.d.ts",
16
+ "exports": {
17
+ ".": {
18
+ "types": "./dist/index.d.ts",
19
+ "import": "./dist/index.js",
20
+ "require": "./dist/index.cjs"
21
+ }
22
+ },
23
+ "dependencies": {
24
+ "@langfuse/client": "^4.6.1"
25
+ },
26
+ "peerDependencies": {
27
+ "@ever-works/plugin": "1.1.0"
28
+ },
29
+ "devDependencies": {
30
+ "@types/node": "^22.0.0",
31
+ "tsup": "^8.4.0",
32
+ "typescript": "^5.7.3",
33
+ "vitest": "^4.1.0",
34
+ "@ever-works/plugin": "1.1.0"
35
+ },
36
+ "everworks": {
37
+ "plugin": {
38
+ "id": "langfuse",
39
+ "name": "Langfuse",
40
+ "version": "1.0.0",
41
+ "category": "utility",
42
+ "capabilities": [
43
+ "prompt-provider"
44
+ ],
45
+ "description": "External prompt management via Langfuse. Enables prompt versioning, A/B testing, and production labels for all pipeline prompts.",
46
+ "author": {
47
+ "name": "Ever Works Team"
48
+ },
49
+ "license": "AGPL-3.0",
50
+ "autoEnable": true,
51
+ "defaultForCapabilities": [
52
+ "prompt-provider"
53
+ ],
54
+ "systemPlugin": false,
55
+ "builtIn": true,
56
+ "envVars": [
57
+ {
58
+ "name": "PLUGIN_LANGFUSE_SECRET_KEY",
59
+ "required": false,
60
+ "secret": true,
61
+ "description": "Langfuse secret key for API authentication"
62
+ },
63
+ {
64
+ "name": "PLUGIN_LANGFUSE_PUBLIC_KEY",
65
+ "required": false,
66
+ "secret": false,
67
+ "description": "Langfuse public key for API authentication"
68
+ },
69
+ {
70
+ "name": "PLUGIN_LANGFUSE_BASE_URL",
71
+ "required": false,
72
+ "secret": false,
73
+ "description": "Langfuse base URL (for self-hosted instances)"
74
+ }
75
+ ],
76
+ "distribution": "registry"
77
+ }
78
+ },
79
+ "publishConfig": {
80
+ "access": "restricted",
81
+ "registry": "https://registry.npmjs.org"
82
+ },
83
+ "scripts": {
84
+ "build": "tsc --noEmit && tsup",
85
+ "dev": "tsup --watch",
86
+ "type-check": "tsc --noEmit",
87
+ "clean": "rm -rf dist",
88
+ "test": "vitest run --passWithNoTests",
89
+ "test:watch": "vitest",
90
+ "test:coverage": "vitest run --coverage"
91
+ }
92
+ }
@@ -0,0 +1,193 @@
1
+ import { describe, it, expect, vi, beforeEach } from 'vitest';
2
+ import { LangfusePlugin } from '../langfuse.plugin.js';
3
+ import type { PluginContext } from '@ever-works/plugin';
4
+
5
+ const mockPromptGet = vi.fn();
6
+
7
+ vi.mock('@langfuse/client', () => {
8
+ const MockLangfuseClient = vi.fn().mockImplementation(function () {
9
+ return { prompt: { get: mockPromptGet } };
10
+ });
11
+ return { LangfuseClient: MockLangfuseClient };
12
+ });
13
+
14
+ describe('LangfusePlugin', () => {
15
+ let plugin: LangfusePlugin;
16
+
17
+ const validSettings = {
18
+ secretKey: 'sk-lf-test-secret',
19
+ publicKey: 'pk-lf-test-public',
20
+ promptLabel: 'production',
21
+ cacheTtlSeconds: 300
22
+ };
23
+
24
+ const mockContext: PluginContext = {
25
+ logger: {
26
+ log: vi.fn(),
27
+ warn: vi.fn(),
28
+ error: vi.fn(),
29
+ debug: vi.fn()
30
+ },
31
+ config: {}
32
+ } as unknown as PluginContext;
33
+
34
+ beforeEach(() => {
35
+ vi.clearAllMocks();
36
+ plugin = new LangfusePlugin();
37
+ });
38
+
39
+ describe('metadata', () => {
40
+ it('should have correct plugin id and name', () => {
41
+ expect(plugin.id).toBe('langfuse');
42
+ expect(plugin.name).toBe('Langfuse');
43
+ expect(plugin.version).toBe('1.0.0');
44
+ });
45
+
46
+ it('should have utility category and prompt-provider capability', () => {
47
+ expect(plugin.category).toBe('utility');
48
+ expect(plugin.capabilities).toContain('prompt-provider');
49
+ });
50
+
51
+ it('should have hybrid configuration mode', () => {
52
+ expect(plugin.configurationMode).toBe('hybrid');
53
+ });
54
+ });
55
+
56
+ describe('settingsSchema', () => {
57
+ it('should require secretKey and publicKey', () => {
58
+ expect(plugin.settingsSchema.required).toContain('secretKey');
59
+ expect(plugin.settingsSchema.required).toContain('publicKey');
60
+ });
61
+
62
+ it('should mark secretKey as secret', () => {
63
+ const secretKeySchema = plugin.settingsSchema.properties?.secretKey as any;
64
+ expect(secretKeySchema['x-secret']).toBe(true);
65
+ expect(secretKeySchema['x-envVar']).toBe('PLUGIN_LANGFUSE_SECRET_KEY');
66
+ });
67
+
68
+ it('should have publicKey with envVar', () => {
69
+ const publicKeySchema = plugin.settingsSchema.properties?.publicKey as any;
70
+ expect(publicKeySchema['x-envVar']).toBe('PLUGIN_LANGFUSE_PUBLIC_KEY');
71
+ });
72
+
73
+ it('should have optional baseUrl for self-hosted with default', () => {
74
+ const baseUrlSchema = plugin.settingsSchema.properties?.baseUrl as any;
75
+ expect(baseUrlSchema['x-envVar']).toBe('PLUGIN_LANGFUSE_BASE_URL');
76
+ expect(baseUrlSchema.default).toBe('https://cloud.langfuse.com');
77
+ expect(plugin.settingsSchema.required).not.toContain('baseUrl');
78
+ });
79
+
80
+ it('should have promptLabel with default', () => {
81
+ const labelSchema = plugin.settingsSchema.properties?.promptLabel as any;
82
+ expect(labelSchema.default).toBe('production');
83
+ });
84
+
85
+ it('should have cacheTtlSeconds with default', () => {
86
+ const cacheSchema = plugin.settingsSchema.properties?.cacheTtlSeconds as any;
87
+ expect(cacheSchema.default).toBe(300);
88
+ expect(cacheSchema.type).toBe('number');
89
+ });
90
+ });
91
+
92
+ describe('isAvailable', () => {
93
+ it('should return false without settings', () => {
94
+ expect(plugin.isAvailable()).toBe(false);
95
+ });
96
+
97
+ it('should return false with empty settings', () => {
98
+ expect(plugin.isAvailable({})).toBe(false);
99
+ });
100
+
101
+ it('should return false with only secretKey', () => {
102
+ expect(plugin.isAvailable({ secretKey: 'sk-lf-test' })).toBe(false);
103
+ });
104
+
105
+ it('should return false with only publicKey', () => {
106
+ expect(plugin.isAvailable({ publicKey: 'pk-lf-test' })).toBe(false);
107
+ });
108
+
109
+ it('should return true with both keys', () => {
110
+ expect(plugin.isAvailable(validSettings)).toBe(true);
111
+ });
112
+ });
113
+
114
+ describe('getPrompt', () => {
115
+ beforeEach(async () => {
116
+ await plugin.onLoad(mockContext);
117
+ });
118
+
119
+ it('should return null when settings are missing', async () => {
120
+ const result = await plugin.getPrompt('test.key');
121
+ expect(result).toBeNull();
122
+ });
123
+
124
+ it('should return null when plugin is not available', async () => {
125
+ const result = await plugin.getPrompt('test.key', { settings: {} });
126
+ expect(result).toBeNull();
127
+ });
128
+
129
+ it('should return prompt result from Langfuse', async () => {
130
+ const mockGet = mockPromptGet;
131
+ mockGet.mockResolvedValue({ prompt: 'Hello {{name}}!', version: 5 });
132
+
133
+ const result = await plugin.getPrompt('greeting', { settings: validSettings });
134
+
135
+ expect(result).toEqual({ template: 'Hello {{name}}!', version: 5 });
136
+ });
137
+
138
+ it('should return null when prompt is not found (404)', async () => {
139
+ const mockGet = mockPromptGet;
140
+ mockGet.mockRejectedValue(new Error('Prompt not found (404)'));
141
+
142
+ const result = await plugin.getPrompt('missing.key', { settings: validSettings });
143
+ expect(result).toBeNull();
144
+ });
145
+
146
+ it('should return null on unexpected error', async () => {
147
+ const mockGet = mockPromptGet;
148
+ mockGet.mockRejectedValue(new Error('Network timeout'));
149
+
150
+ const result = await plugin.getPrompt('test.key', { settings: validSettings });
151
+ expect(result).toBeNull();
152
+ });
153
+
154
+ it('should pass label and cacheTtlSeconds to client', async () => {
155
+ const mockGet = mockPromptGet;
156
+ mockGet.mockResolvedValue({ prompt: 'test', version: 1 });
157
+
158
+ await plugin.getPrompt('test.key', {
159
+ settings: { ...validSettings, promptLabel: 'staging', cacheTtlSeconds: 60 }
160
+ });
161
+
162
+ expect(mockGet).toHaveBeenCalledWith('test.key', {
163
+ label: 'staging',
164
+ cacheTtlSeconds: 60,
165
+ type: 'text'
166
+ });
167
+ });
168
+ });
169
+
170
+ describe('lifecycle', () => {
171
+ it('should load and unload without errors', async () => {
172
+ await expect(plugin.onLoad(mockContext)).resolves.not.toThrow();
173
+ await expect(plugin.onUnload()).resolves.not.toThrow();
174
+ });
175
+ });
176
+
177
+ describe('healthCheck', () => {
178
+ it('should return healthy status', async () => {
179
+ const result = await plugin.healthCheck();
180
+ expect(result.status).toBe('healthy');
181
+ });
182
+ });
183
+
184
+ describe('getManifest', () => {
185
+ it('should return manifest with readme documenting prompt keys', () => {
186
+ const manifest = plugin.getManifest();
187
+ expect(manifest.id).toBe('langfuse');
188
+ expect(manifest.readme).toContain('standard-pipeline.domain-detection');
189
+ expect(manifest.readme).toContain('standard-pipeline.generation');
190
+ expect(manifest.defaultForCapabilities).toContain('prompt-provider');
191
+ });
192
+ });
193
+ });
package/src/index.ts ADDED
@@ -0,0 +1,2 @@
1
+ export { LangfusePlugin } from './langfuse.plugin.js';
2
+ export { LangfusePlugin as default } from './langfuse.plugin.js';