rlhf-feedback-loop 0.6.7 → 0.6.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -21
- package/adapters/chatgpt/openapi.yaml +124 -2
- package/adapters/mcp/server-stdio.js +74 -25
- package/bin/cli.js +34 -3
- package/openapi/openapi.yaml +124 -2
- package/package.json +14 -9
- package/scripts/billing.js +349 -89
- package/scripts/deploy-gcp.sh +19 -0
- package/scripts/prove-adapters.js +135 -5
- package/scripts/prove-subway-upgrades.js +28 -1
- package/src/api/server.js +82 -24
package/README.md
CHANGED
|
@@ -1,27 +1,39 @@
|
|
|
1
|
-
# RLHF Feedback Loop — Agentic Control Plane & Context Engineering Studio
|
|
1
|
+
# RLHF-Ready Feedback Loop — Agentic Control Plane & Context Engineering Studio
|
|
2
2
|
|
|
3
3
|
[](https://github.com/IgorGanapolsky/rlhf-feedback-loop/actions/workflows/ci.yml)
|
|
4
4
|
[](docs/ANTHROPIC_MARKETPLACE_STRATEGY.md)
|
|
5
5
|
[](docs/geo-strategy-for-ai-agents.md)
|
|
6
6
|
|
|
7
|
-
**Stop Vibe Coding. Start Context Engineering.** The RLHF Feedback Loop is the enterprise-grade **Agentic Control Plane** for AI workflows. We provide the operational layer to capture human preference signals, engineer high-density context packs, and enforce machine-readable guardrails to stop your agents from going "off-script."
|
|
7
|
+
**Stop Vibe Coding. Start Context Engineering.** The RLHF-Ready Feedback Loop is the enterprise-grade **Agentic Control Plane** for AI workflows. We provide the operational layer to capture human preference signals, engineer high-density context packs, and enforce machine-readable guardrails to stop your agents from going "off-script."
|
|
8
|
+
|
|
9
|
+
This product captures and structures human feedback data for optimization workflows. It is **RLHF-ready data infrastructure** (not an end-to-end reward-model + RL fine-tuning trainer by itself).
|
|
8
10
|
|
|
9
11
|
## True Plug-and-Play: Zero-Config Integration
|
|
10
12
|
|
|
11
13
|
The RLHF Feedback Loop is now a **Universal Agent Skill**. You can drop it into any repository without manual setup.
|
|
12
14
|
|
|
13
15
|
- **Zero-Config Discovery:** Automatically detects project context. If no local `.rlhf/` directory exists, it safely fallbacks to a project-scoped global store in `~/.rlhf/`.
|
|
14
|
-
- **Global Skill Installation:**
|
|
16
|
+
- **Global Skill Installation (Optional):** One-command installer is available if you want auto-detection.
|
|
15
17
|
- **Vibe-to-Verification (V2V):** Directly converts subjective "vibes" (thumbs up/down) into verifiable repository rules (`CLAUDE.md`).
|
|
16
18
|
|
|
17
|
-
### Quick Start (
|
|
19
|
+
### Quick Start (Stable MCP Commands)
|
|
20
|
+
|
|
21
|
+
Add the MCP server directly in your client config:
|
|
22
|
+
|
|
23
|
+
| Platform | Command |
|
|
24
|
+
|----------|---------|
|
|
25
|
+
| **Claude** | `claude mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
26
|
+
| **Codex** | `codex mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
27
|
+
| **Gemini** | `gemini mcp add rlhf "npx -y rlhf-feedback-loop serve"` |
|
|
28
|
+
| **Amp** | `amp mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
29
|
+
| **Cursor** | `cursor mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
30
|
+
|
|
31
|
+
Optional auto-installer:
|
|
18
32
|
|
|
19
33
|
```bash
|
|
20
|
-
npx rlhf-feedback-loop
|
|
34
|
+
npx add-mcp rlhf-feedback-loop
|
|
21
35
|
```
|
|
22
36
|
|
|
23
|
-
This will auto-detect your platforms (Claude, Codex, Gemini, Cursor) and install the RLHF skill globally.
|
|
24
|
-
|
|
25
37
|
- **Stop Regressions:** Automatically convert negative feedback into `CLAUDE.md` / `AGENTS.md` prevention rules.
|
|
26
38
|
- **Preference Data Engineering:** Capture high-density context (rubrics, guardrails, metadata) for DPO training.
|
|
27
39
|
- **Bayesian Scoring:** Use Thompson Sampling to handle evolving user preferences over time.
|
|
@@ -38,18 +50,7 @@ We are optimized for discovery by next-gen AI tools (Claude Code, Gemini CLI, Pe
|
|
|
38
50
|
|
|
39
51
|
## Get Started
|
|
40
52
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
| Platform | Install |
|
|
44
|
-
|----------|---------|
|
|
45
|
-
| **Claude** | `claude mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
46
|
-
| **Codex** | `codex mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
47
|
-
| **Gemini** | `gemini mcp add rlhf "npx -y rlhf-feedback-loop serve"` |
|
|
48
|
-
| **Amp** | `amp mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
49
|
-
| **Cursor** | `cursor mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
|
|
50
|
-
| **All at once** | `npx add-mcp rlhf-feedback-loop` |
|
|
51
|
-
|
|
52
|
-
That's it. Your agent can now capture feedback, recall past learnings mid-conversation, and block repeated mistakes. Run once per project — the MCP server starts automatically on each session.
|
|
53
|
+
Run one `mcp add` command for your client. The server starts on each session and can capture feedback, recall past learnings, and block repeated mistakes.
|
|
53
54
|
|
|
54
55
|
## How It Works
|
|
55
56
|
|
|
@@ -83,7 +84,7 @@ All data stored locally as **JSONL** files — fully transparent, fully portable
|
|
|
83
84
|
|
|
84
85
|
The open-source package is fully functional and free forever. Cloud Pro is for teams that don't want to self-host.
|
|
85
86
|
|
|
86
|
-
| | Open Source | Cloud Pro ($
|
|
87
|
+
| | Open Source | Cloud Pro ($49/mo) |
|
|
87
88
|
|---|---|---|
|
|
88
89
|
| Feedback capture | Local MCP server | Hosted HTTPS API |
|
|
89
90
|
| Storage | Your machine | Managed cloud |
|
|
@@ -93,7 +94,7 @@ The open-source package is fully functional and free forever. Cloud Pro is for t
|
|
|
93
94
|
| Support | GitHub Issues | Email |
|
|
94
95
|
| Uptime | You manage | We manage (99.9% SLA) |
|
|
95
96
|
|
|
96
|
-
[Get Cloud Pro](https://buy.stripe.com/bJe14neyU4r4f0leOD3sI02) | [Live API](https://rlhf-feedback-loop-710216278770.us-central1.run.app)
|
|
97
|
+
[Get Cloud Pro](https://buy.stripe.com/bJe14neyU4r4f0leOD3sI02) | [Live API](https://rlhf-feedback-loop-710216278770.us-central1.run.app) | [Verification Evidence](docs/VERIFICATION_EVIDENCE.md)
|
|
97
98
|
|
|
98
99
|
## Deep Dive
|
|
99
100
|
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
openapi: 3.1.0
|
|
2
2
|
info:
|
|
3
3
|
title: RLHF Feedback Loop API
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.2.0
|
|
5
5
|
description: |
|
|
6
6
|
Production API for feedback capture, schema-validated memory promotion,
|
|
7
7
|
prevention rule generation, and DPO export.
|
|
8
8
|
servers:
|
|
9
|
-
- url:
|
|
9
|
+
- url: https://rlhf-feedback-loop-710216278770.us-central1.run.app
|
|
10
10
|
security:
|
|
11
11
|
- bearerAuth: []
|
|
12
12
|
components:
|
|
@@ -80,6 +80,59 @@ components:
|
|
|
80
80
|
type: string
|
|
81
81
|
approved:
|
|
82
82
|
type: boolean
|
|
83
|
+
BillingCheckoutRequest:
|
|
84
|
+
type: object
|
|
85
|
+
properties:
|
|
86
|
+
successUrl:
|
|
87
|
+
type: string
|
|
88
|
+
format: uri
|
|
89
|
+
cancelUrl:
|
|
90
|
+
type: string
|
|
91
|
+
format: uri
|
|
92
|
+
customerEmail:
|
|
93
|
+
type: string
|
|
94
|
+
format: email
|
|
95
|
+
installId:
|
|
96
|
+
type: string
|
|
97
|
+
metadata:
|
|
98
|
+
type: object
|
|
99
|
+
additionalProperties:
|
|
100
|
+
type: string
|
|
101
|
+
BillingProvisionRequest:
|
|
102
|
+
type: object
|
|
103
|
+
required: [customerId]
|
|
104
|
+
properties:
|
|
105
|
+
customerId:
|
|
106
|
+
type: string
|
|
107
|
+
installId:
|
|
108
|
+
type: string
|
|
109
|
+
FunnelAnalyticsResponse:
|
|
110
|
+
type: object
|
|
111
|
+
properties:
|
|
112
|
+
totalEvents:
|
|
113
|
+
type: integer
|
|
114
|
+
stageCounts:
|
|
115
|
+
type: object
|
|
116
|
+
properties:
|
|
117
|
+
acquisition:
|
|
118
|
+
type: integer
|
|
119
|
+
activation:
|
|
120
|
+
type: integer
|
|
121
|
+
paid:
|
|
122
|
+
type: integer
|
|
123
|
+
eventCounts:
|
|
124
|
+
type: object
|
|
125
|
+
additionalProperties:
|
|
126
|
+
type: integer
|
|
127
|
+
conversionRates:
|
|
128
|
+
type: object
|
|
129
|
+
properties:
|
|
130
|
+
acquisitionToActivation:
|
|
131
|
+
type: number
|
|
132
|
+
activationToPaid:
|
|
133
|
+
type: number
|
|
134
|
+
acquisitionToPaid:
|
|
135
|
+
type: number
|
|
83
136
|
paths:
|
|
84
137
|
/healthz:
|
|
85
138
|
get:
|
|
@@ -113,6 +166,18 @@ paths:
|
|
|
113
166
|
description: Aggregated feedback statistics
|
|
114
167
|
'401':
|
|
115
168
|
description: Unauthorized
|
|
169
|
+
/v1/analytics/funnel:
|
|
170
|
+
get:
|
|
171
|
+
operationId: getFunnelAnalytics
|
|
172
|
+
responses:
|
|
173
|
+
'200':
|
|
174
|
+
description: Acquisition/activation/paid funnel metrics from append-only ledger
|
|
175
|
+
content:
|
|
176
|
+
application/json:
|
|
177
|
+
schema:
|
|
178
|
+
$ref: '#/components/schemas/FunnelAnalyticsResponse'
|
|
179
|
+
'401':
|
|
180
|
+
description: Unauthorized
|
|
116
181
|
/v1/intents/catalog:
|
|
117
182
|
get:
|
|
118
183
|
operationId: listIntentCatalog
|
|
@@ -290,3 +355,60 @@ paths:
|
|
|
290
355
|
description: Recent provenance events
|
|
291
356
|
'401':
|
|
292
357
|
description: Unauthorized
|
|
358
|
+
/v1/billing/checkout:
|
|
359
|
+
post:
|
|
360
|
+
operationId: createBillingCheckoutSession
|
|
361
|
+
security: []
|
|
362
|
+
requestBody:
|
|
363
|
+
required: false
|
|
364
|
+
content:
|
|
365
|
+
application/json:
|
|
366
|
+
schema:
|
|
367
|
+
$ref: '#/components/schemas/BillingCheckoutRequest'
|
|
368
|
+
responses:
|
|
369
|
+
'200':
|
|
370
|
+
description: Stripe checkout session created
|
|
371
|
+
/v1/billing/usage:
|
|
372
|
+
get:
|
|
373
|
+
operationId: getBillingUsage
|
|
374
|
+
responses:
|
|
375
|
+
'200':
|
|
376
|
+
description: Usage count for authenticated billing key
|
|
377
|
+
'401':
|
|
378
|
+
description: Unauthorized
|
|
379
|
+
/v1/billing/provision:
|
|
380
|
+
post:
|
|
381
|
+
operationId: provisionBillingKey
|
|
382
|
+
requestBody:
|
|
383
|
+
required: true
|
|
384
|
+
content:
|
|
385
|
+
application/json:
|
|
386
|
+
schema:
|
|
387
|
+
$ref: '#/components/schemas/BillingProvisionRequest'
|
|
388
|
+
responses:
|
|
389
|
+
'200':
|
|
390
|
+
description: API key provisioned
|
|
391
|
+
'400':
|
|
392
|
+
description: Missing required customerId
|
|
393
|
+
'401':
|
|
394
|
+
description: Unauthorized
|
|
395
|
+
'403':
|
|
396
|
+
description: Forbidden - requires static RLHF_API_KEY admin token
|
|
397
|
+
/v1/billing/webhook:
|
|
398
|
+
post:
|
|
399
|
+
operationId: stripeBillingWebhook
|
|
400
|
+
security: []
|
|
401
|
+
responses:
|
|
402
|
+
'200':
|
|
403
|
+
description: Webhook accepted
|
|
404
|
+
'400':
|
|
405
|
+
description: Invalid webhook signature or payload
|
|
406
|
+
/v1/billing/github-webhook:
|
|
407
|
+
post:
|
|
408
|
+
operationId: githubMarketplaceWebhook
|
|
409
|
+
security: []
|
|
410
|
+
responses:
|
|
411
|
+
'200':
|
|
412
|
+
description: Webhook accepted
|
|
413
|
+
'400':
|
|
414
|
+
description: Invalid webhook signature or payload
|
|
@@ -556,32 +556,70 @@ async function handleRequest(message) {
|
|
|
556
556
|
throw new Error(`Unsupported method: ${message.method}`);
|
|
557
557
|
}
|
|
558
558
|
|
|
559
|
-
function writeMessage(payload) {
|
|
559
|
+
function writeMessage(payload, transport = 'framed') {
|
|
560
560
|
const json = JSON.stringify(payload);
|
|
561
|
+
if (transport === 'ndjson') {
|
|
562
|
+
process.stdout.write(`${json}\n`);
|
|
563
|
+
return;
|
|
564
|
+
}
|
|
561
565
|
process.stdout.write(`Content-Length: ${Buffer.byteLength(json, 'utf8')}\r\n\r\n${json}`);
|
|
562
566
|
}
|
|
563
567
|
|
|
568
|
+
function parseWithTransport(raw, transport) {
|
|
569
|
+
try {
|
|
570
|
+
return JSON.parse(raw);
|
|
571
|
+
} catch (err) {
|
|
572
|
+
err.transport = transport;
|
|
573
|
+
throw err;
|
|
574
|
+
}
|
|
575
|
+
}
|
|
576
|
+
|
|
564
577
|
let buffer = Buffer.alloc(0);
|
|
578
|
+
let stdioStarted = false;
|
|
579
|
+
|
|
580
|
+
function hasContentLengthPrefix() {
|
|
581
|
+
if (buffer.length === 0) return false;
|
|
582
|
+
const probe = buffer.slice(0, Math.min(buffer.length, 32)).toString('utf8').toLowerCase();
|
|
583
|
+
return 'content-length:'.startsWith(probe) || probe.startsWith('content-length:');
|
|
584
|
+
}
|
|
565
585
|
|
|
566
586
|
function tryReadMessage() {
|
|
567
|
-
const
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
587
|
+
const headerEndCrLf = buffer.indexOf('\r\n\r\n');
|
|
588
|
+
const headerEndLf = buffer.indexOf('\n\n');
|
|
589
|
+
const hasFramedHeader = headerEndCrLf !== -1 || headerEndLf !== -1;
|
|
590
|
+
|
|
591
|
+
if (hasFramedHeader) {
|
|
592
|
+
const useCrLf = headerEndCrLf !== -1 && (headerEndLf === -1 || headerEndCrLf < headerEndLf);
|
|
593
|
+
const headerEnd = useCrLf ? headerEndCrLf : headerEndLf;
|
|
594
|
+
const separatorLength = useCrLf ? 4 : 2;
|
|
595
|
+
const headerRaw = buffer.slice(0, headerEnd).toString('utf8');
|
|
596
|
+
const match = headerRaw.match(/Content-Length:\s*(\d+)/i);
|
|
597
|
+
if (!match) {
|
|
598
|
+
buffer = buffer.slice(headerEnd + separatorLength);
|
|
599
|
+
return null;
|
|
600
|
+
}
|
|
601
|
+
|
|
602
|
+
const length = Number(match[1]);
|
|
603
|
+
const totalSize = headerEnd + separatorLength + length;
|
|
604
|
+
if (buffer.length < totalSize) return null;
|
|
605
|
+
|
|
606
|
+
const body = buffer.slice(headerEnd + separatorLength, totalSize).toString('utf8');
|
|
607
|
+
buffer = buffer.slice(totalSize);
|
|
608
|
+
|
|
609
|
+
return { message: parseWithTransport(body, 'framed'), transport: 'framed' };
|
|
575
610
|
}
|
|
576
611
|
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
612
|
+
// Codex MCP client currently sends newline-delimited JSON during startup.
|
|
613
|
+
if (hasContentLengthPrefix()) return null;
|
|
614
|
+
|
|
615
|
+
const newlineIndex = buffer.indexOf('\n');
|
|
616
|
+
if (newlineIndex === -1) return null;
|
|
580
617
|
|
|
581
|
-
const
|
|
582
|
-
buffer = buffer.slice(
|
|
618
|
+
const line = buffer.slice(0, newlineIndex).toString('utf8').trim();
|
|
619
|
+
buffer = buffer.slice(newlineIndex + 1);
|
|
620
|
+
if (!line) return null;
|
|
583
621
|
|
|
584
|
-
return
|
|
622
|
+
return { message: parseWithTransport(line, 'ndjson'), transport: 'ndjson' };
|
|
585
623
|
}
|
|
586
624
|
|
|
587
625
|
async function onData(chunk) {
|
|
@@ -590,39 +628,50 @@ async function onData(chunk) {
|
|
|
590
628
|
while (true) {
|
|
591
629
|
const message = tryReadMessage();
|
|
592
630
|
if (!message) return;
|
|
631
|
+
const envelope = message;
|
|
632
|
+
const request = envelope.message;
|
|
633
|
+
const transport = envelope.transport;
|
|
593
634
|
|
|
594
|
-
if (!Object.prototype.hasOwnProperty.call(
|
|
635
|
+
if (!Object.prototype.hasOwnProperty.call(request, 'id')) {
|
|
595
636
|
continue;
|
|
596
637
|
}
|
|
597
638
|
|
|
598
639
|
try {
|
|
599
|
-
const result = await handleRequest(
|
|
600
|
-
writeMessage({ jsonrpc: '2.0', id:
|
|
640
|
+
const result = await handleRequest(request);
|
|
641
|
+
writeMessage({ jsonrpc: '2.0', id: request.id, result }, transport);
|
|
601
642
|
} catch (err) {
|
|
602
643
|
writeMessage({
|
|
603
644
|
jsonrpc: '2.0',
|
|
604
|
-
id:
|
|
645
|
+
id: request.id,
|
|
605
646
|
error: {
|
|
606
647
|
code: -32603,
|
|
607
648
|
message: err.message || 'Internal error',
|
|
608
649
|
},
|
|
609
|
-
});
|
|
650
|
+
}, transport);
|
|
610
651
|
}
|
|
611
652
|
}
|
|
612
653
|
}
|
|
613
654
|
|
|
655
|
+
function startStdioServer() {
|
|
656
|
+
if (stdioStarted) return;
|
|
657
|
+
stdioStarted = true;
|
|
658
|
+
process.stdin.on('data', (chunk) => {
|
|
659
|
+
onData(chunk).catch((err) => {
|
|
660
|
+
const transport = err && err.transport === 'ndjson' ? 'ndjson' : 'framed';
|
|
661
|
+
writeMessage({ jsonrpc: '2.0', id: null, error: { code: -32603, message: err.message } }, transport);
|
|
662
|
+
});
|
|
663
|
+
});
|
|
664
|
+
}
|
|
665
|
+
|
|
614
666
|
module.exports = {
|
|
615
667
|
TOOLS,
|
|
616
668
|
handleRequest,
|
|
617
669
|
callTool,
|
|
618
670
|
resolveSafePath,
|
|
619
671
|
SAFE_DATA_DIR,
|
|
672
|
+
startStdioServer,
|
|
620
673
|
};
|
|
621
674
|
|
|
622
675
|
if (require.main === module) {
|
|
623
|
-
|
|
624
|
-
onData(chunk).catch((err) => {
|
|
625
|
-
writeMessage({ jsonrpc: '2.0', id: null, error: { code: -32603, message: err.message } });
|
|
626
|
-
});
|
|
627
|
-
});
|
|
676
|
+
startStdioServer();
|
|
628
677
|
}
|
package/bin/cli.js
CHANGED
|
@@ -14,6 +14,7 @@
|
|
|
14
14
|
|
|
15
15
|
const fs = require('fs');
|
|
16
16
|
const path = require('path');
|
|
17
|
+
const crypto = require('crypto');
|
|
17
18
|
const { execSync } = require('child_process');
|
|
18
19
|
|
|
19
20
|
const COMMAND = process.argv[2];
|
|
@@ -124,6 +125,7 @@ function setupCursor() {
|
|
|
124
125
|
|
|
125
126
|
function init() {
|
|
126
127
|
const rlhfDir = path.join(CWD, '.rlhf');
|
|
128
|
+
const configPath = path.join(rlhfDir, 'config.json');
|
|
127
129
|
|
|
128
130
|
if (!fs.existsSync(rlhfDir)) {
|
|
129
131
|
fs.mkdirSync(rlhfDir, { recursive: true });
|
|
@@ -132,15 +134,28 @@ function init() {
|
|
|
132
134
|
console.log('.rlhf/ already exists — updating config');
|
|
133
135
|
}
|
|
134
136
|
|
|
137
|
+
let existingInstallId = null;
|
|
138
|
+
if (fs.existsSync(configPath)) {
|
|
139
|
+
try {
|
|
140
|
+
const existingConfig = JSON.parse(fs.readFileSync(configPath, 'utf8'));
|
|
141
|
+
if (existingConfig && typeof existingConfig.installId === 'string' && existingConfig.installId.trim()) {
|
|
142
|
+
existingInstallId = existingConfig.installId.trim();
|
|
143
|
+
}
|
|
144
|
+
} catch (_) {
|
|
145
|
+
// Ignore invalid existing config and write a fresh one below.
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
|
|
135
149
|
const config = {
|
|
136
150
|
version: pkgVersion(),
|
|
137
151
|
apiUrl: process.env.RLHF_API_URL || 'http://localhost:3000',
|
|
138
152
|
logPath: '.rlhf/feedback-log.jsonl',
|
|
139
153
|
memoryPath: '.rlhf/memory-log.jsonl',
|
|
154
|
+
installId: existingInstallId || crypto.randomUUID(),
|
|
140
155
|
createdAt: new Date().toISOString(),
|
|
141
156
|
};
|
|
142
157
|
|
|
143
|
-
fs.writeFileSync(
|
|
158
|
+
fs.writeFileSync(configPath, JSON.stringify(config, null, 2) + '\n');
|
|
144
159
|
console.log('Wrote .rlhf/config.json');
|
|
145
160
|
|
|
146
161
|
// Always create .mcp.json (project-level MCP config used by Claude, Codex, Cursor)
|
|
@@ -189,6 +204,22 @@ function init() {
|
|
|
189
204
|
console.log('');
|
|
190
205
|
console.log(`rlhf-feedback-loop v${pkgVersion()} initialized.`);
|
|
191
206
|
console.log('Run: npx rlhf-feedback-loop help');
|
|
207
|
+
|
|
208
|
+
try {
|
|
209
|
+
const { appendFunnelEvent } = require(path.join(PKG_ROOT, 'scripts', 'billing'));
|
|
210
|
+
appendFunnelEvent({
|
|
211
|
+
stage: 'acquisition',
|
|
212
|
+
event: 'cli_init_completed',
|
|
213
|
+
evidence: 'cli_init_completed',
|
|
214
|
+
installId: config.installId,
|
|
215
|
+
metadata: {
|
|
216
|
+
cwd: CWD,
|
|
217
|
+
version: config.version,
|
|
218
|
+
},
|
|
219
|
+
});
|
|
220
|
+
} catch (_) {
|
|
221
|
+
// Avoid failing init if telemetry write cannot be performed.
|
|
222
|
+
}
|
|
192
223
|
}
|
|
193
224
|
|
|
194
225
|
function capture() {
|
|
@@ -351,7 +382,8 @@ function serve() {
|
|
|
351
382
|
|
|
352
383
|
// Start MCP server over stdio
|
|
353
384
|
const mcpServer = path.join(PKG_ROOT, 'adapters', 'mcp', 'server-stdio.js');
|
|
354
|
-
require(mcpServer);
|
|
385
|
+
const { startStdioServer } = require(mcpServer);
|
|
386
|
+
startStdioServer();
|
|
355
387
|
}
|
|
356
388
|
|
|
357
389
|
function install() {
|
|
@@ -442,7 +474,6 @@ switch (COMMAND) {
|
|
|
442
474
|
prove();
|
|
443
475
|
break;
|
|
444
476
|
case 'start-api':
|
|
445
|
-
case 'serve':
|
|
446
477
|
startApi();
|
|
447
478
|
break;
|
|
448
479
|
case 'help':
|
package/openapi/openapi.yaml
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
openapi: 3.1.0
|
|
2
2
|
info:
|
|
3
3
|
title: RLHF Feedback Loop API
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.2.0
|
|
5
5
|
description: |
|
|
6
6
|
Production API for feedback capture, schema-validated memory promotion,
|
|
7
7
|
prevention rule generation, and DPO export.
|
|
8
8
|
servers:
|
|
9
|
-
- url:
|
|
9
|
+
- url: https://rlhf-feedback-loop-710216278770.us-central1.run.app
|
|
10
10
|
security:
|
|
11
11
|
- bearerAuth: []
|
|
12
12
|
components:
|
|
@@ -80,6 +80,59 @@ components:
|
|
|
80
80
|
type: string
|
|
81
81
|
approved:
|
|
82
82
|
type: boolean
|
|
83
|
+
BillingCheckoutRequest:
|
|
84
|
+
type: object
|
|
85
|
+
properties:
|
|
86
|
+
successUrl:
|
|
87
|
+
type: string
|
|
88
|
+
format: uri
|
|
89
|
+
cancelUrl:
|
|
90
|
+
type: string
|
|
91
|
+
format: uri
|
|
92
|
+
customerEmail:
|
|
93
|
+
type: string
|
|
94
|
+
format: email
|
|
95
|
+
installId:
|
|
96
|
+
type: string
|
|
97
|
+
metadata:
|
|
98
|
+
type: object
|
|
99
|
+
additionalProperties:
|
|
100
|
+
type: string
|
|
101
|
+
BillingProvisionRequest:
|
|
102
|
+
type: object
|
|
103
|
+
required: [customerId]
|
|
104
|
+
properties:
|
|
105
|
+
customerId:
|
|
106
|
+
type: string
|
|
107
|
+
installId:
|
|
108
|
+
type: string
|
|
109
|
+
FunnelAnalyticsResponse:
|
|
110
|
+
type: object
|
|
111
|
+
properties:
|
|
112
|
+
totalEvents:
|
|
113
|
+
type: integer
|
|
114
|
+
stageCounts:
|
|
115
|
+
type: object
|
|
116
|
+
properties:
|
|
117
|
+
acquisition:
|
|
118
|
+
type: integer
|
|
119
|
+
activation:
|
|
120
|
+
type: integer
|
|
121
|
+
paid:
|
|
122
|
+
type: integer
|
|
123
|
+
eventCounts:
|
|
124
|
+
type: object
|
|
125
|
+
additionalProperties:
|
|
126
|
+
type: integer
|
|
127
|
+
conversionRates:
|
|
128
|
+
type: object
|
|
129
|
+
properties:
|
|
130
|
+
acquisitionToActivation:
|
|
131
|
+
type: number
|
|
132
|
+
activationToPaid:
|
|
133
|
+
type: number
|
|
134
|
+
acquisitionToPaid:
|
|
135
|
+
type: number
|
|
83
136
|
paths:
|
|
84
137
|
/healthz:
|
|
85
138
|
get:
|
|
@@ -113,6 +166,18 @@ paths:
|
|
|
113
166
|
description: Aggregated feedback statistics
|
|
114
167
|
'401':
|
|
115
168
|
description: Unauthorized
|
|
169
|
+
/v1/analytics/funnel:
|
|
170
|
+
get:
|
|
171
|
+
operationId: getFunnelAnalytics
|
|
172
|
+
responses:
|
|
173
|
+
'200':
|
|
174
|
+
description: Acquisition/activation/paid funnel metrics from append-only ledger
|
|
175
|
+
content:
|
|
176
|
+
application/json:
|
|
177
|
+
schema:
|
|
178
|
+
$ref: '#/components/schemas/FunnelAnalyticsResponse'
|
|
179
|
+
'401':
|
|
180
|
+
description: Unauthorized
|
|
116
181
|
/v1/intents/catalog:
|
|
117
182
|
get:
|
|
118
183
|
operationId: listIntentCatalog
|
|
@@ -290,3 +355,60 @@ paths:
|
|
|
290
355
|
description: Recent provenance events
|
|
291
356
|
'401':
|
|
292
357
|
description: Unauthorized
|
|
358
|
+
/v1/billing/checkout:
|
|
359
|
+
post:
|
|
360
|
+
operationId: createBillingCheckoutSession
|
|
361
|
+
security: []
|
|
362
|
+
requestBody:
|
|
363
|
+
required: false
|
|
364
|
+
content:
|
|
365
|
+
application/json:
|
|
366
|
+
schema:
|
|
367
|
+
$ref: '#/components/schemas/BillingCheckoutRequest'
|
|
368
|
+
responses:
|
|
369
|
+
'200':
|
|
370
|
+
description: Stripe checkout session created
|
|
371
|
+
/v1/billing/usage:
|
|
372
|
+
get:
|
|
373
|
+
operationId: getBillingUsage
|
|
374
|
+
responses:
|
|
375
|
+
'200':
|
|
376
|
+
description: Usage count for authenticated billing key
|
|
377
|
+
'401':
|
|
378
|
+
description: Unauthorized
|
|
379
|
+
/v1/billing/provision:
|
|
380
|
+
post:
|
|
381
|
+
operationId: provisionBillingKey
|
|
382
|
+
requestBody:
|
|
383
|
+
required: true
|
|
384
|
+
content:
|
|
385
|
+
application/json:
|
|
386
|
+
schema:
|
|
387
|
+
$ref: '#/components/schemas/BillingProvisionRequest'
|
|
388
|
+
responses:
|
|
389
|
+
'200':
|
|
390
|
+
description: API key provisioned
|
|
391
|
+
'400':
|
|
392
|
+
description: Missing required customerId
|
|
393
|
+
'401':
|
|
394
|
+
description: Unauthorized
|
|
395
|
+
'403':
|
|
396
|
+
description: Forbidden - requires static RLHF_API_KEY admin token
|
|
397
|
+
/v1/billing/webhook:
|
|
398
|
+
post:
|
|
399
|
+
operationId: stripeBillingWebhook
|
|
400
|
+
security: []
|
|
401
|
+
responses:
|
|
402
|
+
'200':
|
|
403
|
+
description: Webhook accepted
|
|
404
|
+
'400':
|
|
405
|
+
description: Invalid webhook signature or payload
|
|
406
|
+
/v1/billing/github-webhook:
|
|
407
|
+
post:
|
|
408
|
+
operationId: githubMarketplaceWebhook
|
|
409
|
+
security: []
|
|
410
|
+
responses:
|
|
411
|
+
'200':
|
|
412
|
+
description: Webhook accepted
|
|
413
|
+
'400':
|
|
414
|
+
description: Invalid webhook signature or payload
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "rlhf-feedback-loop",
|
|
3
|
-
"version": "0.6.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.6.9",
|
|
4
|
+
"description": "RLHF-ready human feedback capture and DPO data pipeline for AI agents. Optimize agentic reliability with Feedback-Driven Development (FDD): capture preference signals, enforce guardrails, and export training pairs for downstream optimization.",
|
|
5
5
|
"homepage": "https://github.com/IgorGanapolsky/rlhf-feedback-loop#readme",
|
|
6
6
|
"repository": {
|
|
7
7
|
"type": "git",
|
|
@@ -34,7 +34,7 @@
|
|
|
34
34
|
"test:loop": "node scripts/feedback-loop.js --test",
|
|
35
35
|
"test:dpo": "node scripts/export-dpo-pairs.js --test",
|
|
36
36
|
"test:api": "node --test tests/api-server.test.js tests/api-auth-config.test.js tests/mcp-server.test.js tests/adapters.test.js tests/openapi-parity.test.js tests/budget-guard.test.js tests/contextfs.test.js tests/mcp-policy.test.js tests/subagent-profiles.test.js tests/intent-router.test.js tests/rubric-engine.test.js tests/self-healing-check.test.js tests/self-heal.test.js tests/feedback-schema.test.js tests/thompson-sampling.test.js tests/feedback-sequences.test.js tests/diversity-tracking.test.js tests/vector-store.test.js tests/feedback-attribution.test.js tests/hybrid-feedback-context.test.js tests/loop-closure.test.js tests/code-reasoning.test.js tests/feedback-loop.test.js tests/feedback-inbox-read.test.js tests/feedback-to-memory.test.js",
|
|
37
|
-
"test:proof": "node --test tests/prove-adapters.test.js tests/prove-automation.test.js tests/prove-attribution.test.js tests/prove-lancedb.test.js tests/prove-data-quality.test.js tests/prove-intelligence.test.js tests/prove-loop-closure.test.js tests/prove-subway-upgrades.test.js tests/prove-training-export.test.js",
|
|
37
|
+
"test:proof": "node --test --test-concurrency=1 tests/prove-adapters.test.js tests/prove-automation.test.js tests/prove-attribution.test.js tests/prove-lancedb.test.js tests/prove-data-quality.test.js tests/prove-intelligence.test.js tests/prove-loop-closure.test.js tests/prove-subway-upgrades.test.js tests/prove-training-export.test.js",
|
|
38
38
|
"test:rlaif": "node --test tests/rlaif-self-audit.test.js tests/dpo-optimizer.test.js tests/meta-policy.test.js",
|
|
39
39
|
"test:attribution": "node --test tests/feedback-attribution.test.js tests/hybrid-feedback-context.test.js",
|
|
40
40
|
"test:quality": "node --test tests/validate-feedback.test.js",
|
|
@@ -107,13 +107,18 @@
|
|
|
107
107
|
"developer-tools"
|
|
108
108
|
],
|
|
109
109
|
"license": "MIT",
|
|
110
|
+
"funding": {
|
|
111
|
+
"type": "individual",
|
|
112
|
+
"url": "https://buy.stripe.com/bJe14neyU4r4f0leOD3sI02"
|
|
113
|
+
},
|
|
114
|
+
"engines": {
|
|
115
|
+
"node": ">=18.18.0"
|
|
116
|
+
},
|
|
110
117
|
"dependencies": {
|
|
118
|
+
"@huggingface/transformers": "^3.8.1",
|
|
119
|
+
"@lancedb/lancedb": "^0.26.2",
|
|
111
120
|
"apache-arrow": "^18.1.0",
|
|
112
|
-
"stripe": "^20.4.
|
|
121
|
+
"stripe": "^20.4.1"
|
|
113
122
|
},
|
|
114
|
-
"mcpName": "io.github.IgorGanapolsky/rlhf-feedback-loop"
|
|
115
|
-
"optionalDependencies": {
|
|
116
|
-
"@lancedb/lancedb": "^0.26.2",
|
|
117
|
-
"@huggingface/transformers": "^3.8.1"
|
|
118
|
-
}
|
|
123
|
+
"mcpName": "io.github.IgorGanapolsky/rlhf-feedback-loop"
|
|
119
124
|
}
|