rlhf-feedback-loop 0.6.8 → 0.6.10

package/README.md

@@ -1,56 +1,61 @@
-# Agentic Feedback Studio — The Veto Layer & RLHF-Ready Dataset Engine
+# RLHF-Ready Feedback Loop — Agentic Control Plane & Context Engineering Studio
 
 [![CI](https://github.com/IgorGanapolsky/rlhf-feedback-loop/actions/workflows/ci.yml/badge.svg)](https://github.com/IgorGanapolsky/rlhf-feedback-loop/actions/workflows/ci.yml)
 [![Marketplace Ready](https://img.shields.io/badge/Anthropic_Marketplace-Ready-blue)](docs/ANTHROPIC_MARKETPLACE_STRATEGY.md)
-[![Veto Powered](https://img.shields.io/badge/Governance-Veto_Layer-red)](docs/VERIFICATION_EVIDENCE.md)
+[![GEO Optimized](https://img.shields.io/badge/GEO-optimized-orange)](docs/geo-strategy-for-ai-agents.md)
 
-**The operational layer for high-density preference data.** Stop vibe-coding and start context engineering. The Agentic Feedback Studio provides the **Veto Layer** for AI workflows, capturing human feedback to generate **RLHF-ready datasets** and enforce kernel-level guardrails.
+**Stop Vibe Coding. Start Context Engineering.** The RLHF-Ready Feedback Loop is the enterprise-grade **Agentic Control Plane** for AI workflows. We provide the operational layer to capture human preference signals, engineer high-density context packs, and enforce machine-readable guardrails to stop your agents from going "off-script."
 
-## Why This Matters: From Vibes to Verification (V2V)
-
-Most AI agents run on "vibes." We provide the infrastructure to convert those vibes into **Hard Evidence** for continuous improvement.
-
-- **Veto Layer (Governance):** Convert subjective user feedback into non-bypassable architectural constraints (`CLAUDE.md`).
-- **RLHF-Ready Datasets:** Automatically generate high-density DPO (Direct Preference Optimization) pairs from real-world agent interactions.
-- **Online Bayesian Reward Estimation:** Uses Thompson Sampling to model user preferences in real-time, providing a local "Reward Signal" without heavy training.
+This product captures and structures human feedback data for optimization workflows. It is **RLHF-ready data infrastructure** (not an end-to-end reward-model + RL fine-tuning trainer by itself).
 
 ## True Plug-and-Play: Zero-Config Integration
 
-The Feedback Studio is a **Universal Agent Skill**. You can drop it into any repository without manual setup.
+The RLHF Feedback Loop is now a **Universal Agent Skill**. You can drop it into any repository without manual setup.
 
 - **Zero-Config Discovery:** Automatically detects project context. If no local `.rlhf/` directory exists, it safely fallbacks to a project-scoped global store in `~/.rlhf/`.
-- **Global Skill Installation:** Run one command to make the Studio available to all your agents across all projects.
+- **Global Skill Installation (Optional):** One-command installer is available if you want auto-detection.
+- **Vibe-to-Verification (V2V):** Directly converts subjective "vibes" (thumbs up/down) into verifiable repository rules (`CLAUDE.md`).
+
+### Quick Start (Stable MCP Commands)
+
+Add the MCP server directly in your client config:
+
+| Platform | Command |
+|----------|---------|
+| **Claude** | `claude mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
+| **Codex** | `codex mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
+| **Gemini** | `gemini mcp add rlhf "npx -y rlhf-feedback-loop serve"` |
+| **Amp** | `amp mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
+| **Cursor** | `cursor mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
 
-### Quick Start (One Command)
+Optional auto-installer:
 
 ```bash
-npx rlhf-feedback-loop install
+npx add-mcp rlhf-feedback-loop
 ```
 
-This will auto-detect your platforms (Claude, Codex, Gemini, Cursor) and install the feedback skill globally.
+- **Stop Regressions:** Automatically convert negative feedback into `CLAUDE.md` / `AGENTS.md` prevention rules.
+- **Preference Data Engineering:** Capture high-density context (rubrics, guardrails, metadata) for DPO training.
+- **Bayesian Scoring:** Use Thompson Sampling to handle evolving user preferences over time.
 
 ## Use Cases
 
 - **Automated Code Reviews:** Capture PR feedback to enforce team-specific style guides autonomously.
-- **Self-Healing Multi-Agent Systems:** Share Veto rules across a swarm of agents to avoid systemic bottlenecks.
-- **DPO Dataset Engineering:** Collect proprietary preference data to fine-tune smaller, faster models that perform like GPT-4 on your specific codebase.
+- **Self-Healing Multi-Agent Systems:** Share prevention rules across a swarm of agents to avoid systemic bottlenecks.
+- **Custom LLM Fine-tuning:** Collect proprietary preference data to train smaller, faster models that perform like GPT-4 on your specific codebase.
 
-## Get Started
+## Generative Engine Optimization (GEO)
 
-One command. Pick your platform:
+We are optimized for discovery by next-gen AI tools (Claude Code, Gemini CLI, Perplexity). See our [GEO Strategy](docs/geo-strategy-for-ai-agents.md) for how we use **Context-First Publishing** to remain the first-choice recommendation for agent reliability.
 
-| Platform | Install |
-|----------|---------|
-| **Claude** | `claude mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
-| **Codex** | `codex mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
-| **Gemini** | `gemini mcp add rlhf "npx -y rlhf-feedback-loop serve"` |
-| **Amp** | `amp mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
-| **Cursor** | `cursor mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
+## Get Started
+
+Run one `mcp add` command for your client. The server starts on each session and can capture feedback, recall past learnings, and block repeated mistakes.
 
 ## How It Works
 
 ```
-Subjective Signal (Vibe)
+Thumbs up/down
       |
       v
   Capture → JSONL log
@@ -63,14 +68,14 @@ Subjective Signal (Vibe)
  Good    Bad
   |       |
   v       v
-Learn   Veto Layer (Rule)
+Learn   Prevention rule
   |       |
   v       v
 LanceDB   ShieldCortex
 vectors   context packs
   |
   v
-DPO export → RLHF / Fine-tune your model
+DPO export → fine-tune your model
 ```
 
 All data stored locally as **JSONL** files — fully transparent, fully portable, no vendor lock-in. **LanceDB** indexes memories as vector embeddings for semantic search. **ShieldCortex** assembles context packs so your agent starts each task informed.
@@ -79,15 +84,24 @@ All data stored locally as **JSONL** files — fully transparent, fully portable
 
 The open-source package is fully functional and free forever. Cloud Pro is for teams that don't want to self-host.
 
-| | Open Source | Cloud Pro ($10/mo) |
+| | Open Source | Cloud Pro ($49/mo) |
 |---|---|---|
 | Feedback capture | Local MCP server | Hosted HTTPS API |
 | Storage | Your machine | Managed cloud |
 | DPO export | CLI command | API endpoint |
 | Setup | `mcp add` one-liner | Provisioned API key |
 | Team sharing | Manual (share JSONL) | Built-in (shared API) |
+| Support | GitHub Issues | Email |
+| Uptime | You manage | We manage (99.9% SLA) |
+
+[Get Cloud Pro](https://buy.stripe.com/bJe14neyU4r4f0leOD3sI02) | [Live API](https://rlhf-feedback-loop-710216278770.us-central1.run.app) | [Verification Evidence](docs/VERIFICATION_EVIDENCE.md)
+
+## Deep Dive
 
-[Get Cloud Pro](https://buy.stripe.com/bJe14neyU4r4f0leOD3sI02) | [Live API](https://rlhf-feedback-loop-710216278770.us-central1.run.app)
+- [API Reference](openapi/openapi.yaml) — full OpenAPI spec
+- [Context Engine](docs/CONTEXTFS.md) — multi-agent memory orchestration
+- [Autonomous GitOps](docs/AUTONOMOUS_GITOPS.md) — self-healing CI/CD
+- [Contributing](CONTRIBUTING.md)
 
 ## License
 

package/adapters/chatgpt/openapi.yaml

@@ -1,12 +1,12 @@
 openapi: 3.1.0
 info:
   title: RLHF Feedback Loop API
-  version: 1.1.0
+  version: 1.2.0
   description: |
     Production API for feedback capture, schema-validated memory promotion,
     prevention rule generation, and DPO export.
 servers:
-  - url: http://localhost:8787
+  - url: https://rlhf-feedback-loop-710216278770.us-central1.run.app
 security:
   - bearerAuth: []
 components:
@@ -80,6 +80,59 @@ components:
           type: string
         approved:
           type: boolean
+    BillingCheckoutRequest:
+      type: object
+      properties:
+        successUrl:
+          type: string
+          format: uri
+        cancelUrl:
+          type: string
+          format: uri
+        customerEmail:
+          type: string
+          format: email
+        installId:
+          type: string
+        metadata:
+          type: object
+          additionalProperties:
+            type: string
+    BillingProvisionRequest:
+      type: object
+      required: [customerId]
+      properties:
+        customerId:
+          type: string
+        installId:
+          type: string
+    FunnelAnalyticsResponse:
+      type: object
+      properties:
+        totalEvents:
+          type: integer
+        stageCounts:
+          type: object
+          properties:
+            acquisition:
+              type: integer
+            activation:
+              type: integer
+            paid:
+              type: integer
+        eventCounts:
+          type: object
+          additionalProperties:
+            type: integer
+        conversionRates:
+          type: object
+          properties:
+            acquisitionToActivation:
+              type: number
+            activationToPaid:
+              type: number
+            acquisitionToPaid:
+              type: number
 paths:
   /healthz:
     get:
@@ -113,6 +166,18 @@ paths:
           description: Aggregated feedback statistics
         '401':
           description: Unauthorized
+  /v1/analytics/funnel:
+    get:
+      operationId: getFunnelAnalytics
+      responses:
+        '200':
+          description: Acquisition/activation/paid funnel metrics from append-only ledger
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/FunnelAnalyticsResponse'
+        '401':
+          description: Unauthorized
   /v1/intents/catalog:
     get:
       operationId: listIntentCatalog
@@ -290,3 +355,60 @@ paths:
           description: Recent provenance events
         '401':
           description: Unauthorized
+  /v1/billing/checkout:
+    post:
+      operationId: createBillingCheckoutSession
+      security: []
+      requestBody:
+        required: false
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/BillingCheckoutRequest'
+      responses:
+        '200':
+          description: Stripe checkout session created
+  /v1/billing/usage:
+    get:
+      operationId: getBillingUsage
+      responses:
+        '200':
+          description: Usage count for authenticated billing key
+        '401':
+          description: Unauthorized
+  /v1/billing/provision:
+    post:
+      operationId: provisionBillingKey
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/BillingProvisionRequest'
+      responses:
+        '200':
+          description: API key provisioned
+        '400':
+          description: Missing required customerId
+        '401':
+          description: Unauthorized
+        '403':
+          description: Forbidden - requires static RLHF_API_KEY admin token
+  /v1/billing/webhook:
+    post:
+      operationId: stripeBillingWebhook
+      security: []
+      responses:
+        '200':
+          description: Webhook accepted
+        '400':
+          description: Invalid webhook signature or payload
+  /v1/billing/github-webhook:
+    post:
+      operationId: githubMarketplaceWebhook
+      security: []
+      responses:
+        '200':
+          description: Webhook accepted
+        '400':
+          description: Invalid webhook signature or payload

package/adapters/mcp/server-stdio.js

@@ -556,32 +556,70 @@ async function handleRequest(message) {
   throw new Error(`Unsupported method: ${message.method}`);
 }
 
-function writeMessage(payload) {
+function writeMessage(payload, transport = 'framed') {
   const json = JSON.stringify(payload);
+  if (transport === 'ndjson') {
+    process.stdout.write(`${json}\n`);
+    return;
+  }
   process.stdout.write(`Content-Length: ${Buffer.byteLength(json, 'utf8')}\r\n\r\n${json}`);
 }
 
+function parseWithTransport(raw, transport) {
+  try {
+    return JSON.parse(raw);
+  } catch (err) {
+    err.transport = transport;
+    throw err;
+  }
+}
+
 let buffer = Buffer.alloc(0);
+let stdioStarted = false;
+
+function hasContentLengthPrefix() {
+  if (buffer.length === 0) return false;
+  const probe = buffer.slice(0, Math.min(buffer.length, 32)).toString('utf8').toLowerCase();
+  return 'content-length:'.startsWith(probe) || probe.startsWith('content-length:');
+}
 
 function tryReadMessage() {
-  const headerEnd = buffer.indexOf('\r\n\r\n');
-  if (headerEnd === -1) return null;
-
-  const headerRaw = buffer.slice(0, headerEnd).toString('utf8');
-  const match = headerRaw.match(/Content-Length:\s*(\d+)/i);
-  if (!match) {
-    buffer = buffer.slice(headerEnd + 4);
-    return null;
+  const headerEndCrLf = buffer.indexOf('\r\n\r\n');
+  const headerEndLf = buffer.indexOf('\n\n');
+  const hasFramedHeader = headerEndCrLf !== -1 || headerEndLf !== -1;
+
+  if (hasFramedHeader) {
+    const useCrLf = headerEndCrLf !== -1 && (headerEndLf === -1 || headerEndCrLf < headerEndLf);
+    const headerEnd = useCrLf ? headerEndCrLf : headerEndLf;
+    const separatorLength = useCrLf ? 4 : 2;
+    const headerRaw = buffer.slice(0, headerEnd).toString('utf8');
+    const match = headerRaw.match(/Content-Length:\s*(\d+)/i);
+    if (!match) {
+      buffer = buffer.slice(headerEnd + separatorLength);
+      return null;
+    }
+
+    const length = Number(match[1]);
+    const totalSize = headerEnd + separatorLength + length;
+    if (buffer.length < totalSize) return null;
+
+    const body = buffer.slice(headerEnd + separatorLength, totalSize).toString('utf8');
+    buffer = buffer.slice(totalSize);
+
+    return { message: parseWithTransport(body, 'framed'), transport: 'framed' };
   }
 
-  const length = Number(match[1]);
-  const totalSize = headerEnd + 4 + length;
-  if (buffer.length < totalSize) return null;
+  // Codex MCP client currently sends newline-delimited JSON during startup.
+  if (hasContentLengthPrefix()) return null;
+
+  const newlineIndex = buffer.indexOf('\n');
+  if (newlineIndex === -1) return null;
 
-  const body = buffer.slice(headerEnd + 4, totalSize).toString('utf8');
-  buffer = buffer.slice(totalSize);
+  const line = buffer.slice(0, newlineIndex).toString('utf8').trim();
+  buffer = buffer.slice(newlineIndex + 1);
+  if (!line) return null;
 
-  return JSON.parse(body);
+  return { message: parseWithTransport(line, 'ndjson'), transport: 'ndjson' };
 }
 
 async function onData(chunk) {
@@ -590,39 +628,60 @@ async function onData(chunk) {
   while (true) {
     const message = tryReadMessage();
     if (!message) return;
+    const envelope = message;
+    const request = envelope.message;
+    const transport = envelope.transport;
 
-    if (!Object.prototype.hasOwnProperty.call(message, 'id')) {
+    if (!Object.prototype.hasOwnProperty.call(request, 'id')) {
       continue;
     }
 
     try {
-      const result = await handleRequest(message);
-      writeMessage({ jsonrpc: '2.0', id: message.id, result });
+      const result = await handleRequest(request);
+      writeMessage({ jsonrpc: '2.0', id: request.id, result }, transport);
     } catch (err) {
       writeMessage({
         jsonrpc: '2.0',
-        id: message.id,
+        id: request.id,
         error: {
           code: -32603,
           message: err.message || 'Internal error',
         },
-      });
+      }, transport);
     }
   }
 }
 
+function startStdioServer() {
+  if (stdioStarted) return;
+  stdioStarted = true;
+
+  // Keep the process alive even if stdin closes (prevents premature exit
+  // when launched by MCP clients like Claude Code, Codex, Gemini CLI).
+  const keepAlive = setInterval(() => {}, 60_000);
+
+  process.stdin.resume();
+  process.stdin.on('data', (chunk) => {
+    onData(chunk).catch((err) => {
+      const transport = err && err.transport === 'ndjson' ? 'ndjson' : 'framed';
+      writeMessage({ jsonrpc: '2.0', id: null, error: { code: -32603, message: err.message } }, transport);
+    });
+  });
+  process.stdin.on('end', () => {
+    // stdin closed — clean up and exit gracefully
+    clearInterval(keepAlive);
+  });
+}
+
 module.exports = {
   TOOLS,
   handleRequest,
   callTool,
   resolveSafePath,
   SAFE_DATA_DIR,
+  startStdioServer,
 };
 
 if (require.main === module) {
-  process.stdin.on('data', (chunk) => {
-    onData(chunk).catch((err) => {
-      writeMessage({ jsonrpc: '2.0', id: null, error: { code: -32603, message: err.message } });
-    });
-  });
+  startStdioServer();
 }

package/bin/cli.js

@@ -14,6 +14,7 @@
 
 const fs = require('fs');
 const path = require('path');
+const crypto = require('crypto');
 const { execSync } = require('child_process');
 
 const COMMAND = process.argv[2];
@@ -124,6 +125,7 @@ function setupCursor() {
 
 function init() {
   const rlhfDir = path.join(CWD, '.rlhf');
+  const configPath = path.join(rlhfDir, 'config.json');
 
   if (!fs.existsSync(rlhfDir)) {
     fs.mkdirSync(rlhfDir, { recursive: true });
@@ -132,15 +134,28 @@ function init() {
     console.log('.rlhf/ already exists — updating config');
   }
 
+  let existingInstallId = null;
+  if (fs.existsSync(configPath)) {
+    try {
+      const existingConfig = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      if (existingConfig && typeof existingConfig.installId === 'string' && existingConfig.installId.trim()) {
+        existingInstallId = existingConfig.installId.trim();
+      }
+    } catch (_) {
+      // Ignore invalid existing config and write a fresh one below.
+    }
+  }
+
   const config = {
     version: pkgVersion(),
     apiUrl: process.env.RLHF_API_URL || 'http://localhost:3000',
     logPath: '.rlhf/feedback-log.jsonl',
     memoryPath: '.rlhf/memory-log.jsonl',
+    installId: existingInstallId || crypto.randomUUID(),
     createdAt: new Date().toISOString(),
   };
 
-  fs.writeFileSync(path.join(rlhfDir, 'config.json'), JSON.stringify(config, null, 2) + '\n');
+  fs.writeFileSync(configPath, JSON.stringify(config, null, 2) + '\n');
   console.log('Wrote .rlhf/config.json');
 
   // Always create .mcp.json (project-level MCP config used by Claude, Codex, Cursor)
@@ -189,6 +204,22 @@ function init() {
   console.log('');
   console.log(`rlhf-feedback-loop v${pkgVersion()} initialized.`);
   console.log('Run: npx rlhf-feedback-loop help');
+
+  try {
+    const { appendFunnelEvent } = require(path.join(PKG_ROOT, 'scripts', 'billing'));
+    appendFunnelEvent({
+      stage: 'acquisition',
+      event: 'cli_init_completed',
+      evidence: 'cli_init_completed',
+      installId: config.installId,
+      metadata: {
+        cwd: CWD,
+        version: config.version,
+      },
+    });
+  } catch (_) {
+    // Avoid failing init if telemetry write cannot be performed.
+  }
 }
 
 function capture() {
@@ -351,7 +382,8 @@ function serve() {
 
   // Start MCP server over stdio
   const mcpServer = path.join(PKG_ROOT, 'adapters', 'mcp', 'server-stdio.js');
-  require(mcpServer);
+  const { startStdioServer } = require(mcpServer);
+  startStdioServer();
 }
 
 function install() {
@@ -442,7 +474,6 @@ switch (COMMAND) {
     prove();
     break;
   case 'start-api':
-  case 'serve':
     startApi();
     break;
   case 'help':

package/openapi/openapi.yaml

@@ -1,12 +1,12 @@
 openapi: 3.1.0
 info:
   title: RLHF Feedback Loop API
-  version: 1.1.0
+  version: 1.2.0
   description: |
     Production API for feedback capture, schema-validated memory promotion,
     prevention rule generation, and DPO export.
 servers:
-  - url: http://localhost:8787
+  - url: https://rlhf-feedback-loop-710216278770.us-central1.run.app
 security:
   - bearerAuth: []
 components:
@@ -80,6 +80,59 @@ components:
           type: string
         approved:
           type: boolean
+    BillingCheckoutRequest:
+      type: object
+      properties:
+        successUrl:
+          type: string
+          format: uri
+        cancelUrl:
+          type: string
+          format: uri
+        customerEmail:
+          type: string
+          format: email
+        installId:
+          type: string
+        metadata:
+          type: object
+          additionalProperties:
+            type: string
+    BillingProvisionRequest:
+      type: object
+      required: [customerId]
+      properties:
+        customerId:
+          type: string
+        installId:
+          type: string
+    FunnelAnalyticsResponse:
+      type: object
+      properties:
+        totalEvents:
+          type: integer
+        stageCounts:
+          type: object
+          properties:
+            acquisition:
+              type: integer
+            activation:
+              type: integer
+            paid:
+              type: integer
+        eventCounts:
+          type: object
+          additionalProperties:
+            type: integer
+        conversionRates:
+          type: object
+          properties:
+            acquisitionToActivation:
+              type: number
+            activationToPaid:
+              type: number
+            acquisitionToPaid:
+              type: number
 paths:
   /healthz:
     get:
@@ -113,6 +166,18 @@ paths:
           description: Aggregated feedback statistics
         '401':
           description: Unauthorized
+  /v1/analytics/funnel:
+    get:
+      operationId: getFunnelAnalytics
+      responses:
+        '200':
+          description: Acquisition/activation/paid funnel metrics from append-only ledger
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/FunnelAnalyticsResponse'
+        '401':
+          description: Unauthorized
   /v1/intents/catalog:
     get:
       operationId: listIntentCatalog
@@ -290,3 +355,60 @@ paths:
           description: Recent provenance events
         '401':
           description: Unauthorized
+  /v1/billing/checkout:
+    post:
+      operationId: createBillingCheckoutSession
+      security: []
+      requestBody:
+        required: false
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/BillingCheckoutRequest'
+      responses:
+        '200':
+          description: Stripe checkout session created
+  /v1/billing/usage:
+    get:
+      operationId: getBillingUsage
+      responses:
+        '200':
+          description: Usage count for authenticated billing key
+        '401':
+          description: Unauthorized
+  /v1/billing/provision:
+    post:
+      operationId: provisionBillingKey
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/BillingProvisionRequest'
+      responses:
+        '200':
+          description: API key provisioned
+        '400':
+          description: Missing required customerId
+        '401':
+          description: Unauthorized
+        '403':
+          description: Forbidden - requires static RLHF_API_KEY admin token
+  /v1/billing/webhook:
+    post:
+      operationId: stripeBillingWebhook
+      security: []
+      responses:
+        '200':
+          description: Webhook accepted
+        '400':
+          description: Invalid webhook signature or payload
+  /v1/billing/github-webhook:
+    post:
+      operationId: githubMarketplaceWebhook
+      security: []
+      responses:
+        '200':
+          description: Webhook accepted
+        '400':
+          description: Invalid webhook signature or payload