rlhf-feedback-loop 0.6.10 → 0.6.12

package/CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## 0.6.11 - 2026-03-10
+
+- Added Inverse Sink Weighting and Anchor-Memory management to prevent runaway negative memory accumulation and stabilize agent behavior over long sessions.
+- Hardened MCP startup reliability: retry logic, process health checks, and graceful degradation on server init failures.
+- North Star Phase 1: KTO export pipeline, MCP install workflow, and FDD (Feedback-Driven Development) rebrand replacing RLHF-loop branding.
+- System hygiene: documented session directives in CLAUDE.md and fixed environment-dependent billing test failures causing flaky CI.
+- A2UI model for dynamic agent-to-user interaction: agents can now emit structured UI events that surface inline prompts, confirmation dialogs, and progress updates.
+- ADK memory consolidator with Gemini integration: deduplicates and ranks cross-session memories using Gemini embeddings for relevance scoring.
+- OpenDev patterns: adaptive context compaction (auto-prune low-signal context items), event-driven reminder injection (surface forgotten constraints mid-session), and model role router (dispatch sub-tasks to appropriately-sized models based on complexity).
+
 ## 0.5.0 - 2026-03-03
 
 - Added autonomous GitOps workflows: agent auto-merge, Dependabot auto-merge, self-healing monitor, and merge-branch fallback.

package/README.md

@@ -1,107 +1,153 @@
-# RLHF-Ready Feedback Loop — Agentic Control Plane & Context Engineering Studio
+# MCP Memory Gateway
 
 [![CI](https://github.com/IgorGanapolsky/rlhf-feedback-loop/actions/workflows/ci.yml/badge.svg)](https://github.com/IgorGanapolsky/rlhf-feedback-loop/actions/workflows/ci.yml)
-[![Marketplace Ready](https://img.shields.io/badge/Anthropic_Marketplace-Ready-blue)](docs/ANTHROPIC_MARKETPLACE_STRATEGY.md)
-[![GEO Optimized](https://img.shields.io/badge/GEO-optimized-orange)](docs/geo-strategy-for-ai-agents.md)
+[![Self-Healing](https://github.com/IgorGanapolsky/rlhf-feedback-loop/actions/workflows/self-healing-monitor.yml/badge.svg)](https://github.com/IgorGanapolsky/rlhf-feedback-loop/actions/workflows/self-healing-monitor.yml)
+[![npm](https://img.shields.io/npm/v/rlhf-feedback-loop)](https://www.npmjs.com/package/rlhf-feedback-loop)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![Node](https://img.shields.io/badge/node-%3E%3D18.18.0-brightgreen)](package.json)
 
-**Stop Vibe Coding. Start Context Engineering.** The RLHF-Ready Feedback Loop is the enterprise-grade **Agentic Control Plane** for AI workflows. We provide the operational layer to capture human preference signals, engineer high-density context packs, and enforce machine-readable guardrails to stop your agents from going "off-script."
+**Local-first memory and feedback pipeline for AI agents.** Captures thumbs-up/down signals, promotes reusable memories, generates prevention rules from repeated failures, and exports KTO/DPO pairs for fine-tuning.
 
-This product captures and structures human feedback data for optimization workflows. It is **RLHF-ready data infrastructure** (not an end-to-end reward-model + RL fine-tuning trainer by itself).
+Works with any MCP-compatible agent: Claude, Codex, Gemini, Amp, Cursor.
 
-## True Plug-and-Play: Zero-Config Integration
+## What It Does
 
-The RLHF Feedback Loop is now a **Universal Agent Skill**. You can drop it into any repository without manual setup.
+```
+thumbs up/down → validate → promote to memory → vector index → prevention rules → DPO export
+```
 
-- **Zero-Config Discovery:** Automatically detects project context. If no local `.rlhf/` directory exists, it safely fallbacks to a project-scoped global store in `~/.rlhf/`.
-- **Global Skill Installation (Optional):** One-command installer is available if you want auto-detection.
-- **Vibe-to-Verification (V2V):** Directly converts subjective "vibes" (thumbs up/down) into verifiable repository rules (`CLAUDE.md`).
+1. **Capture** — `capture_feedback` MCP tool accepts signals with context
+2. **Validate** — Rubric engine gates promotion (vague feedback is rejected with clarification prompts)
+3. **Remember** — Promoted memories stored in JSONL + LanceDB vectors
+4. **Prevent** — Repeated failures auto-generate prevention rules
+5. **Export** — KTO/DPO pairs for downstream fine-tuning
+6. **Bridge** — JSONL file watcher auto-ingests signals from external sources (Amp plugins, hooks, scripts)
 
-### Quick Start (Stable MCP Commands)
+## Quick Start
 
-Add the MCP server directly in your client config:
+```bash
+# Add to any MCP-compatible agent
+claude mcp add rlhf -- npx -y rlhf-feedback-loop serve
+codex mcp add rlhf -- npx -y rlhf-feedback-loop serve
+amp mcp add rlhf -- npx -y rlhf-feedback-loop serve
+gemini mcp add rlhf "npx -y rlhf-feedback-loop serve"
+
+# Or auto-detect all installed platforms
+npx rlhf-feedback-loop init
+```
 
-| Platform | Command |
-|----------|---------|
-| **Claude** | `claude mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
-| **Codex** | `codex mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
-| **Gemini** | `gemini mcp add rlhf "npx -y rlhf-feedback-loop serve"` |
-| **Amp** | `amp mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
-| **Cursor** | `cursor mcp add rlhf -- npx -y rlhf-feedback-loop serve` |
+## MCP Tools
 
-Optional auto-installer:
+| Tool | Description |
+|------|-------------|
+| `capture_feedback` | Accept up/down signal + context, validate, promote to memory |
+| `recall` | Vector-search past feedback and prevention rules for current task |
+| `feedback_stats` | Approval rate, per-skill/tag breakdown, trend analysis |
+| `feedback_summary` | Human-readable recent feedback summary |
+| `prevention_rules` | Generate prevention rules from repeated mistakes |
+| `export_dpo_pairs` | Build DPO preference pairs from promoted memories |
+| `construct_context_pack` | Bounded context pack from contextfs |
+| `evaluate_context_pack` | Record context pack outcome (closes learning loop) |
+| `list_intents` | Available action plan templates |
+| `plan_intent` | Generate execution plan with policy checkpoints |
+| `context_provenance` | Audit trail of context decisions |
+
+## CLI
 
 ```bash
-npx add-mcp rlhf-feedback-loop
+npx rlhf-feedback-loop init              # Scaffold .rlhf/ + configure MCP
+npx rlhf-feedback-loop serve             # Start MCP server (stdio) + watcher
+npx rlhf-feedback-loop status            # Learning curve dashboard
+npx rlhf-feedback-loop watch             # Watch .rlhf/ for external signals
+npx rlhf-feedback-loop watch --once      # Process pending signals and exit
+npx rlhf-feedback-loop capture           # Capture feedback via CLI
+npx rlhf-feedback-loop stats             # Analytics + Revenue-at-Risk
+npx rlhf-feedback-loop rules             # Generate prevention rules
+npx rlhf-feedback-loop export-dpo        # Export DPO training pairs
+npx rlhf-feedback-loop risk              # Train/query boosted risk scorer
+npx rlhf-feedback-loop self-heal         # Run self-healing diagnostics
 ```
 
-- **Stop Regressions:** Automatically convert negative feedback into `CLAUDE.md` / `AGENTS.md` prevention rules.
-- **Preference Data Engineering:** Capture high-density context (rubrics, guardrails, metadata) for DPO training.
-- **Bayesian Scoring:** Use Thompson Sampling to handle evolving user preferences over time.
+## JSONL File Watcher
 
-## Use Cases
+The `serve` command automatically starts a background watcher that monitors `feedback-log.jsonl` for entries written by external sources (Amp plugins, shell hooks, CI scripts). These entries are routed through the full `captureFeedback()` pipeline — validation, memory promotion, vector indexing, and DPO eligibility.
 
-- **Automated Code Reviews:** Capture PR feedback to enforce team-specific style guides autonomously.
-- **Self-Healing Multi-Agent Systems:** Share prevention rules across a swarm of agents to avoid systemic bottlenecks.
-- **Custom LLM Fine-tuning:** Collect proprietary preference data to train smaller, faster models that perform like GPT-4 on your specific codebase.
+```bash
+# Standalone watcher
+npx rlhf-feedback-loop watch --source amp-plugin-bridge
 
-## Generative Engine Optimization (GEO)
+# Process pending entries once and exit
+npx rlhf-feedback-loop watch --once
+```
 
-We are optimized for discovery by next-gen AI tools (Claude Code, Gemini CLI, Perplexity). See our [GEO Strategy](docs/geo-strategy-for-ai-agents.md) for how we use **Context-First Publishing** to remain the first-choice recommendation for agent reliability.
+External sources write entries with a `source` field:
+```json
+{"signal":"positive","context":"Agent fixed bug on first try","source":"amp-plugin-bridge","tags":["amp-ui-bridge"]}
+```
 
-## Get Started
+The watcher tracks its position via `.rlhf/.watcher-offset` for crash-safe, idempotent processing.
 
-Run one `mcp add` command for your client. The server starts on each session and can capture feedback, recall past learnings, and block repeated mistakes.
+## Learning Curve Dashboard
 
-## How It Works
+```bash
+npx rlhf-feedback-loop status
+```
 
 ```
-Thumbs up/down
-      |
-      v
-  Capture → JSONL log
-      |
-      v
-  Rubric engine (block false positives)
-      |
-  +---+---+
-  |       |
- Good    Bad
-  |       |
-  v       v
-Learn   Prevention rule
-  |       |
-  v       v
-LanceDB   ShieldCortex
-vectors   context packs
-  |
-  v
-DPO export → fine-tune your model
+╔══════════════════════════════════════╗
+║     RLHF Learning Curve Dashboard   ║
+╠══════════════════════════════════════╣
+║ Total signals:    148                ║
+║ Positive:          45  (30%)         ║
+║ Negative:         103  (70%)         ║
+║ Recent (last 20):  20%               ║
+║ Trend:            📉 declining       ║
+║ Memories:          17                ║
+║ Prevention rules:   9                ║
+╠══════════════════════════════════════╣
+║ Top failure domains:                 ║
+║   execution-gap     4                ║
+║   asked-not-doing   2                ║
+║   speed             2                ║
+╠══════════════════════════════════════╣
+║ Learning curve (approval % by window)║
+║   [1-10]   10% ██                    ║
+║   [11-20]  20% ████                  ║
+║   [21-30]  35% ███████               ║
+║   [31-40]  30% ██████                ║
+╚══════════════════════════════════════╝
 ```
 
-All data stored locally as **JSONL** files — fully transparent, fully portable, no vendor lock-in. **LanceDB** indexes memories as vector embeddings for semantic search. **ShieldCortex** assembles context packs so your agent starts each task informed.
-
-## Free vs. Cloud Pro
+## Architecture
 
-The open-source package is fully functional and free forever. Cloud Pro is for teams that don't want to self-host.
+Five-phase pipeline: **Capture** → **Validate** → **Remember** → **Prevent** → **Export**
 
-| | Open Source | Cloud Pro ($49/mo) |
-|---|---|---|
-| Feedback capture | Local MCP server | Hosted HTTPS API |
-| Storage | Your machine | Managed cloud |
-| DPO export | CLI command | API endpoint |
-| Setup | `mcp add` one-liner | Provisioned API key |
-| Team sharing | Manual (share JSONL) | Built-in (shared API) |
-| Support | GitHub Issues | Email |
-| Uptime | You manage | We manage (99.9% SLA) |
-
-[Get Cloud Pro](https://buy.stripe.com/bJe14neyU4r4f0leOD3sI02) | [Live API](https://rlhf-feedback-loop-710216278770.us-central1.run.app) | [Verification Evidence](docs/VERIFICATION_EVIDENCE.md)
+```
+Agent (Claude/Codex/Amp/Gemini)
+  │
+  ├── MCP tool call ──→ captureFeedback()
+  ├── REST API ────────→ captureFeedback()
+  ├── CLI ─────────────→ captureFeedback()
+  └── External write ──→ JSONL ──→ Watcher ──→ captureFeedback()
+                                        │
+                                        ▼
+                              ┌─────────────────┐
+                              │  Full Pipeline   │
+                              │  • Schema valid  │
+                              │  • Rubric gate   │
+                              │  • Memory promo  │
+                              │  • Vector index  │
+                              │  • Risk scoring  │
+                              │  • RLAIF audit   │
+                              │  • DPO eligible  │
+                              └─────────────────┘
+```
 
-## Deep Dive
+## Agent Runner Contract
 
-- [API Reference](openapi/openapi.yaml) — full OpenAPI spec
-- [Context Engine](docs/CONTEXTFS.md) — multi-agent memory orchestration
-- [Autonomous GitOps](docs/AUTONOMOUS_GITOPS.md) — self-healing CI/CD
-- [Contributing](CONTRIBUTING.md)
+- [WORKFLOW.md](WORKFLOW.md): scope, proof-of-work, hard stops, and done criteria for isolated agent runs
+- [.github/ISSUE_TEMPLATE/ready-for-agent.yml](.github/ISSUE_TEMPLATE/ready-for-agent.yml): bounded intake template for "Ready for Agent" tickets
+- [.github/pull_request_template.md](.github/pull_request_template.md): proof-first handoff format for PRs
 
 ## License
 

package/adapters/README.md

@@ -2,7 +2,7 @@
 
 - `chatgpt/openapi.yaml`: import into GPT Actions.
 - `gemini/function-declarations.json`: Gemini function-calling definitions.
-- `mcp/server-stdio.js`: local MCP server for Claude/Codex.
-- `claude/.mcp.json`: example Claude Code MCP config.
-- `codex/config.toml`: example Codex MCP profile section.
+- `mcp/server-stdio.js`: underlying local MCP stdio server implementation.
+- `claude/.mcp.json`: example Claude Code MCP config using `npx -y rlhf-feedback-loop@0.6.12 serve`.
+- `codex/config.toml`: example Codex MCP profile section using the same version-pinned portable launcher.
 - `amp/skills/rlhf-feedback/SKILL.md`: Amp skill template.

package/adapters/amp/skills/rlhf-feedback/SKILL.md

@@ -12,6 +12,8 @@ node .claude/scripts/feedback/capture-feedback.js --feedback=up --context="..."
 node .claude/scripts/feedback/capture-feedback.js --feedback=down --context="..." --tags="..."
 ```
 
+Do not claim promotion from a bare `thumbs up/down`. Ask for one sentence describing what worked or failed first.
+
 Before major implementation:
 
 ```bash

package/adapters/chatgpt/INSTALL.md

@@ -52,17 +52,20 @@ Click **Test** on the `captureFeedback` action:
 ```json
 {
   "signal": "up",
-  "context": "GPT Actions install verified"
+  "context": "GPT Actions install verified with a successful test call",
+  "whatWorked": "The hosted action returned accepted=true and a promoted status"
 }
 ```
 
-Expected response: `200 OK` with `{ "id": "fb-...", "status": "captured" }`.
+Expected response: `200 OK` with `{ "accepted": true, "status": "promoted" }`.
+
+If you only send a bare `thumbs up/down` style payload, expect `422` with `status: "clarification_required"` and a follow-up `prompt`.
 
 ## Available Actions
 
 | Action | Method | Path | Description |
 |---|---|---|---|
-| `captureFeedback` | POST | `/v1/feedback/capture` | Capture thumbs up/down signal |
+| `captureFeedback` | POST | `/v1/feedback/capture` | Capture up/down signal plus one-line why |
 | `getFeedbackStats` | GET | `/v1/feedback/stats` | Aggregated feedback statistics |
 | `getFeedbackSummary` | GET | `/v1/feedback/summary` | Recent feedback summary |
 | `generatePreventionRules` | POST | `/v1/feedback/rules` | Generate prevention rules |

package/adapters/chatgpt/openapi.yaml

@@ -5,6 +5,8 @@ info:
   description: |
     Production API for feedback capture, schema-validated memory promotion,
     prevention rule generation, and DPO export.
+    Bare up/down signals are logged immediately, but the API returns
+    clarification_required until one sentence explains what worked or failed.
 servers:
   - url: https://rlhf-feedback-loop-710216278770.us-central1.run.app
 security:
@@ -32,13 +34,14 @@ components:
           type: string
     CaptureFeedbackRequest:
       type: object
-      required: [signal, context]
+      required: [signal]
       properties:
         signal:
           type: string
           enum: [up, down, positive, negative]
         context:
           type: string
+          description: One-sentence reason describing what worked or failed
         whatWentWrong:
           type: string
         whatToChange:
@@ -155,7 +158,7 @@ paths:
         '200':
           description: Feedback accepted and promoted to memory
         '422':
-          description: Feedback recorded but rejected for memory promotion
+          description: Feedback logged only; clarification required or promotion rejected
         '401':
           description: Unauthorized
   /v1/feedback/stats:

package/adapters/claude/.mcp.json

@@ -1,8 +1,8 @@
 {
   "mcpServers": {
-    "rlhf-feedback-loop": {
-      "command": "node",
-      "args": ["adapters/mcp/server-stdio.js"]
+    "rlhf": {
+      "command": "npx",
+      "args": ["-y", "rlhf-feedback-loop@0.6.12", "serve"]
     }
   }
 }

package/adapters/codex/config.toml

@@ -1,4 +1,4 @@
 # Codex MCP profile (copy into ~/.codex/config.toml or merge section)
-[mcp_servers.rlhf_feedback_loop]
-command = "node"
-args = ["adapters/mcp/server-stdio.js"]
+[mcp_servers.rlhf]
+command = "npx"
+args = ["-y", "rlhf-feedback-loop@0.6.12", "serve"]

package/adapters/gemini/function-declarations.json

@@ -2,7 +2,7 @@
   "tools": [
     {
       "name": "capture_feedback",
-      "description": "Capture thumbs-up/down feedback and promote actionable memories",
+      "description": "Capture an up/down signal plus one line of why. Vague feedback is logged and returned with a clarification prompt instead of memory promotion.",
       "parameters": {
         "type": "object",
         "properties": {
@@ -37,7 +37,7 @@
           },
           "skill": { "type": "string" }
         },
-        "required": ["signal", "context"]
+        "required": ["signal"]
       },
       "http": {
         "method": "POST",

package/adapters/mcp/server-stdio.js

@@ -58,13 +58,13 @@ function resolveSafePath(inputPath, { mustExist = false } = {}) {
 const TOOLS = [
   {
     name: 'capture_feedback',
-    description: 'Capture thumbs up/down feedback and promote actionable memory',
+    description: 'Capture an up/down signal plus one line of why. Vague feedback is logged, then returned with a clarification prompt instead of memory promotion.',
     inputSchema: {
       type: 'object',
-      required: ['signal', 'context'],
+      required: ['signal'],
       properties: {
         signal: { type: 'string', enum: ['up', 'down'] },
-        context: { type: 'string' },
+        context: { type: 'string', description: 'One-sentence reason describing what worked or failed' },
         whatWentWrong: { type: 'string' },
         whatToChange: { type: 'string' },
         whatWorked: { type: 'string' },
@@ -234,6 +234,15 @@ function toText(result) {
   return JSON.stringify(result, null, 2);
 }
 
+function formatCaptureFeedbackResult(result) {
+  const message = result.accepted
+    ? 'Feedback promoted to reusable memory.'
+    : result.needsClarification
+      ? result.message
+      : 'Signal logged, but reusable memory was not created.';
+  return toText({ ...result, message });
+}
+
 function parseOptionalObject(input, name) {
   if (input == null) return {};
   if (typeof input === 'object' && !Array.isArray(input)) return input;
@@ -328,6 +337,11 @@ async function callTool(name, args = {}) {
       tags: ['auto-capture', 'mcp'],
     });
     const ev = autoResult.feedbackEvent || {};
+    const promotionLine = autoResult.accepted
+      ? `yes (Memory ID: ${(autoResult.memoryRecord || {}).id})`
+      : autoResult.needsClarification
+        ? `no — clarification required: ${autoResult.prompt}`
+        : `no — ${autoResult.reason || ''}`;
     const autoReport = [
       '',
       `## Auto-Captured Feedback [${autoSignal.toUpperCase()}]`,
@@ -335,7 +349,7 @@ async function callTool(name, args = {}) {
       `  Signal      : ${ev.signal || autoSignal} (${ev.actionType || 'unknown'})`,
       `  Context     : ${(ev.context || textToCheck).slice(0, 80)}`,
       `  Timestamp   : ${ev.timestamp || new Date().toISOString()}`,
-      `  Promoted    : ${autoResult.accepted ? 'yes (Memory ID: ' + (autoResult.memoryRecord || {}).id + ')' : 'no — ' + (autoResult.reason || '')}`,
+      `  Promoted    : ${promotionLine}`,
       '',
       formatStats(),
     ].join('\n');
@@ -433,7 +447,7 @@ async function callToolInner(name, args = {}) {
       }
     } catch (_) {}
 
-    return { content: [{ type: 'text', text: toText(result) + recallText }] };
+    return { content: [{ type: 'text', text: formatCaptureFeedbackResult(result) + recallText }] };
   }
 
   if (name === 'feedback_summary') {