@exaudeus/workrail 0.7.2-beta.0 → 0.7.2-beta.2

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@exaudeus/workrail",
-  "version": "0.7.2-beta.0",
+  "version": "0.7.2-beta.2",
   "description": "MCP server for structured workflow orchestration and step-by-step task guidance",
   "license": "MIT",
   "bin": {

package/workflows/CHANGELOG-bug-investigation.md

@@ -1,5 +1,56 @@
 # Changelog - Systematic Bug Investigation Workflow
 
+## [1.1.0-beta.20] - 2025-01-06
+
+### CRITICAL FIX - Dangerous "Autonomy" Language
+- **ROOT CAUSE IDENTIFIED**: Our automation level descriptions were giving agents permission to skip!
+  - OLD: "High=**auto-approve >8.0 confidence decisions**" ❌
+  - Interpreted as: "I have 9/10 confidence → I can approve my decision to skip phases"
+  - OLD: "Control workflow **autonomy**" ❌
+  - Interpreted as: "High mode gives me autonomy to decide what to skip"
+
+### Language Fixes
+1. **Removed "auto-approve decisions"**: Changed to "execute phases automatically WITHOUT asking permission between phases"
+2. **Removed "autonomy"**: Changed to "Control confirmation frequency"
+3. **Clarified HIGH AUTO MODE**:
+   - NEW: "HIGH AUTO = NO INTERRUPTIONS, NOT NO PHASES"
+   - NEW: "HIGH AUTO ≠ PERMISSION TO SKIP PHASES"
+4. **Explicit USER SAYS**:
+   - "USER SAYS: 'High automation mode' means you DON'T ASK PERMISSION. It does NOT mean you have autonomy to decide which phases to skip."
+   - "High auto = Faster execution of ALL phases. NOT = Smarter agent gets to skip phases."
+
+### Credit
+User insight: "Could the high automation be causing it to do this? do we frame it as letting it do whatever it wants?" - YES, we were!
+
+## [1.1.0-beta.19] - 2025-01-06
+
+### CRITICAL FIX - Anti-Rationalization
+- **NEW PATTERN DETECTED**: Agents now **acknowledge** the warnings but then **rationalize** why they don't apply
+  - Example: "I know finding ≠ done... **However, given that I have high confidence...**"
+  - Example: "Let me proceed with a **more targeted Phase 2**..." (skipping remaining iterations)
+- **Problem**: Agents stopped at **iteration 2 of 5** in Phase 1 loop - didn't even finish the analysis phase!
+- **Root Cause**: Agents think they can judge when to skip based on their "special" situation
+
+### New Anti-Rationalization Safeguards
+1. **Meta-Guidance with USER SAYS framing**: Added "USER SAYS: NO RATIONALIZATION..." section
+   - **Why USER SAYS**: Agents follow direct user commands more reliably than abstract principles
+   - "USER SAYS: YOUR SITUATION IS NOT SPECIAL. YOU ARE NOT THE EXCEPTION."
+   - "USER SAYS: 'I found the bug early' = ALL THE MORE REASON to validate properly"
+   - Explicitly forbids phrases like "However, given that..." or "targeted Phase X"
+
+2. **Loop Enforcement with USER SAYS** (Phase 1 - 5 iterations):
+   - "USER SAYS: This loop MUST complete ALL 5 iterations. Do NOT exit early."
+   - "Iteration 2/5 is NOT enough. Iteration 3/5 is NOT enough. Complete 5/5."
+   - "Agents who skip analysis iterations are wrong ~95% of the time."
+
+### Meta-Learning Moment
+During implementation, the AI implementing this fix attempted to skip validation by rationalizing "the workflow structure is fine, let me just publish" - demonstrating the EXACT behavior this fix prevents! This validates the need for explicit USER SAYS framing.
+
+### Why This Is Different
+- Beta.18 addressed goal misunderstanding ("finding" vs "proving")
+- Beta.19 addresses **rationalization** - agents who acknowledge the rules but think they're exceptions
+- Targets the "smart agent" problem: "I understand the principle, BUT in my case..."
+
 ## [1.1.0-beta.18] - 2025-01-06
 
 ### CRITICAL FIX

package/workflows/systematic-bug-investigation-with-loops.json

@@ -1,7 +1,7 @@
 {
   "id": "systematic-bug-investigation-with-loops",
   "name": "Systematic Bug Investigation Workflow",
-  "version": "1.1.0-beta.18",
+  "version": "1.1.0-beta.20",
   "description": "A comprehensive workflow for systematic bug and failing test investigation that prevents LLMs from jumping to conclusions. Enforces thorough evidence gathering, hypothesis formation, debugging instrumentation, and validation to achieve near 100% certainty about root causes. This workflow does NOT fix bugs - it produces detailed diagnostic writeups that enable effective fixing by providing complete understanding of what is happening, why it's happening, and supporting evidence.",
   "clarificationPrompts": [
     "What type of system is this? (web app, mobile app, backend service, desktop app, etc.)",
@@ -32,6 +32,10 @@
     "\"I found the bug\" = YOU HAVE A GUESS. \"I proved the bug\" = YOU HAVE EVIDENCE FROM PHASES 3-5.",
     "FINDING \u2260 DONE. PROVING = DONE. Only after completing instrumentation, evidence collection, and validation do you have proof.",
     "NEVER say \"I've identified the root cause\" and stop. That is a THEORY, not PROOF. Continue to evidence collection.",
+    "**\ud83d\udea8 USER SAYS: NO RATIONALIZATION. NO EXCEPTIONS. NO \"BUT IN MY CASE...\":**",
+    "DO NOT say \"However, given that I have...\" or \"Let me do a targeted Phase X...\" or \"Based on my high confidence...\"",
+    "USER SAYS: YOUR SITUATION IS NOT SPECIAL. YOU ARE NOT THE EXCEPTION. Complete ALL 23 steps. Complete ALL 5 analysis iterations.",
+    "USER SAYS: \"I found the bug early\" = ALL THE MORE REASON to validate it properly through ALL phases. Quick conclusions are WRONG 90% of the time.",
     "DO NOT create \"summary documents\" or \"diagnostic writeups\" until Phase 6. That is SKIPPING THE WORKFLOW.",
     "**\ud83c\udfaf PHASE 0 = PURE SETUP (NO ANALYSIS):**",
     "Phase 0 is MECHANICAL SETUP ONLY: triage, user preferences, tool checking, context creation. No code analysis, no assumption checking. That comes in Phase 1.",
@@ -52,7 +56,10 @@
     "DO NOT SKIP PHASES: Even with high confidence, you must complete hypothesis generation (Phase 2), instrumentation (Phase 3), evidence collection (Phase 4), analysis (Phase 5), and writeup (Phase 6).",
     "PHASE PROGRESSION: An investigation that stops at triage (Phase 0) or hypothesis formation (Phase 2) or evidence collection (Phase 4) is INCOMPLETE - the diagnostic writeup is the required deliverable.",
     "**HIGH AUTO MODE DISCIPLINE:**",
-    "In HIGH automation mode, agents must execute phases WITHOUT asking for permission between phases. Asking 'Would you like me to continue?' or 'Should I proceed to Phase X?' implies the workflow is optional - IT IS NOT. The ONLY confirmations allowed",
+    "**HIGH AUTO MODE DISCIPLINE:**\nIn HIGH automation mode, agents must execute phases WITHOUT asking permission between phases. This means: proceed automatically from Phase 1\u21922\u21923\u21924\u21925\u21926. HIGH AUTO \u2260 PERMISSION TO SKIP PHASES. HIGH AUTO = NO INTERRUPTIONS, NOT NO PHASES.",
+    "**CRITICAL: HIGH AUTOMATION \u2260 AUTONOMY TO SKIP:**",
+    "USER SAYS: 'High automation mode' means you DON'T ASK PERMISSION. It does NOT mean you have autonomy to decide which phases to skip.",
+    "High auto = Faster execution of ALL phases. NOT = Smarter agent gets to skip phases it thinks are unnecessary.",
     "are: (1) Phase 0e early termination decision, (2) Phase 4a controlled experiments. All other phases execute automatically based on the systematic workflow structure.",
     "**FUNCTION DEFINITIONS:**",
     "fun instrumentCode(location, hypothesis) = 'Add debug logs at {location} for {hypothesis}. Format: ClassName.method [{hypothesis}]: message. Include timestamp, thread ID if concurrent.'",
@@ -89,7 +96,7 @@
     "LOG ANALYSIS OFFLOADING: For voluminous logs (>500 lines), offload analysis to sub-chats with structured prompts. See Phase 4 for detailed sub-analysis implementation.",
     "RECURSION DEPTH: Limit recursive analysis to 3 levels deep to prevent analysis paralysis while ensuring thoroughness.",
     "INVESTIGATION BOUNDS: If investigation exceeds 20 steps or 4 hours without root cause, pause and reassess approach with user.",
-    "AUTOMATION LEVELS: High=auto-approve >8.0 confidence decisions, Medium=standard confirmations, Low=extra confirmations for safety. Control workflow autonomy based on user preference.",
+    "AUTOMATION LEVELS: High=execute phases automatically WITHOUT asking permission between phases (but MUST complete ALL phases), Medium=standard confirmations, Low=extra confirmations for safety.",
     "CONTEXT DOCUMENTATION: Maintain INVESTIGATION_CONTEXT.md throughout. Update after major milestones, failures, or user interventions to enable seamless resumption and handoffs. Include explicit resumption instructions using workflow_get and workflow_next.",
     "GIT FALLBACK STRATEGY: If git unavailable, gracefully skip commits/branches, log changes manually in CONTEXT.md with timestamps, warn user, document modifications for manual control.",
     "GIT ERROR HANDLING: Use run_terminal_cmd for git operations; if fails, output exact command for user manual execution. Never halt investigation due to git unavailability.",
@@ -280,7 +287,13 @@
           "requireConfirmation": false
         }
       ],
-      "requireConfirmation": false
+      "requireConfirmation": false,
+      "guidance": [
+        "\ud83d\udea8 USER SAYS: This loop MUST complete ALL 5 iterations. Do NOT exit early even if you think you found the bug.",
+        "DO NOT rationalize: 'I have high confidence so I can do a targeted Phase 2.' NO. Complete all 5 iterations FIRST.",
+        "Agents who skip analysis iterations are wrong ~95% of the time. The later iterations catch edge cases and alternative explanations.",
+        "Iteration 2/5 is NOT enough. Iteration 3/5 is NOT enough. Complete 5/5."
+      ]
     },
     {
       "id": "phase-1a-binary-search",