bereach-openclaw 1.3.3 → 1.4.0-beta.10

package/BATTLE-TEST-PLAN.md

@@ -0,0 +1,339 @@
+# BeReach OpenClaw Plugin — Battle Test Plan
+
+Comprehensive test plan covering all 20 fixes + critical bug fixes from the audit.
+Each section maps to one or more fixes and is marked **[PASS]** / **[FAIL]** during testing.
+
+---
+
+## Pre-requisites
+
+- [ ] Plugin beta published with all fixes
+- [ ] Plugin installed globally: `npm install -g bereach-openclaw@beta`
+- [ ] Verify installed version: `openclaw plugin list` or check `~/.openclaw/workspace/node_modules/bereach-openclaw/package.json`
+- [ ] Staging API running and accessible (`api-staging.berea.ch`)
+- [ ] `BEREACH_API_KEY` configured in OpenClaw plugin settings
+
+---
+
+## 1. First Session & Profile Auto-Init (Fix 1 + Fix 11)
+
+**Goal**: When a brand new user starts their first session, the plugin auto-fetches their LinkedIn profile and saves it as context. The prompt includes a "First Session" guidance section.
+
+### Tests
+
+- [ ] **1.1** Delete all context entries and session-meta state before test:
+  ```
+  bereach_context_delete (all entries)
+  bereach_state_delete('session-meta')
+  ```
+- [ ] **1.2** Start a new session. Check logs for:
+  - `[bereach:context] profile context auto-initialized`
+  - `sections: ... contexts=1 ... firstSession=true`
+- [ ] **1.3** Verify the prompt contains `### First Session` section with onboarding guidance
+- [ ] **1.4** Verify the prompt contains `### User Context` with the auto-initialized profile (name, headline, location)
+- [ ] **1.5** Start another session immediately. Verify:
+  - `### First Session` section is NOT present (session-meta now has `lastSessionAt`)
+  - Profile is NOT re-fetched (guard: `profileInitDone`)
+  - Logs show `cache HIT` for the profile check
+
+---
+
+## 2. Daily Profile Refresh (Fix 6)
+
+**Goal**: If the user's LinkedIn profile changes (headline, location), the plugin detects and refreshes it after 24h.
+
+### Tests
+
+- [ ] **2.1** Manually set `lastProfileRefreshAt` to >24h ago:
+  ```
+  bereach_state_set('session-meta', { lastSessionAt: "...", lastProfileRefreshAt: "2026-03-19T00:00:00Z" })
+  ```
+- [ ] **2.2** Start a new session. Check logs for:
+  - `[bereach:context] profile context refreshed` OR `profile refresh: no changes detected`
+- [ ] **2.3** Verify `lastProfileRefreshAt` was updated in session-meta state
+- [ ] **2.4** Start another session. Verify no refresh happens (< 24h since last)
+
+---
+
+## 3. Campaign Create → State Sync (Fix 2 + Cache Bug Fix)
+
+**Goal**: When the agent creates a campaign via `bereach_contacts_create_campaign`, the campaign is immediately visible in the agent's campaign state AND the prompt's Campaign Dispatch.
+
+### Tests
+
+- [ ] **3.1** Start with no campaigns. Prompt should show `### Execution Modes`
+- [ ] **3.2** Ask the agent to create a lead-gen campaign. Check logs for:
+  - `[bereach:track] campaign sync: added "..." (...) to agent state`
+- [ ] **3.3** On the next prompt turn, verify:
+  - `### Campaign Dispatch` section is present
+  - Campaign name and type appear correctly
+  - `### Execution Modes` is NOT present (replaced by dispatch)
+- [ ] **3.4** Verify the campaign was persisted to agent-state (use `bereach_state_get('campaigns')`)
+- [ ] **3.5** **Critical**: Verify ALL existing campaigns are still present (not wiped by the sync). Create a 2nd campaign and confirm both appear in state.
+
+---
+
+## 4. Campaign Delete → State Sync (Fix 2 + Cache Bug Fix)
+
+**Goal**: Deleting a campaign removes it from both DB and agent state immediately.
+
+### Tests
+
+- [ ] **4.1** With at least 2 campaigns, delete one via `bereach_contacts_delete_campaign`
+- [ ] **4.2** Check logs for: `[bereach:track] campaign sync: removed ... from agent state`
+- [ ] **4.3** On the next prompt turn, verify the deleted campaign no longer appears in Campaign Dispatch
+- [ ] **4.4** Verify remaining campaigns are still intact
+
+---
+
+## 5. Campaign Reconciliation (Fix 3)
+
+**Goal**: Campaigns created via the API/UI (outside the plugin) are automatically discovered and added to agent state.
+
+### Tests
+
+- [ ] **5.1** Create a campaign directly via the API (curl or Postman), bypassing the plugin
+- [ ] **5.2** Start a new session (or wait for cache TTL to expire)
+- [ ] **5.3** Check logs for: `[bereach:cache] campaign reconciliation: added 1 DB campaign(s) to agent state`
+- [ ] **5.4** Verify the campaign appears in Campaign Dispatch
+
+---
+
+## 6. Campaign Context/Playbook Injection (Fix 9)
+
+**Goal**: Campaign playbook/context stored in the DB is injected as an excerpt in the Campaign Dispatch section.
+
+### Tests
+
+- [ ] **6.1** Create a campaign with a detailed context/playbook (e.g., "Target VP Engineering at B2B SaaS companies with 50-500 employees...")
+- [ ] **6.2** On the next prompt turn, verify Campaign Dispatch includes: `Playbook: "Target VP Engineering..."`
+- [ ] **6.3** Verify long playbooks are truncated to 200 chars with `...`
+
+---
+
+## 7. Cache Invalidation Correctness (Fix 4 + Critical Bug Fix)
+
+**Goal**: After mutations on free tools (context, pipeline, drafts, campaigns), the Live Status shows fresh data — not stale defaults.
+
+### Tests
+
+- [ ] **7.1** **Context mutation**: Call `bereach_context_set` to add/update a context entry. On the next prompt, verify the new context appears in `### User Context` (not empty).
+- [ ] **7.2** **Context delete**: Call `bereach_context_delete`. Verify the entry disappears on next prompt.
+- [ ] **7.3** **Pipeline mutation**: Call `bereach_contacts_add` to add a contact. Verify `### Pipeline` table updates on next prompt.
+- [ ] **7.4** **Draft mutation**: Call `bereach_scheduled_message_create` to create a draft. Verify `### Pending Actions` shows the draft count on next prompt.
+- [ ] **7.5** **Account switch**: Call `bereach_switch_account`. Verify logs show `account switch: cleared all caches`. Verify all data refreshes for the new account.
+
+---
+
+## 8. Multi-Account Context Isolation (Fix 5)
+
+**Goal**: When switching accounts, only contexts scoped to the active account (+ user-scoped) are shown.
+
+### Tests
+
+- [ ] **8.1** With 2 accounts, set a context entry with `scope: "account:ACCOUNT_1_ID"` and another with `scope: "user"`
+- [ ] **8.2** Switch to account 1. Verify both context entries appear
+- [ ] **8.3** Switch to account 2. Verify only the `scope: "user"` entry appears (account-1 scoped entry is hidden)
+
+---
+
+## 9. Auto-Resume: Lead-Gen (Fix 7 + Fix 8)
+
+**Goal**: When a lead-gen session ends with incomplete work, the auto-resume flag is set. When work is complete, the flag is cleared.
+
+### Tests
+
+- [ ] **9.1** Start a lead-gen task with target > 0 (e.g., "find 20 leads for..."). Let it run partially (e.g., find 5).
+- [ ] **9.2** End the session. Check logs for: `auto-resume: incomplete lead-gen (phase=..., found=5/20)`
+- [ ] **9.3** Start a new session. Verify `### AUTO-RESUME` section appears with correct progress
+- [ ] **9.4** Complete the lead-gen task (reach target). End the session.
+- [ ] **9.5** Check logs for: `auto-resume: tasks complete, cleared flag`
+- [ ] **9.6** Start a new session. Verify `### AUTO-RESUME` is NOT present
+
+---
+
+## 10. Auto-Resume: Outreach (Fix 7)
+
+**Goal**: Auto-resume also works for outreach tasks, not just lead-gen.
+
+### Tests
+
+- [ ] **10.1** Start an outreach task (e.g., "send connection requests to these 20 leads"). Let it run partially.
+- [ ] **10.2** End the session. Check logs for: `auto-resume: incomplete outreach (phase=...)`
+- [ ] **10.3** Start a new session. Verify `### AUTO-RESUME` mentions outreach
+- [ ] **10.4** Verify it shows `bereach_state_get('outreach')` (not lead-gen)
+
+---
+
+## 11. Campaign Lock / Concurrency (Fix 19)
+
+**Goal**: `lockedUntil` prevents concurrent cron sessions from processing the same campaign.
+
+### Tests
+
+- [ ] **11.1** In Campaign Dispatch, verify that campaigns with `lockedUntil` in the future are skipped
+- [ ] **11.2** Manually set `lockedUntil` to 10 minutes from now on a campaign. Start a session. Verify the campaign is NOT shown in Campaign Dispatch
+- [ ] **11.3** Wait for the lock to expire (or manually clear it). Verify the campaign reappears
+
+---
+
+## 12. Enforcement: Rate Limit Invalidation (Fix 14)
+
+**Goal**: After a rate-limit block, the limits cache is invalidated so the next check gets fresh data.
+
+### Tests
+
+- [ ] **12.1** Exhaust a daily rate limit (e.g., profile visits). Verify `rate limit BLOCKED` in logs
+- [ ] **12.2** Check logs for no subsequent block with stale cached data
+- [ ] **12.3** If limits reset (e.g., server-side correction), verify the agent can proceed on retry (cache refreshed)
+
+---
+
+## 13. Pending Actions: Failed Drafts (Fix 18)
+
+**Goal**: Failed scheduled messages are surfaced in Live Status.
+
+### Tests
+
+- [ ] **13.1** If there are any failed scheduled messages, verify `### Pending Actions` shows: `N scheduled message(s) FAILED to send`
+- [ ] **13.2** Verify it suggests `bereach_scheduled_message_list` to review
+
+---
+
+## 14. Pending Actions: Sent Invitations (Fix 17)
+
+**Goal**: Pending sent invitations are surfaced in Live Status.
+
+### Tests
+
+- [ ] **14.1** Send a few connection requests. On the next prompt, verify `### Pending Actions` shows: `N pending sent invitation(s)`
+- [ ] **14.2** Verify it suggests checking for stale ones >3 weeks
+
+---
+
+## 15. Prompt Size Guard (Fix 13)
+
+**Goal**: If the injected context exceeds 30,000 chars, it's truncated.
+
+### Tests
+
+- [ ] **15.1** Create many large context entries (e.g., 20 entries × 2KB each)
+- [ ] **15.2** Start a session. Check logs for: `context size guard: ... chars exceeds 30000 limit, truncating`
+- [ ] **15.3** Verify the prompt includes a truncation notice: `[Context truncated: ... chars removed]`
+- [ ] **15.4** Verify the prompt still functions correctly (not malformed)
+
+---
+
+## 16. Engagement Eviction (Fix 15)
+
+**Goal**: Engagement tracking state is capped at 500 entries.
+
+### Tests
+
+- [ ] **16.1** Manually set engagement-tracking state with >500 entries
+- [ ] **16.2** End a session with some engagement signals
+- [ ] **16.3** Check logs for: `engagement eviction: ... -> ... entries`
+- [ ] **16.4** Verify the saved state has ≤500 entries
+
+---
+
+## 17. bereach-setup Command (Fix 12)
+
+**Goal**: `/bereach-setup` command actually checks for and registers the orchestrator cron.
+
+### Tests
+
+- [ ] **17.1** Run `/bereach-setup` in OpenClaw
+- [ ] **17.2** If cron API is available: verify it either says "already registered" or "just registered"
+- [ ] **17.3** If cron API is unavailable: verify it shows the `openclaw cron add` command with correct JSON
+
+---
+
+## 18. Lifecycle: Consecutive Error Auto-Pause (Fix 16)
+
+**Goal**: Campaigns with >= 3 consecutive errors are auto-paused at session end.
+
+### Tests
+
+- [ ] **18.1** Manually set a campaign's `consecutiveErrors` to 5 in agent state
+- [ ] **18.2** End a session (with at least 1 tool call)
+- [ ] **18.3** Verify the campaign status changes to `"error"` with message `"Auto-paused after 5 consecutive errors"`
+- [ ] **18.4** Verify it appears in the Campaign Dispatch error section: `⚠ [ERROR]`
+
+---
+
+## 19. SDK-to-Tool Name Mapping (Fix 20)
+
+**Goal**: Sub-skill documentation includes correct tool name mapping.
+
+### Tests
+
+- [ ] **19.1** Verify `lead-gen.md`, `outreach.md`, and `lead-magnet.md` contain the "Tool naming" note
+- [ ] **19.2** Agent should correctly map SDK notation to `bereach_*` tool names
+
+---
+
+## 20. End-to-End: Full Campaign Lifecycle
+
+**Goal**: Complete lifecycle from campaign creation to orchestration to completion.
+
+### Tests
+
+- [ ] **20.1** Create a new lead-gen campaign with context/playbook
+- [ ] **20.2** Verify campaign appears in Campaign Dispatch with playbook excerpt
+- [ ] **20.3** Let the orchestrator cron fire (or manually trigger a cron session)
+- [ ] **20.4** Verify the campaign is processed (lead search, qualification, etc.)
+- [ ] **20.5** Check auto-resume flag is set when session ends mid-task
+- [ ] **20.6** Verify next cron session picks up where the previous left off
+- [ ] **20.7** Complete the campaign. Verify auto-resume flag is cleared
+- [ ] **20.8** Verify `lastSuccessAt` and `lastCheckAt` are updated
+
+---
+
+## 21. End-to-End: Long-Running Lead Gen with Cron
+
+**Goal**: Lead generation continues across multiple cron sessions without data loss.
+
+### Tests
+
+- [ ] **21.1** Start a lead-gen for 50+ targets
+- [ ] **21.2** Let it run through at least 3 cron cycles
+- [ ] **21.3** Verify progress accumulates correctly (found count increases)
+- [ ] **21.4** Verify no duplicate leads are created
+- [ ] **21.5** Verify engagement tracking persists across sessions
+- [ ] **21.6** Verify pipeline counts update correctly
+- [ ] **21.7** Verify credits are consumed and tracked
+
+---
+
+## 22. Edge Cases & Stress
+
+### Tests
+
+- [ ] **22.1** **Rapid tool calls**: Execute 10 `bereach_contacts_add` calls in quick succession. Verify pipeline is correct after all complete.
+- [ ] **22.2** **API timeout**: Disconnect network briefly during `fetchAll`. Verify plugin recovers gracefully (falls back to soul template).
+- [ ] **22.3** **Empty API responses**: Verify the plugin handles 404s and empty responses without crashing.
+- [ ] **22.4** **Corrupted campaign state**: Set campaign state to `{ schemaVersion: 99 }`. Verify auto-repair logs appear and state is reset cleanly.
+- [ ] **22.5** **Stale auto-resume (>24h)**: Set auto-resume `savedAt` to >24h ago. Verify it's ignored.
+- [ ] **22.6** **Session with 0 tool calls**: End a session without calling any tools. Verify no state is saved (no unnecessary API calls).
+- [ ] **22.7** **Multiple accounts**: Switch between accounts mid-session. Verify context isolation and cache clearing.
+
+---
+
+## Summary of Bugs Fixed in This Round
+
+| Bug | Severity | Description |
+|-----|----------|-------------|
+| Cache invalidation on non-sentinel keys | **CRITICAL** | `invalidate(key)` for non-sentinel keys (pipeline, contexts, drafts, campaigns) left sentinels intact. `getOrFetch` returned cache HIT with null/0/[] defaults. |
+| Campaign create wiping existing campaigns | **CRITICAL** | After `invalidate("campaignState")`, `getOrFetch` returned `campaignState: null`. Sync code created fresh state with only the new campaign, losing all existing campaigns. |
+| Auto-resume DELETE on every session | Low | Unnecessary DELETE API call + misleading log even when no flag existed. |
+| Free tool mutations not triggering cache refresh | Medium | `bereach_context_set`, `bereach_contacts_add`, `bereach_scheduled_message_create` etc. invalidated their keys but not the sentinels, so stale data was served. |
+
+### Fixes Applied
+
+- Added `invalidateAndRefresh(key)` — deletes the key AND both sentinels to force a full refetch
+- Added `cacheSet(key, data)` — directly updates a cache entry without refetching
+- Campaign create/delete now use `cacheSet` to update the local cache directly
+- Campaign update/status-transition use `invalidateAndRefresh` for full refresh
+- All free-tool mutations use `invalidateAndRefresh` instead of bare `invalidate`
+- Auto-resume DELETE now checks if a flag actually exists before calling DELETE