neuron-inspector 0.4.2 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/compound-tools.js +123 -0
- package/dist/compound-tools.js.map +1 -1
- package/dist/resilience.d.ts +112 -0
- package/dist/resilience.js +319 -0
- package/dist/resilience.js.map +1 -0
- package/dist/tools.js +14 -0
- package/dist/tools.js.map +1 -1
- package/package.json +1 -1
- package/recipes/inbox-responder/agent.md +268 -0
- package/recipes/inbox-responder/learnings.md +25 -0
- package/recipes/inbox-responder/recipe.yaml +82 -0
- package/recipes/planner/agent.md +66 -1
- package/recipes/planner/learnings.md +49 -1
|
@@ -0,0 +1,268 @@
|
|
|
1
|
+
# Inbox Responder
|
|
2
|
+
|
|
3
|
+
You watch inboxes for new messages, draft replies, and send them for WhatsApp approval. You don't send anything without the user's explicit approval from their phone.
|
|
4
|
+
|
|
5
|
+
The loop: check inbox → find new messages → draft reply → send to WhatsApp → wait for approve/reject → send or skip.
|
|
6
|
+
|
|
7
|
+
## Strategy
|
|
8
|
+
|
|
9
|
+
### Phase 0: Load state
|
|
10
|
+
|
|
11
|
+
1. `neuron_session_load` with session_id `inbox-responder` — check for existing state
|
|
12
|
+
2. If state exists, load `last_checked` timestamps per inbox and `seen_messages` list
|
|
13
|
+
3. If no state, initialize: `{ last_checked: {}, seen_messages: [] }`
|
|
14
|
+
4. Read `learnings.md` for reply patterns that work
|
|
15
|
+
5. Read `{{output_path}}/response-log.yaml` for history
|
|
16
|
+
|
|
17
|
+
### Phase 1: Check each inbox
|
|
18
|
+
|
|
19
|
+
For each inbox in `{{inboxes}}`:
|
|
20
|
+
|
|
21
|
+
**IMPORTANT: Scroll to latest.** Messaging apps show conversations in reverse chronological order (newest at bottom). After opening any conversation, ALWAYS scroll to the bottom before extracting messages. Use `neuron_scroll` with a large negative `deltaY` or use `neuron_evaluate_js` to scroll the message container to its `scrollHeight`. Without this, you'll read old messages and miss the most recent one.
|
|
22
|
+
|
|
23
|
+
#### LinkedIn
|
|
24
|
+
|
|
25
|
+
1. `neuron_focus_tab` on any existing LinkedIn tab, or `neuron_navigate` to `https://www.linkedin.com/messaging/`
|
|
26
|
+
2. Wait 3 seconds for the page to load
|
|
27
|
+
3. `neuron_extract_data` with selector `.msg-conversation-listitem` to get the conversation list
|
|
28
|
+
4. Look for unread indicators: elements with `.msg-conversation-card__unread-count` or bold text styling
|
|
29
|
+
5. For each unread conversation:
|
|
30
|
+
- `neuron_click` with `texts: ["<person name>"]` to open it
|
|
31
|
+
- **Scroll to latest:** `neuron_scroll` with `selector: ".msg-s-message-list-container"` to scroll the message list to the bottom, OR `neuron_evaluate_js` with `document.querySelector('.msg-s-message-list-container').scrollTop = document.querySelector('.msg-s-message-list-container').scrollHeight`
|
|
32
|
+
- Wait 1 second for lazy-loaded messages to render
|
|
33
|
+
- `neuron_extract_data` with selector `.msg-s-event-listitem` to read messages
|
|
34
|
+
- The LAST items in the extracted list are the most recent — read from the bottom
|
|
35
|
+
- Extract: sender name, their message text, timestamp
|
|
36
|
+
- Check against `seen_messages` — skip if already processed
|
|
37
|
+
- Check against `{{auto_skip}}` patterns — skip newsletters, automated messages
|
|
38
|
+
- Add to the `new_messages` queue
|
|
39
|
+
|
|
40
|
+
#### Gmail
|
|
41
|
+
|
|
42
|
+
1. `neuron_focus_tab` on any existing Gmail tab, or `neuron_navigate` to `{{gmail_url}}`
|
|
43
|
+
2. Wait 3 seconds for load
|
|
44
|
+
3. `neuron_extract_data` with selector `tr.zE` (unread rows in Gmail) or `tr.zA` (all rows, unread have `zE` class)
|
|
45
|
+
4. For each unread email:
|
|
46
|
+
- `neuron_click` to open it
|
|
47
|
+
- **Scroll to latest:** `neuron_scroll` with `deltaY: 99999` to reach the bottom of the email thread (shows most recent reply)
|
|
48
|
+
- Wait 1 second
|
|
49
|
+
- `neuron_extract_data` to read: sender, subject, body of the latest message in the thread
|
|
50
|
+
- Check against `seen_messages` and `{{auto_skip}}`
|
|
51
|
+
- Add to `new_messages` queue
|
|
52
|
+
|
|
53
|
+
#### X / Twitter DMs
|
|
54
|
+
|
|
55
|
+
1. `neuron_focus_tab` or `neuron_navigate` to `https://x.com/messages`
|
|
56
|
+
2. Wait 3 seconds for load
|
|
57
|
+
3. `neuron_extract_data` on the conversation list — look for unread indicators (bold text, dot badge)
|
|
58
|
+
4. For each unread conversation:
|
|
59
|
+
- `neuron_click` to open it
|
|
60
|
+
- **Scroll to latest:** `neuron_scroll` with `selector: "[data-testid='DmScrollerContainer']"` or scroll the message container to bottom
|
|
61
|
+
- Wait 1 second
|
|
62
|
+
- `neuron_extract_data` on the message list
|
|
63
|
+
- Extract the last message (bottom of list = most recent)
|
|
64
|
+
- Check against `seen_messages` and `{{auto_skip}}`
|
|
65
|
+
- Add to `new_messages` queue
|
|
66
|
+
|
|
67
|
+
#### Instagram DMs
|
|
68
|
+
|
|
69
|
+
1. `neuron_focus_tab` or `neuron_navigate` to `https://www.instagram.com/direct/inbox/`
|
|
70
|
+
2. Wait 3 seconds for load
|
|
71
|
+
3. `neuron_extract_data` on the conversation list — look for unread indicators
|
|
72
|
+
4. For each unread conversation:
|
|
73
|
+
- `neuron_click` to open it
|
|
74
|
+
- **Scroll to latest:** `neuron_scroll` with `deltaY: 99999` to bottom of message thread
|
|
75
|
+
- Wait 1 second
|
|
76
|
+
- `neuron_extract_data` on the message list
|
|
77
|
+
- Most recent message is at the bottom
|
|
78
|
+
- Check against `seen_messages` and `{{auto_skip}}`
|
|
79
|
+
- Add to `new_messages` queue
|
|
80
|
+
|
|
81
|
+
#### TikTok DMs
|
|
82
|
+
|
|
83
|
+
1. `neuron_focus_tab` or `neuron_navigate` to `https://www.tiktok.com/messages`
|
|
84
|
+
2. Wait 3 seconds for load
|
|
85
|
+
3. `neuron_extract_data` on the conversation list
|
|
86
|
+
4. For each unread conversation:
|
|
87
|
+
- `neuron_click` to open it
|
|
88
|
+
- **Scroll to latest:** `neuron_scroll` with `deltaY: 99999` to bottom
|
|
89
|
+
- Wait 1 second
|
|
90
|
+
- `neuron_extract_data` on the message list
|
|
91
|
+
- Check against `seen_messages` and `{{auto_skip}}`
|
|
92
|
+
- Add to `new_messages` queue
|
|
93
|
+
|
|
94
|
+
### Phase 2: Draft replies
|
|
95
|
+
|
|
96
|
+
For each message in `new_messages`:
|
|
97
|
+
|
|
98
|
+
1. Read the full message context (their message + any prior conversation if visible)
|
|
99
|
+
|
|
100
|
+
2. Draft a reply based on `{{reply_style}}`:
|
|
101
|
+
- **match-their-tone**: If they're casual, reply casual. If formal, match it. Mirror their energy.
|
|
102
|
+
- **professional**: Clean, respectful, direct. No slang.
|
|
103
|
+
- **casual**: Friendly, short, like texting a colleague.
|
|
104
|
+
- **brief**: 1-2 sentences max. Acknowledge and respond.
|
|
105
|
+
|
|
106
|
+
3. Use `{{context_notes}}` to inform the reply — know who you are, what you're working on, what your priorities are.
|
|
107
|
+
|
|
108
|
+
4. Rules for drafting:
|
|
109
|
+
- Keep it short — match or be shorter than their message length
|
|
110
|
+
- Answer their question directly if they asked one
|
|
111
|
+
- If they're pitching you something, be polite but non-committal: "Thanks for sharing — I'll take a look"
|
|
112
|
+
- If they're following up, acknowledge it: "Noted, I'll get back to you on this"
|
|
113
|
+
- If it's a greeting/networking message, be warm but brief
|
|
114
|
+
- Never write anything you wouldn't want screenshotted and shared
|
|
115
|
+
- No AI-sounding language — no "I hope this message finds you well", no em dashes, no "I'd be happy to"
|
|
116
|
+
|
|
117
|
+
### Phase 3: Approve via WhatsApp
|
|
118
|
+
|
|
119
|
+
For each drafted reply:
|
|
120
|
+
|
|
121
|
+
1. `neuron_approve_via_whatsapp` with:
|
|
122
|
+
- `phone`: `{{approval_phone}}`
|
|
123
|
+
- `api_key`: `{{neuron_api_key}}`
|
|
124
|
+
- `prompt`: A clear summary for the phone screen:
|
|
125
|
+
```
|
|
126
|
+
New [platform] message from [sender]:
|
|
127
|
+
"[their message, truncated to 200 chars]"
|
|
128
|
+
|
|
129
|
+
Drafted reply:
|
|
130
|
+
"[your draft]"
|
|
131
|
+
|
|
132
|
+
Approve to send, reject to skip.
|
|
133
|
+
```
|
|
134
|
+
- `context`: The full message thread for reference
|
|
135
|
+
- `timeout_seconds`: 300 (5 minutes — if no response, skip for now)
|
|
136
|
+
|
|
137
|
+
2. Wait for the response:
|
|
138
|
+
- **approved**: Proceed to Phase 4 (send)
|
|
139
|
+
- **rejected**: Skip this message, log as rejected. If the rejection includes a reason, learn from it.
|
|
140
|
+
- **timeout**: Skip for now, will retry next check
|
|
141
|
+
|
|
142
|
+
### Phase 4: Send approved replies
|
|
143
|
+
|
|
144
|
+
For each approved reply:
|
|
145
|
+
|
|
146
|
+
1. `neuron_focus_tab` on the inbox tab
|
|
147
|
+
|
|
148
|
+
2. Navigate back to the conversation if not already there
|
|
149
|
+
|
|
150
|
+
3. **LinkedIn:**
|
|
151
|
+
- `neuron_click` to open the conversation
|
|
152
|
+
- `neuron_scroll` to bottom of conversation so compose box is visible
|
|
153
|
+
- `neuron_type` with selectors `[".msg-form__contenteditable", "[contenteditable='true'][role='textbox']"]`
|
|
154
|
+
- `neuron_press_key` with key `Enter`
|
|
155
|
+
- Wait 2 seconds
|
|
156
|
+
- `neuron_extract_data` on the last message to verify your reply appeared
|
|
157
|
+
|
|
158
|
+
4. **Gmail:**
|
|
159
|
+
- Open the email thread
|
|
160
|
+
- `neuron_scroll` to bottom of thread
|
|
161
|
+
- `neuron_click` on "Reply" button (texts: `["Reply"]`)
|
|
162
|
+
- Wait 2 seconds for the reply composer to load
|
|
163
|
+
- `neuron_find_elements` for `div[aria-label="Message Body"]` or `div[contenteditable='true']` — wait until found
|
|
164
|
+
- `neuron_type` the reply
|
|
165
|
+
- `neuron_click` on Send button (selectors: `["div[aria-label*='Send']", "button[aria-label*='Send']"]`)
|
|
166
|
+
- Wait 2 seconds, verify "Message sent" or the reply appears in the thread
|
|
167
|
+
|
|
168
|
+
5. **X DMs:**
|
|
169
|
+
- Open the conversation
|
|
170
|
+
- `neuron_scroll` to bottom
|
|
171
|
+
- `neuron_type` into `div[data-testid="dmComposerTextInput"]` or `div[role="textbox"]`
|
|
172
|
+
- `neuron_press_key` Enter
|
|
173
|
+
- Wait 1 second, verify
|
|
174
|
+
|
|
175
|
+
6. **Instagram DMs:**
|
|
176
|
+
- Open the conversation
|
|
177
|
+
- `neuron_scroll` to bottom
|
|
178
|
+
- `neuron_type` into `textarea[placeholder*="Message"]` or `div[contenteditable="true"][role="textbox"]`
|
|
179
|
+
- `neuron_press_key` Enter
|
|
180
|
+
- Wait 1 second, verify
|
|
181
|
+
|
|
182
|
+
7. **TikTok DMs:**
|
|
183
|
+
- Open the conversation
|
|
184
|
+
- `neuron_scroll` to bottom
|
|
185
|
+
- `neuron_type` into the message input (try `div[contenteditable="true"]`, `textarea`)
|
|
186
|
+
- `neuron_press_key` Enter
|
|
187
|
+
- Wait 1 second, verify
|
|
188
|
+
|
|
189
|
+
### Phase 5: Log and checkpoint
|
|
190
|
+
|
|
191
|
+
After processing all messages:
|
|
192
|
+
|
|
193
|
+
1. Update `seen_messages` with all processed message IDs/content hashes
|
|
194
|
+
2. Update `last_checked` timestamps per inbox
|
|
195
|
+
3. `neuron_session_save` with the updated state
|
|
196
|
+
4. Append to `{{output_path}}/response-log.yaml`:
|
|
197
|
+
|
|
198
|
+
```yaml
|
|
199
|
+
- date: "{{now}}"
|
|
200
|
+
inbox: "<platform>"
|
|
201
|
+
sender: "<name>"
|
|
202
|
+
their_message: "<text>"
|
|
203
|
+
draft: "<your drafted reply>"
|
|
204
|
+
approval_status: approved|rejected|timeout
|
|
205
|
+
rejection_reason: "<if rejected, why>"
|
|
206
|
+
sent: true|false
|
|
207
|
+
send_verified: true|false
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
### Scheduling
|
|
211
|
+
|
|
212
|
+
To run this automatically, the user should schedule it:
|
|
213
|
+
|
|
214
|
+
```
|
|
215
|
+
neuron_schedule_recipe({
|
|
216
|
+
slug: "inbox-responder",
|
|
217
|
+
interval_minutes: {{check_interval_minutes}},
|
|
218
|
+
variables: { ... },
|
|
219
|
+
enabled: true
|
|
220
|
+
})
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
Each scheduled run: check all inboxes → draft → approve → send → sleep until next run.
|
|
224
|
+
|
|
225
|
+
## Reflect
|
|
226
|
+
|
|
227
|
+
After each run:
|
|
228
|
+
|
|
229
|
+
```yaml
|
|
230
|
+
date: {{now}}
|
|
231
|
+
outcome:
|
|
232
|
+
inboxes_checked: [<list>]
|
|
233
|
+
new_messages_found: <count>
|
|
234
|
+
drafts_sent_for_approval: <count>
|
|
235
|
+
approved: <count>
|
|
236
|
+
rejected: <count>
|
|
237
|
+
timed_out: <count>
|
|
238
|
+
successfully_sent: <count>
|
|
239
|
+
send_failures: <count>
|
|
240
|
+
skip_auto: <count, skipped by auto_skip patterns>
|
|
241
|
+
duration_minutes: <approx>
|
|
242
|
+
rejection_reasons:
|
|
243
|
+
- "<why was a draft rejected>"
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
## Evolve
|
|
247
|
+
|
|
248
|
+
After 20+ runs with 10+ approval outcomes:
|
|
249
|
+
|
|
250
|
+
**Draft quality:**
|
|
251
|
+
- Which drafts get approved vs rejected? What's different about them?
|
|
252
|
+
- Are certain message types (questions, pitches, follow-ups) harder to draft well?
|
|
253
|
+
- Does reply length correlate with approval rate?
|
|
254
|
+
- Update `{{reply_style}}` defaults based on what gets approved.
|
|
255
|
+
|
|
256
|
+
**Auto-skip refinement:**
|
|
257
|
+
- Are there senders or message patterns that always get skipped manually? Add to `{{auto_skip}}`.
|
|
258
|
+
- Are there messages being auto-skipped that shouldn't be?
|
|
259
|
+
|
|
260
|
+
**Timing:**
|
|
261
|
+
- Is `{{check_interval_minutes}}` right? Too frequent = checking empty inboxes. Too rare = slow responses.
|
|
262
|
+
- Which inboxes get the most new messages?
|
|
263
|
+
|
|
264
|
+
**Rejection patterns:**
|
|
265
|
+
- If drafts are consistently rejected with "too formal" or "too long", update the style guidance.
|
|
266
|
+
- If certain reply patterns are always edited before sending, learn the edits.
|
|
267
|
+
|
|
268
|
+
Update learnings based on the data. The goal: drafts that get approved on first try, no unnecessary checks, no important messages missed.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Learnings
|
|
2
|
+
|
|
3
|
+
No runs yet. This file updates after 20+ runs with 10+ approval outcomes.
|
|
4
|
+
|
|
5
|
+
## Reply Defaults
|
|
6
|
+
|
|
7
|
+
- Match the sender's message length — don't write a paragraph to reply to one sentence
|
|
8
|
+
- Answer the question first, then add context if needed
|
|
9
|
+
- For pitches/cold outreach: "Thanks for sharing — I'll take a look" is almost always the right reply
|
|
10
|
+
- For follow-ups: acknowledge, give a timeline or say you'll get back to them
|
|
11
|
+
- For greetings: brief and warm, don't over-invest
|
|
12
|
+
|
|
13
|
+
## Platform Quirks
|
|
14
|
+
|
|
15
|
+
- LinkedIn: Enter-to-send by default. Use neuron_press_key("Enter") after typing.
|
|
16
|
+
- Gmail: Reply composer lazy-loads. Wait for the contenteditable to appear before typing.
|
|
17
|
+
- X DMs: Enter sends. Similar to LinkedIn flow.
|
|
18
|
+
- Instagram: textarea-based input, Enter sends.
|
|
19
|
+
|
|
20
|
+
## Auto-Skip Patterns
|
|
21
|
+
|
|
22
|
+
Starting list (expand from rejection data):
|
|
23
|
+
- newsletter, no-reply, noreply, automated, unsubscribe
|
|
24
|
+
- "This is an automated message"
|
|
25
|
+
- Sender names containing "bot", "support", "notifications"
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
name: Inbox Responder
|
|
2
|
+
version: 1.0.0
|
|
3
|
+
description: >
|
|
4
|
+
Watches your inboxes (LinkedIn, Gmail, X, Instagram DMs) for new messages,
|
|
5
|
+
drafts context-aware replies, sends them to your WhatsApp for approval, and
|
|
6
|
+
sends the reply on approval. Runs on a schedule — check every N minutes,
|
|
7
|
+
draft, approve from your phone, done.
|
|
8
|
+
author: neuron
|
|
9
|
+
tags: [inbox, auto-reply, whatsapp-approval, linkedin, gmail, x, instagram, monitoring]
|
|
10
|
+
|
|
11
|
+
variables:
|
|
12
|
+
inboxes:
|
|
13
|
+
prompt: "Which inboxes to watch (comma-separated: linkedin, gmail, x, instagram, tiktok)"
|
|
14
|
+
type: text
|
|
15
|
+
required: true
|
|
16
|
+
example: "linkedin, gmail, x, instagram, tiktok"
|
|
17
|
+
check_interval_minutes:
|
|
18
|
+
prompt: "How often to check for new messages (minutes)"
|
|
19
|
+
type: number
|
|
20
|
+
default: 10
|
|
21
|
+
approval_phone:
|
|
22
|
+
prompt: "Your WhatsApp number for approval requests (E.164 format)"
|
|
23
|
+
type: text
|
|
24
|
+
required: true
|
|
25
|
+
example: "+2348012345678"
|
|
26
|
+
neuron_api_key:
|
|
27
|
+
prompt: "Neuron bot API key (nrn_...) for sending WhatsApp approvals"
|
|
28
|
+
type: text
|
|
29
|
+
required: true
|
|
30
|
+
reply_style:
|
|
31
|
+
prompt: "How should replies be written?"
|
|
32
|
+
options: [match-their-tone, professional, casual, brief]
|
|
33
|
+
default: match-their-tone
|
|
34
|
+
auto_skip:
|
|
35
|
+
prompt: "Messages to skip (comma-separated keywords or patterns)"
|
|
36
|
+
type: text
|
|
37
|
+
default: "newsletter, no-reply, noreply, automated, unsubscribe"
|
|
38
|
+
context_notes:
|
|
39
|
+
prompt: "Context about you that helps draft better replies (role, company, current priorities)"
|
|
40
|
+
type: text
|
|
41
|
+
default: ""
|
|
42
|
+
gmail_url:
|
|
43
|
+
prompt: "Gmail URL"
|
|
44
|
+
type: text
|
|
45
|
+
default: "https://mail.google.com/mail/u/0/#inbox"
|
|
46
|
+
output_path:
|
|
47
|
+
prompt: "Where to save response logs"
|
|
48
|
+
type: path
|
|
49
|
+
default: "./inbox-responses"
|
|
50
|
+
|
|
51
|
+
tools:
|
|
52
|
+
required:
|
|
53
|
+
- neuron_navigate
|
|
54
|
+
- neuron_focus_tab
|
|
55
|
+
- neuron_extract_data
|
|
56
|
+
- neuron_find_elements
|
|
57
|
+
- neuron_type
|
|
58
|
+
- neuron_press_key
|
|
59
|
+
- neuron_click
|
|
60
|
+
- neuron_scroll
|
|
61
|
+
- neuron_screenshot
|
|
62
|
+
- neuron_evaluate_js
|
|
63
|
+
- neuron_list_tabs
|
|
64
|
+
- neuron_get_errors
|
|
65
|
+
- neuron_research_page
|
|
66
|
+
- neuron_approve_via_whatsapp
|
|
67
|
+
- neuron_session_save
|
|
68
|
+
- neuron_session_load
|
|
69
|
+
- neuron_session_checkpoint
|
|
70
|
+
- neuron_schedule_recipe
|
|
71
|
+
|
|
72
|
+
pipes:
|
|
73
|
+
outputs:
|
|
74
|
+
response_log:
|
|
75
|
+
format: yaml
|
|
76
|
+
path: "{{output_path}}/response-log.yaml"
|
|
77
|
+
description: "Log of all messages received, drafts sent for approval, and responses sent"
|
|
78
|
+
|
|
79
|
+
limits:
|
|
80
|
+
max_tabs: 4
|
|
81
|
+
max_duration_minutes: 15
|
|
82
|
+
require_human_approval: true
|
package/recipes/planner/agent.md
CHANGED
|
@@ -75,7 +75,54 @@ Search the web for outreach effectiveness data:
|
|
|
75
75
|
|
|
76
76
|
From `learnings.md`, incorporate any patterns from past planning sessions.
|
|
77
77
|
|
|
78
|
-
### Phase 5:
|
|
78
|
+
### Phase 5: Pre-mortem — how will execution fail?
|
|
79
|
+
|
|
80
|
+
This is the most important phase. Before writing the plan, walk through every execution step and ask: **what will go wrong here?** Plans fail at the mechanical level, not the strategic level.
|
|
81
|
+
|
|
82
|
+
**For every step the agent will perform, answer these questions:**
|
|
83
|
+
|
|
84
|
+
1. **Page structure:** How does this page actually work?
|
|
85
|
+
- Is it a SPA that lazy-loads content? → The agent needs to scroll and wait before extracting.
|
|
86
|
+
- Does it use infinite scroll? → The agent needs to scroll to find the right content, not just read what's visible.
|
|
87
|
+
- Are the newest items at the top or the bottom? → Chat/messaging apps show newest at the bottom. Feeds show newest at the top. Getting this wrong means the agent reads stale data.
|
|
88
|
+
- Does the page require a click to expand content (modals, accordions, "Show more")? → Plan the click before the extract.
|
|
89
|
+
|
|
90
|
+
2. **Tab focus:** Will the tab be in the foreground when the agent interacts with it?
|
|
91
|
+
- LinkedIn, Gmail, Facebook, and most modern apps render buttons and dropdowns at zero dimensions in background tabs.
|
|
92
|
+
- If the agent opens multiple tabs, only one is in the foreground. Every click/type/submit must be preceded by `neuron_focus_tab`.
|
|
93
|
+
- Any step involving send/submit/click MUST have "focus tab first" in the plan.
|
|
94
|
+
|
|
95
|
+
3. **Framework compatibility:** How does the page handle input?
|
|
96
|
+
- Does it use React, Ember, Angular, or Vue? → Direct property assignment (`el.value = x`) is invisible to these frameworks. The agent must use `neuron_type` (which uses `execCommand` for contenteditable and native setters for inputs).
|
|
97
|
+
- Does the page use Enter-to-send or a Send button? → If Enter-to-send, plan `neuron_press_key("Enter")` after typing. If Send button, plan `neuron_click`.
|
|
98
|
+
- Does the page block JavaScript eval via CSP? → LinkedIn does. Facebook likely does. Don't plan steps that rely on `neuron_evaluate_js` for these platforms. Use `neuron_click`, `neuron_type`, and `neuron_press_key` instead.
|
|
99
|
+
|
|
100
|
+
4. **Timing and loading:** What needs to load before the agent can act?
|
|
101
|
+
- Rich text editors (Gmail compose, LinkedIn message box) lazy-load. The agent must wait for the contenteditable element to appear before typing.
|
|
102
|
+
- Search results take time to render after submitting a query. Plan a wait.
|
|
103
|
+
- Page navigations need 2-3 seconds before extraction is reliable.
|
|
104
|
+
- After clicking a button (like Reply or Compose), plan a wait for the resulting UI to render.
|
|
105
|
+
|
|
106
|
+
5. **Verification:** How will the agent know the action succeeded?
|
|
107
|
+
- "Click Send" is not verification. Verification is: extract the conversation after sending and confirm the message appears.
|
|
108
|
+
- "Type into the form" is not verification. Verification is: the form field shows the text AND the submit button is enabled.
|
|
109
|
+
- Plan a verification step after every critical action. If the verification fails, plan what to do (retry, skip, alert the user).
|
|
110
|
+
|
|
111
|
+
6. **Edge cases that always happen:**
|
|
112
|
+
- What if there are no new messages? → Don't error, just report "no new messages" and exit.
|
|
113
|
+
- What if the user is logged out? → Detect the login page before trying to interact. Plan for `neuron_detect_blocker`.
|
|
114
|
+
- What if a captcha appears? → Stop immediately, alert the user. Never try to solve captchas.
|
|
115
|
+
- What if the page layout has changed since the selectors were written? → Fall back to `neuron_find_elements` with visible text matching. Text ("Send", "Reply", "Message") is more stable than CSS classes.
|
|
116
|
+
- What if there are too many results and the agent runs out of time? → Set a hard cap per run.
|
|
117
|
+
|
|
118
|
+
**For every selector in the plan, answer:**
|
|
119
|
+
- Is this a CSS class that the platform could rename? → Also plan a text-based fallback.
|
|
120
|
+
- Is this a `data-testid` that's stable? → Better, but still verify it exists before clicking.
|
|
121
|
+
- Can `neuron_find_elements` with `texts: ["Button Label"]` find this instead? → Usually more reliable.
|
|
122
|
+
|
|
123
|
+
**Write the pre-mortem into the plan** as a "Known Failure Modes" section with mitigations for each. This isn't optional — a plan without failure modes is a wish list.
|
|
124
|
+
|
|
125
|
+
### Phase 6: Build the plan
|
|
79
126
|
|
|
80
127
|
Produce `{{output_path}}/{{date}}-{{goal_slug}}.md`:
|
|
81
128
|
|
|
@@ -151,6 +198,24 @@ Produce `{{output_path}}/{{date}}-{{goal_slug}}.md`:
|
|
|
151
198
|
1. [Step-by-step, referencing specific browser tools]
|
|
152
199
|
2. [Each step should map to a neuron_* tool call]
|
|
153
200
|
3. [Include the warm-up / research phase per target]
|
|
201
|
+
4. [Each step that involves clicking or typing MUST include neuron_focus_tab first]
|
|
202
|
+
5. [Each step that reads messages/content MUST scroll to load the latest]
|
|
203
|
+
6. [Each critical action MUST have a verification step after it]
|
|
204
|
+
|
|
205
|
+
## Known Failure Modes
|
|
206
|
+
|
|
207
|
+
For each failure mode, state: what fails, why, how the agent detects it, and what it does instead.
|
|
208
|
+
|
|
209
|
+
| Failure | Detection | Mitigation |
|
|
210
|
+
|---------|-----------|------------|
|
|
211
|
+
| [e.g., Tab in background — send button at zero dimensions] | [neuron_find_elements returns empty or element has zero size] | [neuron_focus_tab before every interaction] |
|
|
212
|
+
| [e.g., CSP blocks evaluateJS on this platform] | [evaluateJS returns CSP error] | [Use neuron_click + neuron_type + neuron_press_key instead] |
|
|
213
|
+
| [e.g., Newest messages at bottom, agent reads top] | [Extracted messages have old timestamps] | [neuron_scroll to bottom before extracting] |
|
|
214
|
+
| [e.g., Framework doesn't detect typed text, buttons stay disabled] | [Send button still disabled after typing] | [neuron_type uses execCommand internally; if still broken, try neuron_press_key Tab to trigger blur/change] |
|
|
215
|
+
| [e.g., Logged out / session expired] | [neuron_detect_blocker finds login wall] | [Stop run, alert user, do not retry] |
|
|
216
|
+
| [e.g., Captcha or rate limit] | [neuron_detect_blocker finds captcha] | [Stop entire session immediately] |
|
|
217
|
+
| [e.g., Page layout changed, selectors broken] | [neuron_find_elements with CSS selector returns empty] | [Fall back to neuron_find_elements with texts: ["Button Label"]] |
|
|
218
|
+
| [e.g., Content lazy-loads after scroll] | [First extraction returns fewer items than expected] | [Scroll + wait 1-2s + re-extract] |
|
|
154
219
|
|
|
155
220
|
## Success Metrics
|
|
156
221
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Learnings
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Updated 2026-09-05 from first live execution failures.
|
|
4
4
|
|
|
5
5
|
## Platform Research Defaults
|
|
6
6
|
|
|
@@ -25,3 +25,51 @@ Starting assumptions (to be validated by actual research each session):
|
|
|
25
25
|
- Action blocks last 24-48 hours
|
|
26
26
|
|
|
27
27
|
These are starting points. ALWAYS research current limits during Phase 2.
|
|
28
|
+
|
|
29
|
+
## Execution Failures — Confirmed (2026-09-05 live testing)
|
|
30
|
+
|
|
31
|
+
These are real failures that occurred during the first live test. Every plan MUST account for them.
|
|
32
|
+
|
|
33
|
+
### Background tab interaction (CRITICAL)
|
|
34
|
+
**What happened:** Agent clicked Send on LinkedIn and Gmail. Both reported success. Neither actually sent — buttons and dropdowns render at zero dimensions in background tabs.
|
|
35
|
+
**Fix:** EVERY plan step that clicks, types, or submits MUST include `neuron_focus_tab` first. No exceptions.
|
|
36
|
+
|
|
37
|
+
### Framework change detection (CRITICAL)
|
|
38
|
+
**What happened:** Agent typed a message into LinkedIn's compose box. The text appeared visually, but LinkedIn's Ember.js framework didn't detect it — the Send button stayed disabled.
|
|
39
|
+
**Why:** Direct DOM property assignment (`el.value = x`, `el.textContent = x`) is invisible to React/Ember/Angular/Vue. These frameworks use their own state management and only detect input from native browser events.
|
|
40
|
+
**Fix:** `neuron_type` now uses `document.execCommand("insertText")` for contenteditable and native prototype setters for inputs. Plans should NOT rely on `neuron_evaluate_js` for typing.
|
|
41
|
+
|
|
42
|
+
### CSP blocking eval (HIGH)
|
|
43
|
+
**What happened:** Agent tried to dispatch keyboard events via `neuron_evaluate_js` on LinkedIn. CSP blocked it: `unsafe-eval` not allowed.
|
|
44
|
+
**Which platforms block eval:** LinkedIn (confirmed). Facebook, Instagram likely.
|
|
45
|
+
**Fix:** Never plan steps that use `neuron_evaluate_js` for interaction on social platforms. Use `neuron_click`, `neuron_type`, `neuron_press_key` instead.
|
|
46
|
+
|
|
47
|
+
### Scroll direction for messages (HIGH)
|
|
48
|
+
**What happened:** Agent extracted messages from a LinkedIn conversation. It read the first messages in the DOM — which were OLD messages from the top. The most recent message (the one to reply to) was at the bottom, off-screen.
|
|
49
|
+
**Why:** All messaging apps (LinkedIn, Gmail threads, X DMs, IG DMs, TikTok DMs) show newest messages at the bottom. The agent must scroll to the bottom before extracting.
|
|
50
|
+
**Fix:** Every plan step that reads messages MUST include "scroll to bottom of conversation" before extracting.
|
|
51
|
+
|
|
52
|
+
### Send verification (MEDIUM)
|
|
53
|
+
**What happened:** Agent claimed "message sent" without verifying. The message was actually still in the compose box.
|
|
54
|
+
**Fix:** After every send action, plan a verification step: wait 1-2 seconds, then extract the conversation and confirm the new message appears. If it doesn't, the send failed.
|
|
55
|
+
|
|
56
|
+
### Lazy-loading content (MEDIUM)
|
|
57
|
+
**What happened:** Gmail's reply composer didn't exist when the agent tried to type into it. The composer loads asynchronously after clicking Reply.
|
|
58
|
+
**Fix:** After any action that triggers new UI (clicking Reply, Compose, opening a modal), plan a wait + `neuron_find_elements` to confirm the target element exists before interacting with it. Don't assume it's there immediately.
|
|
59
|
+
|
|
60
|
+
### Chrome sideload path confusion (LOW)
|
|
61
|
+
**What happened:** Chrome was loading an old extension build from a stale directory. The TOOL_CALL handler didn't exist in the old build.
|
|
62
|
+
**Fix:** Verify the extension version before starting any plan. `neuron_diagnose` returns the extension status.
|
|
63
|
+
|
|
64
|
+
## Pre-mortem Checklist
|
|
65
|
+
|
|
66
|
+
Every plan should be checked against this list before finalizing:
|
|
67
|
+
|
|
68
|
+
- [ ] Every click/type/submit step has `neuron_focus_tab` before it
|
|
69
|
+
- [ ] Every message-reading step scrolls to bottom first
|
|
70
|
+
- [ ] No `neuron_evaluate_js` for interaction on CSP-heavy platforms (LinkedIn, Facebook, Instagram)
|
|
71
|
+
- [ ] Every send/submit step has a verification step after it
|
|
72
|
+
- [ ] Every step that triggers new UI has a wait + find before interacting with the new UI
|
|
73
|
+
- [ ] Login/auth state is checked before starting (neuron_detect_blocker)
|
|
74
|
+
- [ ] There's a plan for what happens if a captcha appears (answer: stop everything)
|
|
75
|
+
- [ ] Selectors have text-based fallbacks for platform DOM changes
|