localagentbrowser 1.61__tar.gz → 1.62__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. {localagentbrowser-1.61 → localagentbrowser-1.62}/CHANGELOG.md +20 -1
  2. {localagentbrowser-1.61 → localagentbrowser-1.62}/MANIFEST.in +1 -0
  3. {localagentbrowser-1.61/localagentbrowser.egg-info → localagentbrowser-1.62}/PKG-INFO +28 -3
  4. {localagentbrowser-1.61 → localagentbrowser-1.62}/README.md +27 -2
  5. localagentbrowser-1.62/agent_browser.py +23 -0
  6. localagentbrowser-1.62/localagentbrowser/__init__.py +26 -0
  7. localagentbrowser-1.62/localagentbrowser/agent.py +406 -0
  8. localagentbrowser-1.62/localagentbrowser/checklist.py +93 -0
  9. localagentbrowser-1.62/localagentbrowser/checks.py +248 -0
  10. localagentbrowser-1.62/localagentbrowser/config.py +96 -0
  11. localagentbrowser-1.62/localagentbrowser/coverage.py +175 -0
  12. localagentbrowser-1.62/localagentbrowser/grounding.py +121 -0
  13. localagentbrowser-1.62/localagentbrowser/pages.py +396 -0
  14. localagentbrowser-1.62/localagentbrowser/prompts.py +104 -0
  15. localagentbrowser-1.62/localagentbrowser/report.py +208 -0
  16. localagentbrowser-1.62/localagentbrowser/text.py +54 -0
  17. {localagentbrowser-1.61 → localagentbrowser-1.62/localagentbrowser.egg-info}/PKG-INFO +28 -3
  18. {localagentbrowser-1.61 → localagentbrowser-1.62}/localagentbrowser.egg-info/SOURCES.txt +13 -0
  19. localagentbrowser-1.62/localagentbrowser.egg-info/entry_points.txt +2 -0
  20. localagentbrowser-1.62/localagentbrowser.egg-info/top_level.txt +2 -0
  21. {localagentbrowser-1.61 → localagentbrowser-1.62}/pyproject.toml +3 -2
  22. localagentbrowser-1.62/tests/golden/prompts.json +2288 -0
  23. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_decisions.py +3 -2
  24. localagentbrowser-1.62/tests/test_golden_prompts.py +133 -0
  25. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_main_loop.py +4 -3
  26. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_model_turns.py +2 -1
  27. {localagentbrowser-1.61 → localagentbrowser-1.62}/tools/live_test.py +3 -2
  28. localagentbrowser-1.61/agent_browser.py +0 -1805
  29. localagentbrowser-1.61/localagentbrowser.egg-info/entry_points.txt +0 -2
  30. localagentbrowser-1.61/localagentbrowser.egg-info/top_level.txt +0 -1
  31. {localagentbrowser-1.61 → localagentbrowser-1.62}/LICENSE +0 -0
  32. {localagentbrowser-1.61 → localagentbrowser-1.62}/localagentbrowser.egg-info/dependency_links.txt +0 -0
  33. {localagentbrowser-1.61 → localagentbrowser-1.62}/localagentbrowser.egg-info/requires.txt +0 -0
  34. {localagentbrowser-1.61 → localagentbrowser-1.62}/setup.cfg +0 -0
  35. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/conftest.py +0 -0
  36. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/fakes.py +0 -0
  37. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_checklist.py +0 -0
  38. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_duplicates.py +0 -0
  39. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_journal.py +0 -0
  40. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_page_text.py +0 -0
  41. {localagentbrowser-1.61 → localagentbrowser-1.62}/tests/test_regression.py +0 -0
  42. {localagentbrowser-1.61 → localagentbrowser-1.62}/tools/regression.py +0 -0
@@ -1,11 +1,30 @@
1
1
  # Changelog
2
2
 
3
- Every version of `agent_browser.py`, newest first. Each released version is also a tagged commit (`v1.11` … `v1.61`); 1.54 to 1.56 and 1.58 to 1.60 were internal steps
3
+ Every version of LocalAgentBrowser, newest first. Each released version is also a tagged commit (`v1.11` … `v1.62`); 1.54 to 1.56 and 1.58 to 1.60 were internal steps
4
4
  measured on the regression suite and released as part of 1.57 and 1.61 (see
5
5
  [docs/regression-history.md](docs/regression-history.md)).
6
6
  Timings are from live runs on the development machine (Intel i5-9500, 6 cores, no GPU, Qwen2.5-7B-Instruct
7
7
  Q4_K_M in LM Studio), where reading the prompt takes about 90% of every model call.
8
8
 
9
+ ## [1.62] - 2026-10-25
10
+ No change in what the agent does: the same prompts, decisions and results as 1.61.
11
+ ### Changed
12
+ - The code is now a package, `localagentbrowser`, with one file per job (settings, prompts, page reading,
13
+ grounding, checklist, coverage, checks, report, and the loop) instead of one 1,805-line `agent_browser.py`. The
14
+ code was moved, not rewritten: every line of the old file is in the package once, in its original order with its
15
+ comments. `python agent_browser.py`, the `localagentbrowser` command and `import agent_browser` work as before.
16
+ The README's Code layout section says which file holds what.
17
+ ### Added
18
+ - Golden prompts (`tests/test_golden_prompts.py`): five scripted runs, including one through the outline pointer,
19
+ a refusal of a finding not on the page, a coverage send-back with a fresh prompt and caveats, compared character
20
+ for character with a recorded copy of every message sent to the model, the journal and the report. It was
21
+ recorded before the move and passed unchanged after it; a one-word change to a check's wording makes it fail and
22
+ name the model call that differs.
23
+
24
+ Regression run on the reorganized code: 7 of 7 finished (168.3 model minutes), against 6 of 7 (222.1) on 1.61. With
25
+ the same behaviour this is run-to-run variation (the documentation task finished in 5 turns this time), not an
26
+ improvement; each task's first prompt was identical to 1.61's, and every check still fired where its path came up.
27
+
9
28
  ## [1.61] - 2026-10-21
10
29
  Released together with the internal steps 1.58, 1.59 and 1.60, each measured on the regression suite
11
30
  ([docs/regression-history.md](docs/regression-history.md) has every step's results, task by task).
@@ -2,3 +2,4 @@
2
2
  include CHANGELOG.md
3
3
  recursive-include tests *.py
4
4
  recursive-include tools *.py
5
+ recursive-include tests *.json
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: localagentbrowser
3
- Version: 1.61
3
+ Version: 1.62
4
4
  Summary: An autonomous web agent driven by a local LLM (LM Studio) and Playwright. Runs entirely on your own machine.
5
5
  Author: SnowFX
6
6
  License-Expression: MIT
@@ -237,6 +237,25 @@ flowchart LR
237
237
  C -->|FINISH| R[research_report.md]
238
238
  ```
239
239
 
240
+ ### Code layout
241
+
242
+ The code is in the `localagentbrowser` package, one file per job, each depending only on the ones above it:
243
+
244
+ | File | What it holds |
245
+ |---|---|
246
+ | `config.py` | Settings: the LM Studio connection (from environment variables) and every limit and threshold |
247
+ | `prompts.py` | Every fixed text the model reads: the system prompt, turn instructions, the checklist, coverage and final-result prompts |
248
+ | `text.py` | Small word helpers the checks share |
249
+ | `pages.py` | Reading the web: search results, page parts, the outline and its pointer, links |
250
+ | `grounding.py` | Whether a finding is really on the page shown, or already recorded |
251
+ | `checklist.py` | The objective split into a fixed checklist, and findings linked to its items |
252
+ | `coverage.py` | The coverage check before finishing, verified in code, and what is reported as not found or not proven |
253
+ | `checks.py` | Decisions refused or confirmed before they are carried out (leaving, reading on, finishing, ...) |
254
+ | `report.py` | The journal, the research log, the final result and the report file |
255
+ | `agent.py` | The loop: each turn's conversation with the model, in-turn retries, and `main()` |
256
+
257
+ `agent_browser.py` keeps `python agent_browser.py` working from a clone.
258
+
240
259
  ### Keeping a small model on track
241
260
 
242
261
  - **Rules enforced in code, not just stated in the prompt.** Repeated searches, visiting the current page,
@@ -307,8 +326,8 @@ the conversation nears the context limit. The journal uses a compact one-line-pe
307
326
  | `LMSTUDIO_API_KEY` | `lm-studio` | API key, if your server requires one |
308
327
  | `LMSTUDIO_CONTEXT_LENGTH` | `8192` | Context length the model is loaded with; the conversation budget follows it |
309
328
 
310
- Behaviour limits (part size, retry counts, check intervals and so on) are named constants at the top of
311
- `agent_browser.py`, each with a comment explaining it.
329
+ Behaviour limits (part size, retry counts, check intervals and so on) are named constants in
330
+ `localagentbrowser/config.py`, each with a comment explaining it.
312
331
 
313
332
  ## Performance
314
333
 
@@ -428,6 +447,12 @@ python -m pytest
428
447
  python -m ruff check .
429
448
  ```
430
449
 
450
+ `tests/test_golden_prompts.py` replays five scripted runs and compares every message sent to the model, and the
451
+ journal and report, with a recorded copy in `tests/golden/prompts.json`. A change meant to leave the agent's
452
+ behaviour alone must keep it passing unchanged; a deliberate change to what the model sees is recorded again with
453
+ `GOLDEN_UPDATE=1 python -m pytest tests/test_golden_prompts.py`, and the diff of the recorded copy reviewed with
454
+ the rest of the change.
455
+
431
456
  To try the real agent with a fixed objective and a time limit, logging the time and tokens of every model
432
457
  call:
433
458
 
@@ -205,6 +205,25 @@ flowchart LR
205
205
  C -->|FINISH| R[research_report.md]
206
206
  ```
207
207
 
208
+ ### Code layout
209
+
210
+ The code is in the `localagentbrowser` package, one file per job, each depending only on the ones above it:
211
+
212
+ | File | What it holds |
213
+ |---|---|
214
+ | `config.py` | Settings: the LM Studio connection (from environment variables) and every limit and threshold |
215
+ | `prompts.py` | Every fixed text the model reads: the system prompt, turn instructions, the checklist, coverage and final-result prompts |
216
+ | `text.py` | Small word helpers the checks share |
217
+ | `pages.py` | Reading the web: search results, page parts, the outline and its pointer, links |
218
+ | `grounding.py` | Whether a finding is really on the page shown, or already recorded |
219
+ | `checklist.py` | The objective split into a fixed checklist, and findings linked to its items |
220
+ | `coverage.py` | The coverage check before finishing, verified in code, and what is reported as not found or not proven |
221
+ | `checks.py` | Decisions refused or confirmed before they are carried out (leaving, reading on, finishing, ...) |
222
+ | `report.py` | The journal, the research log, the final result and the report file |
223
+ | `agent.py` | The loop: each turn's conversation with the model, in-turn retries, and `main()` |
224
+
225
+ `agent_browser.py` keeps `python agent_browser.py` working from a clone.
226
+
208
227
  ### Keeping a small model on track
209
228
 
210
229
  - **Rules enforced in code, not just stated in the prompt.** Repeated searches, visiting the current page,
@@ -275,8 +294,8 @@ the conversation nears the context limit. The journal uses a compact one-line-pe
275
294
  | `LMSTUDIO_API_KEY` | `lm-studio` | API key, if your server requires one |
276
295
  | `LMSTUDIO_CONTEXT_LENGTH` | `8192` | Context length the model is loaded with; the conversation budget follows it |
277
296
 
278
- Behaviour limits (part size, retry counts, check intervals and so on) are named constants at the top of
279
- `agent_browser.py`, each with a comment explaining it.
297
+ Behaviour limits (part size, retry counts, check intervals and so on) are named constants in
298
+ `localagentbrowser/config.py`, each with a comment explaining it.
280
299
 
281
300
  ## Performance
282
301
 
@@ -396,6 +415,12 @@ python -m pytest
396
415
  python -m ruff check .
397
416
  ```
398
417
 
418
+ `tests/test_golden_prompts.py` replays five scripted runs and compares every message sent to the model, and the
419
+ journal and report, with a recorded copy in `tests/golden/prompts.json`. A change meant to leave the agent's
420
+ behaviour alone must keep it passing unchanged; a deliberate change to what the model sees is recorded again with
421
+ `GOLDEN_UPDATE=1 python -m pytest tests/test_golden_prompts.py`, and the diff of the recorded copy reviewed with
422
+ the rest of the change.
423
+
399
424
  To try the real agent with a fixed objective and a time limit, logging the time and tokens of every model
400
425
  call:
401
426
 
@@ -0,0 +1,23 @@
1
+ """LocalAgentBrowser: an autonomous web agent driven by a local language model.
2
+
3
+ Run: python agent_browser.py (or `localagentbrowser` once installed; it asks for the objective)
4
+
5
+ The code lives in the localagentbrowser package (see its __init__.py for how it is organized). This file keeps
6
+ `python agent_browser.py` working from a clone, and `import agent_browser` working for the tests and tools: it
7
+ gives access to every name in the package. To replace a part of the agent in a test, patch the module that
8
+ uses it (for example localagentbrowser.agent.AsyncOpenAI), not this file.
9
+ """
10
+ from localagentbrowser.config import * # noqa: F403
11
+ from localagentbrowser.prompts import * # noqa: F403
12
+ from localagentbrowser.text import * # noqa: F403
13
+ from localagentbrowser.pages import * # noqa: F403
14
+ from localagentbrowser.grounding import * # noqa: F403
15
+ from localagentbrowser.checklist import * # noqa: F403
16
+ from localagentbrowser.coverage import * # noqa: F403
17
+ from localagentbrowser.checks import * # noqa: F403
18
+ from localagentbrowser.report import * # noqa: F403
19
+ from localagentbrowser.agent import * # noqa: F403
20
+ from localagentbrowser.agent import main, run # noqa: F401
21
+
22
+ if __name__ == "__main__":
23
+ run()
@@ -0,0 +1,26 @@
1
+ """LocalAgentBrowser: an autonomous web agent driven by a local language model.
2
+
3
+ A model served by LM Studio decides, one JSON action at a time, whether to SEARCH, NAVIGATE, READ_MORE,
4
+ DISCOVER or FINISH; Playwright carries the actions out in a visible Chromium window. Everything the
5
+ agent does and finds is written to research_log.txt.
6
+
7
+ Run: python agent_browser.py (or `localagentbrowser` once installed; it asks for the objective)
8
+
9
+ Settings (environment variables):
10
+ LMSTUDIO_BASE_URL OpenAI-compatible server address (default http://localhost:1234/v1)
11
+ LMSTUDIO_MODEL model identifier (default qwen2.5-7b-instruct)
12
+ LMSTUDIO_API_KEY API key, if the server needs one (default lm-studio)
13
+ LMSTUDIO_CONTEXT_LENGTH context length the model is loaded with (default 8192)
14
+
15
+ The package, one file per job; each imports only from the ones listed before it:
16
+ config settings and every limit and threshold
17
+ prompts every fixed text the model reads
18
+ text small word helpers the checks share
19
+ pages search results, page parts, the outline and its pointer, links
20
+ grounding whether a finding is really on the page shown, or already recorded
21
+ checklist the objective split into a fixed checklist; findings linked to its items
22
+ coverage the coverage check before finishing, verified in code
23
+ checks decisions refused or confirmed before they are carried out
24
+ report the journal, research log, final result and report file
25
+ agent the loop: each turn's conversation with the model, and main()
26
+ """
@@ -0,0 +1,406 @@
1
+ """The agent loop: each turn's conversation with the model, the fresh prompt, in-turn retries, and main()."""
2
+ import asyncio
3
+ import json
4
+ import re
5
+ from openai import AsyncOpenAI
6
+ from playwright.async_api import async_playwright
7
+ from .config import API_KEY, BASE_URL, MAX_CHEAP_RETRIES, MAX_CONVERSATION_TOKENS, MAX_DISCOVERY_PROMPT_CHARS, MAX_FOLLOWUPS, MAX_JSON_RETRIES, MAX_VISITS_PER_URL, MODEL, OBJECTIVE_CHECK_EVERY, REPORT_FILE, STALL_TURNS, TURN_PAUSE_SECONDS
8
+ from .prompts import FINAL_RESULT, FIRST_TURN_INSTRUCTION, GIVE_UP_RULE, READ_FIRST, RESPONSE_FORMAT, SYSTEM_PROMPT, TURN_INSTRUCTIONS
9
+ from .text import extract_message_content
10
+ from .pages import build_environment, count_search_results, determine_url, load_page, normalize_url, read_more_target, render_part, search_query, where_you_are
11
+ from .checklist import after_discovery_advice, check_parts, checklist_lines, link_discovery, split_objective
12
+ from .coverage import coverage_refusal, not_found_items
13
+ from .checks import check_decision, escalation_message, numbered_discoveries, numbered_lines, refusal_correction, source_note
14
+ from .report import compact_journal, compose_final_result, visited_summary, write_report, write_research_log
15
+
16
+ def new_conversation():
17
+ return {"messages": None, "followups": 0, "tokens": 0, "note": None, "note_url": None, "shown_links": set()}
18
+
19
+ def set_note(conversation, note, current_url):
20
+ # A note is only used if the agent is still on the page it describes when the next turn starts
21
+ conversation.update(note=note, note_url=normalize_url(current_url) if current_url else None)
22
+
23
+ async def run_agent_turn(client, page, objective, journal, current_url, tried, page_state, conversation, loop_warning=None):
24
+ warning_prompt = f"\n\n⚠️ LOOP DETECTION WARNING: {loop_warning}" if loop_warning else ""
25
+
26
+ # Turns continue the previous conversation: LM Studio has already processed all of it, so only the
27
+ # note about what changed (a result, a new page part, or a whole new page) is new. Too many
28
+ # follow-ups, or a conversation nearing the context limit, starts over with a full prompt.
29
+ note = conversation["note"]
30
+ estimated_tokens = conversation["tokens"] + len(note or "") // 3
31
+ if (conversation["messages"] and note and current_url
32
+ and conversation["note_url"] == normalize_url(current_url)
33
+ and conversation["followups"] < MAX_FOLLOWUPS
34
+ and estimated_tokens < MAX_CONVERSATION_TOKENS):
35
+ conversation["followups"] += 1
36
+ print(f"⚡ Continuing the conversation (follow-up {conversation['followups']}/{MAX_FOLLOWUPS}, ~{estimated_tokens} tokens)")
37
+ messages = conversation["messages"] + [{"role": "user", "content": f"{note}{warning_prompt}\n\n{TURN_INSTRUCTIONS}"}]
38
+ else:
39
+ conversation.update(followups=0, shown_links=set())
40
+ messages = await build_full_prompt(page, objective, journal, current_url, tried, page_state, warning_prompt, conversation["shown_links"])
41
+ conversation["note"] = None
42
+
43
+ decision, refused, messages, tokens = await request_decision(client, messages, journal, tried, page_state, current_url)
44
+ conversation.update(messages=messages, tokens=tokens)
45
+ return decision, refused
46
+
47
+ async def build_full_prompt(page, objective, journal, current_url, tried, page_state, warning_prompt, shown_links=None):
48
+ print("🔍 Scraping and preparing current environment text layout...")
49
+ environment = await build_environment(page, current_url, page_state, {normalize_url(u) for u in tried["urls"]}, shown_links)
50
+
51
+ # Shortened in the prompt (the full text is in the log and the report, and duplicates are checked against
52
+ # it in code): the growing discoveries list had become a quarter of every fresh prompt
53
+ discoveries = "\n".join(numbered_lines(journal, MAX_DISCOVERY_PROMPT_CHARS)) or "[None yet]"
54
+ recent_journal = compact_journal(journal)
55
+
56
+ # LM Studio only reuses the part of the prompt that is identical to the previous turn's, up to the
57
+ # first change. Sections are ordered from least to most often changed, carry no counters, and the
58
+ # journal only grows at its end, so each turn mostly adds text instead of changing earlier text.
59
+ # The page content goes last, where small models pay the most attention.
60
+ user_payload = f"""### HIGH-LEVEL OBJECTIVE
61
+ {objective}
62
+
63
+ ### DISCOVERIES SO FAR
64
+ {discoveries}
65
+
66
+ ### OBJECTIVE CHECKLIST (every part of the objective, with the discoveries recorded for each)
67
+ {checklist_lines(tried.get("checklist")) or "[None]"}
68
+
69
+ ### RECENT RESEARCH JOURNAL
70
+ {recent_journal}
71
+
72
+ ### ALREADY TRIED
73
+ Queries (do not repeat): {json.dumps(tried["queries"]) if tried["queries"] else "[None]"}
74
+ URLs visited (you may return to one that still has information you need, at most {MAX_VISITS_PER_URL} visits each): {visited_summary(tried["urls"])}
75
+
76
+ ### CURRENT ENVIRONMENT
77
+ Active URL: {current_url if current_url else "None (Loop Initializing)"}
78
+ {environment}{warning_prompt}
79
+
80
+ {"" if journal else FIRST_TURN_INSTRUCTION}{TURN_INSTRUCTIONS}"""
81
+ return [
82
+ {"role": "system", "content": SYSTEM_PROMPT},
83
+ {"role": "user", "content": user_payload}
84
+ ]
85
+
86
+ PATH_ESCAPES = {"\t": "\\t", "\r": "\\r", "\b": "\\b", "\f": "\\f"}
87
+
88
+ def restore_path_escapes(text):
89
+ """Undoes JSON escapes the model did not mean: recording the Windows command "python -m venv
90
+ C:\\path\\to\\new\\virtual\\environment", it wrote single backslashes, and its \\t, \\n and \\r became a tab, a
91
+ newline and a carriage return. A tab, carriage return, backspace or form feed right before a letter is
92
+ treated as such a path; a newline is only restored where it sits between word characters (as in "to\\new").
93
+ Only text that also has a backslash or a drive letter is touched: page tables are read with real tabs."""
94
+ if not re.search(r"[\t\r\b\f][A-Za-z]", text) or not re.search(r"\\|\b[A-Za-z]:[\\\"\t\r\b\f\n]", text):
95
+ return text
96
+ text = re.sub(r"(?<=\w)\n(?=[a-z])", r"\\n", text)
97
+ return re.sub(r"[\t\r\b\f](?=[A-Za-z])", lambda m: PATH_ESCAPES[m.group()], text)
98
+
99
+ async def request_decision(client, messages, journal, tried, page_state, current_url):
100
+ # Cheap retries: a reply that is invalid or would be refused is answered with a short correction
101
+ # appended to the same conversation. Everything before the correction is identical to what LM Studio
102
+ # just processed, so it only reads the correction instead of the whole prompt again. Resending the
103
+ # prompt unchanged would not help: the same prompt returns the same answer.
104
+ messages = list(messages)
105
+ refused = []
106
+ json_failures = 0
107
+ while True:
108
+ response = await client.chat.completions.create(
109
+ model=MODEL,
110
+ messages=messages,
111
+ temperature=0.2,
112
+ response_format=RESPONSE_FORMAT
113
+ )
114
+ original_content = extract_message_content(response)
115
+ raw_content = original_content
116
+
117
+ if raw_content.startswith("```"):
118
+ raw_content = re.sub(r'^```json\s*|^```\s*|```$', '', raw_content, flags=re.MULTILINE).strip()
119
+
120
+ try:
121
+ action_decision = json.loads(raw_content)
122
+ if not isinstance(action_decision, dict):
123
+ raise ValueError("the reply was JSON but not an object")
124
+ for key in ("parameter", "thought_process"):
125
+ if isinstance(action_decision.get(key), str):
126
+ action_decision[key] = restore_path_escapes(action_decision[key])
127
+ except (json.JSONDecodeError, ValueError) as parse_err:
128
+ json_failures += 1
129
+ print(f"⚠️ Model returned invalid JSON (attempt {json_failures}/{MAX_JSON_RETRIES}): {raw_content[:200]}")
130
+ if json_failures >= MAX_JSON_RETRIES:
131
+ raise ValueError(f"No valid JSON decision after {MAX_JSON_RETRIES} attempts.") from parse_err
132
+ messages += [{"role": "assistant", "content": original_content},
133
+ {"role": "user", "content": f"That reply was not a valid JSON object ({parse_err}). Reply with only the JSON object in the mandatory format."}]
134
+ continue
135
+
136
+ action = str(action_decision.get("action", "FINISH")).upper()
137
+ parameter = str(action_decision.get("parameter", "")).strip()
138
+ refusal = check_decision(action, parameter, journal, tried, page_state, current_url,
139
+ action_decision.get("thought_process", ""))
140
+ if not refusal or len(refused) >= MAX_CHEAP_RETRIES:
141
+ usage = getattr(response, "usage", None)
142
+ tokens = (getattr(usage, "prompt_tokens", 0) or 0) + (getattr(usage, "completion_tokens", 0) or 0)
143
+ return action_decision, refused, messages + [{"role": "assistant", "content": original_content}], tokens
144
+
145
+ refused.append({"action": action, "parameter": parameter, "journal_action": refusal[0], "reason": refusal[1]})
146
+ print(f"♻️ Cheap retry {len(refused)}/{MAX_CHEAP_RETRIES}: [{action}] refused. {refusal[1]}")
147
+ correction = refusal_correction(action, refusal[1], page_state, current_url)
148
+ if refusal[0].startswith("CONFIRM_"):
149
+ # Not a true refusal: repeating the action is an allowed answer, so no "do not repeat" wording
150
+ correction = f"CHECK: {refusal[1]} Reply with only the JSON object."
151
+ elif refusal[0] == "BLOCKED_NOT_ON_PAGE" and len(refused) < 2:
152
+ # The refusal says what to do instead (DISCOVER the quoted sentence or read on); with the usual "Do NOT
153
+ # choose DISCOVER ... choose a different action" around it, the model read on 3 times out of 3 past the
154
+ # sentence it was shown
155
+ correction = f"REFUSED: {refusal[1]} Reply with only the JSON object."
156
+ elif len(refused) >= 2:
157
+ correction = escalation_message(len(refused), action, refusal[1], page_state, current_url) + "\nReply with only the JSON object."
158
+ messages += [{"role": "assistant", "content": original_content}, {"role": "user", "content": correction}]
159
+
160
+ async def main():
161
+ client = AsyncOpenAI(
162
+ base_url=BASE_URL,
163
+ api_key=API_KEY
164
+ )
165
+
166
+ print("🤖 ==================================================== 🤖")
167
+ print("🚀 Initializing Autonomous Discovery Agent Setup...")
168
+ objective = input("🎯 Define the Agent's High-Level Objective: ").strip()
169
+ if not objective:
170
+ print("⚠️ Objective cannot be empty.")
171
+ return
172
+
173
+ journal = []
174
+ current_url = None
175
+ turn_counter = 1
176
+ max_turns = 100
177
+ tried = {"queries": [], "urls": [], "objective": objective}
178
+ print("📋 Splitting the objective into a checklist...")
179
+ tried["checklist"] = await split_objective(client, objective)
180
+ print("📋 Objective checklist:\n" + checklist_lines(tried["checklist"]))
181
+ page_state = {"url": None, "parts": [], "index": 0}
182
+ conversation = new_conversation()
183
+ loop_msg = None
184
+ consecutive_refusals = 0
185
+ turns_without_discovery = 0
186
+ turns_since_objective_check = 0
187
+
188
+ async with async_playwright() as p:
189
+ browser = await p.chromium.launch(headless=False)
190
+ # No user_agent override: headed Chromium already reports a real, version-matched Chrome UA
191
+ context = await browser.new_context(
192
+ java_script_enabled=True,
193
+ viewport={"width": 1280, "height": 720}
194
+ )
195
+ page = await context.new_page()
196
+
197
+ try:
198
+ while turn_counter <= max_turns:
199
+ print(f"\n🧠 [TURN {turn_counter}/{max_turns}] Active reasoning cycle executing...")
200
+ tried["finish_confirmed"] = False # The finish check runs once per turn
201
+
202
+ try:
203
+ decision, refused = await run_agent_turn(client, page, objective, journal, current_url, tried, page_state, conversation, loop_warning=loop_msg)
204
+ thought = decision.get("thought_process", "No reasoning recorded.")
205
+ action = str(decision.get("action", "FINISH")).upper()
206
+ parameter = str(decision.get("parameter", "")).strip()
207
+ loop_msg = None
208
+ except ValueError as parse_err:
209
+ print(f"⚠️ {parse_err} Skipping turn.")
210
+ journal.append({"turn": turn_counter, "action": "INVALID_OUTPUT", "error": str(parse_err)})
211
+ turn_counter += 1
212
+ continue
213
+ except Exception as err:
214
+ print(f"❌ Local AI API transaction failed: {err}")
215
+ print(f"💡 Check that LM Studio is running at {BASE_URL} and model '{MODEL}' is loaded.")
216
+ break
217
+
218
+ # Replies refused and retried within this turn stay in the journal so the model sees what failed;
219
+ # "check" names the check that refused them, for counting in the regression runs (not shown to the model)
220
+ for r in refused:
221
+ journal.append({"turn": turn_counter, "action": "REFUSED_IN_TURN", "requested": f"{r['action']} {r['parameter'][:120]}", "reason": r["reason"],
222
+ "check": r["journal_action"]})
223
+
224
+ print(f"💡 Agent Thoughts: {thought}")
225
+ print(f"🎬 Chosen Action: [{action}] -> Parameter: {parameter}")
226
+
227
+ refusal = check_decision(action, parameter, journal, tried, page_state, current_url, thought)
228
+ if not refusal and action == "FINISH":
229
+ refusal = await coverage_refusal(client, objective, tried, journal)
230
+ if refusal:
231
+ # Only reached once this turn's cheap retries are used up
232
+ print(f"⚠️ Action refused: {refusal[1]}")
233
+ journal.append({"turn": turn_counter, "thought": thought, "action": refusal[0], "requested": parameter, "reason": refusal[1]})
234
+ loop_msg = f"Your {action} was refused. {refusal[1]}"
235
+ set_note(conversation, f"RESULT: Nothing changed. {where_you_are(page_state, current_url)}", current_url)
236
+ if refusal[0] == "BLOCKED_COVERAGE":
237
+ # The coverage prompts pushed the conversation out of LM Studio's cache, and re-reading it took
238
+ # 626-651 s; a fresh prompt is shorter and starts with the system prompt they left cached
239
+ conversation["messages"] = None
240
+
241
+ elif action == "FINISH":
242
+ print("\n🎯 Agent has signaled mission complete. Writing the final result...")
243
+ finish_entry = {"turn": turn_counter, "thought": thought, "action": "FINISH", "finding": parameter}
244
+ journal.append(finish_entry)
245
+ try:
246
+ missing, unproven = await not_found_items(client, objective, tried, journal)
247
+ final = await compose_final_result(client, objective, journal, tried.get("checklist"), missing,
248
+ unproven)
249
+ if final:
250
+ finish_entry.update(draft=parameter, finding=final)
251
+ except Exception as err:
252
+ print(f"⚠️ Could not write the final result separately ({err}); keeping the agent's FINISH text.")
253
+ break
254
+
255
+ elif action in ["SEARCH", "NAVIGATE"]:
256
+ target_url = determine_url(action, parameter)
257
+ print(f"🌐 Driving Playwright to cross-domain target: {target_url}")
258
+ try:
259
+ response = await load_page(page, target_url)
260
+ current_url = page.url
261
+ status = getattr(response, "status", None) or 0
262
+
263
+ if action == "SEARCH":
264
+ # The search engine sometimes returns an error page; reload once before giving up
265
+ if await count_search_results(page) == 0:
266
+ print("⚠️ Search returned no results. Retrying once...")
267
+ await asyncio.sleep(3)
268
+ await load_page(page)
269
+ current_url = page.url
270
+ if await count_search_results(page) == 0:
271
+ print("❌ Search still returned no results.")
272
+ journal.append({"turn": turn_counter, "thought": thought, "action": "FAILED_SEARCH", "requested": parameter,
273
+ "error": "No results (search engine error or no matches). This query may be tried again."})
274
+ else:
275
+ tried["queries"].append(search_query(parameter))
276
+ journal.append({"turn": turn_counter, "thought": thought, "action": action, "requested": parameter, "destination": current_url})
277
+ else:
278
+ tried["urls"].append(target_url)
279
+ if normalize_url(current_url) != normalize_url(target_url):
280
+ tried["urls"].append(current_url) # Also remember where redirects landed
281
+ if status >= 400:
282
+ # Often a guessed URL; reported as a failure so the model does not treat the
283
+ # error page as content (and is not asked to record something from it)
284
+ print(f"❌ The page returned HTTP {status}.")
285
+ journal.append({"turn": turn_counter, "thought": thought, "action": "FAILED_NAVIGATE", "requested": parameter,
286
+ "error": f"The page returned HTTP {status} (for example 404: not found). "
287
+ "Use a link shown on a page or in search results instead of guessing URLs."})
288
+ else:
289
+ journal.append({"turn": turn_counter, "thought": thought, "action": action, "requested": parameter, "destination": current_url})
290
+ except Exception as b_err:
291
+ print(f"❌ Failed to reach target destination: {b_err}")
292
+ if action == "NAVIGATE":
293
+ tried["urls"].append(target_url)
294
+ journal.append({"turn": turn_counter, "thought": thought, "action": f"FAILED_{action}", "requested": parameter, "error": str(b_err)})
295
+
296
+ # The new page joins the running conversation instead of forcing a full prompt rebuild
297
+ outcome = journal[-1]
298
+ if "error" in outcome:
299
+ result = f"RESULT: {outcome['action']} '{parameter}' failed: {outcome['error'][:200]}"
300
+ else:
301
+ added = search_query(parameter) if action == "SEARCH" else target_url
302
+ result = f"RESULT: {action} '{parameter}' done. ALREADY TRIED now also includes: {added}"
303
+ if action == "NAVIGATE":
304
+ visits = [normalize_url(u) for u in tried["urls"]].count(normalize_url(target_url))
305
+ result = f"RESULT: NAVIGATE '{parameter}' done (visit {visits} of {MAX_VISITS_PER_URL} allowed for this URL)."
306
+ environment = await build_environment(page, current_url, page_state, {normalize_url(u) for u in tried["urls"]}, conversation["shown_links"])
307
+ if "error" in outcome and page_state["url"] == normalize_url(current_url):
308
+ page_state["leave_confirmed"] = True # Nothing to record on an error page
309
+ set_note(conversation, f"{result}\n\n### CURRENT ENVIRONMENT (replaces the earlier one)\n"
310
+ f"Active URL: {current_url}\n{environment}", current_url)
311
+
312
+ elif action == "READ_MORE":
313
+ target = read_more_target(parameter, page_state)
314
+ page_state["index"] = target
315
+ page_state.setdefault("seen_parts", {0}).add(target)
316
+ print(f"📖 Showing part {target + 1} of {len(page_state['parts'])} of the current page.")
317
+ journal.append({"turn": turn_counter, "thought": thought, "action": "READ_MORE", "part": f"{target + 1} of {len(page_state['parts'])}"})
318
+ set_note(conversation, f"RESULT: READ_MORE done. {where_you_are(page_state, current_url)} "
319
+ f"The CURRENT ENVIRONMENT now shows:\n{render_part(page_state)}\n"
320
+ f"The links listed earlier for this page still apply.", current_url)
321
+
322
+ elif action == "DISCOVER":
323
+ print(f"📝 Logging critical discovery element: {parameter}")
324
+ number = sum(1 for e in journal if e.get("action") == "DISCOVER") + 1
325
+ linked = link_discovery(tried["checklist"], decision.get("checklist_items"), number)
326
+ journal.append({"turn": turn_counter, "thought": thought, "action": "DISCOVER", "source": current_url,
327
+ "extracted_data": parameter, "checklist_items": linked})
328
+ page_state["visit_discoveries"] = page_state.get("visit_discoveries", 0) + 1
329
+ page_state.setdefault("recorded_parts", set()).add(page_state["index"])
330
+ # A discovery is when the objective can become complete; on a one-answer question the model
331
+ # recorded the full answer, said "the objective has been met", then kept reading for
332
+ # "interesting" details, so completion is checked right here. On an explanation task it recorded
333
+ # the Calvin cycle step by step while the light reactions were still missing, so the note also
334
+ # points to the parts not yet covered, and the checklist with its discoveries shows which ones are open
335
+ linked_note = (f" for checklist item {', '.join(map(str, linked))}" if linked else
336
+ " (linked to no checklist item)")
337
+ set_note(conversation, f"RESULT: Your DISCOVER was recorded as discovery [{number}]{linked_note}. "
338
+ f"{where_you_are(page_state, current_url)} The page is unchanged.\n"
339
+ f"OBJECTIVE CHECKLIST now:\n{checklist_lines(tried['checklist'])}\n"
340
+ f"Do not record the same information again. {after_discovery_advice(tried['checklist'])}",
341
+ current_url)
342
+
343
+ # A single refusal gets a short warning; repeated refusals get an explicit list of other options
344
+ last_entry = journal[-1]["action"] if journal else ""
345
+ if last_entry.startswith(("BLOCKED_", "UNKNOWN_")) or last_entry == "DUPLICATE_DISCOVER":
346
+ consecutive_refusals += 1
347
+ if consecutive_refusals >= 2:
348
+ print(f"⚠️ {consecutive_refusals} refused actions in a row. Escalating loop warning.")
349
+ loop_msg = escalation_message(consecutive_refusals, action, journal[-1].get("reason", ""), page_state, current_url)
350
+ else:
351
+ consecutive_refusals = 0
352
+
353
+ # Without a nudge the model keeps browsing after it already has enough; after several turns
354
+ # with no new discovery it is asked to weigh its discoveries against the objective
355
+ if action == "DISCOVER" and last_entry == "DISCOVER":
356
+ turns_without_discovery = 0
357
+ else:
358
+ turns_without_discovery += 1
359
+ # Fires every STALL_TURNS stalled turns rather than every turn: repeated each turn it crowded out
360
+ # the current page and pushed the model to navigate away without reading what was in front of it
361
+ if turns_without_discovery and turns_without_discovery % STALL_TURNS == 0:
362
+ print(f"⚠️ No new discoveries in {turns_without_discovery} turns. Adding progress check.")
363
+ progress = (f"PROGRESS CHECK: no new discoveries in the last {turns_without_discovery} turns."
364
+ f"{numbered_discoveries(journal)} {READ_FIRST} "
365
+ f"Then {check_parts(tried)} If every part is covered, choose FINISH with {FINAL_RESULT} in the "
366
+ "parameter. If not, state in thought_process exactly what is still missing, then go and find "
367
+ f"that specific thing. {GIVE_UP_RULE}")
368
+ loop_msg = f"{loop_msg}\n{progress}" if loop_msg else progress
369
+ turns_since_objective_check = 0
370
+ else:
371
+ # Discoveries on loosely related topics keep resetting the stall count, so the objective is
372
+ # also re-checked on a fixed schedule to stop the model drifting away from it
373
+ turns_since_objective_check += 1
374
+ if turns_since_objective_check >= OBJECTIVE_CHECK_EVERY:
375
+ turns_since_objective_check = 0
376
+ count = sum(1 for e in journal if e.get("action") == "DISCOVER")
377
+ print(f"🧭 Objective check after {turn_counter} turns ({count} discoveries).")
378
+ check = (f"OBJECTIVE CHECK: you have recorded {count} discoveries in {turn_counter} turns."
379
+ f"{numbered_discoveries(journal)} "
380
+ f"{source_note(journal)} {READ_FIRST} "
381
+ f"Then {check_parts(tried)} If every part is covered, choose FINISH with {FINAL_RESULT} in the "
382
+ "parameter. If not, state in thought_process exactly what is still missing and go directly for it; "
383
+ f"do not spend turns on topics outside the objective. {GIVE_UP_RULE}")
384
+ loop_msg = f"{loop_msg}\n{check}" if loop_msg else check
385
+
386
+ # Saved every turn as well, so a crash or a killed process still leaves a current report
387
+ write_research_log(objective, journal, tried.get("checklist"))
388
+ write_report(objective, journal, tried.get("checklist"))
389
+
390
+ turn_counter += 1
391
+ print("\n⏱️ Cycle pacing check complete. Refreshing navigation loop pipeline...")
392
+ await asyncio.sleep(TURN_PAUSE_SECONDS)
393
+ finally:
394
+ # Always saved, however the run ends: FINISH, the turn limit, Ctrl+C or an outside time limit
395
+ write_research_log(objective, journal, tried.get("checklist"))
396
+ write_report(objective, journal, tried.get("checklist"))
397
+
398
+ print(f"\n🔒 Shutting down browser instance. Report: {REPORT_FILE}; full journal: research_log.txt.")
399
+ await browser.close()
400
+
401
+ def run():
402
+ """Command-line entry point (installed as `localagentbrowser`)."""
403
+ try:
404
+ asyncio.run(main())
405
+ except KeyboardInterrupt:
406
+ print("\n⏹️ Stopped. Findings so far are in research_log.txt.")