localagentbrowser 1.62__tar.gz → 1.64__tar.gz

{localagentbrowser-1.62 → localagentbrowser-1.64}/CHANGELOG.md

@@ -1,11 +1,162 @@
 # Changelog
 
-Every version of LocalAgentBrowser, newest first. Each released version is also a tagged commit (`v1.11` … `v1.62`); 1.54 to 1.56 and 1.58 to 1.60 were internal steps
+Every version of LocalAgentBrowser, newest first. Each released version is also a tagged commit (`v1.11` … `v1.63`); 1.54 to 1.56 and 1.58 to 1.60 were internal steps
 measured on the regression suite and released as part of 1.57 and 1.61 (see
 [docs/regression-history.md](docs/regression-history.md)).
 Timings are from live runs on the development machine (Intel i5-9500, 6 cores, no GPU, Qwen2.5-7B-Instruct
 Q4_K_M in LM Studio), where reading the prompt takes about 90% of every model call.
 
+## [1.64] - 2026-11-08
+The theme is **silent omission** ([issue #11](https://github.com/SnowFXXX/LocalAgentBrowser/issues/11)): a part of
+the objective dropping out of the final answer without being marked "not found". It was measured on two PCs (PC A
+and PC B, both Intel i5-9500, no GPU) and then on four models with identical code.
+
+### Added
+- **Windows launcher and shortcuts.** `tools/windows/LocalAgentBrowser.cmd` checks that LM Studio is answering,
+  runs the agent in its own folder under `Documents\LocalAgentBrowser runs`, and opens the report when the run ends.
+  `tools/windows/make_shortcuts.ps1` adds Start-menu and desktop shortcuts with the project icon (from a clone of
+  the repository).
+- **The report opens by itself** when a launcher run ends: `research_report.md` is written next to it as a dark web
+  page, in the project site's colors, with the final result, checklist and discoveries in separate cards and
+  citations shown as tags (`tools/windows/show_report.py`, standard library only). Windows has no program for `.md`
+  files by default.
+- Regression suite: every run folder gets `run_info.json` (PC label from `LAB_PC_LABEL`, commit, and a fingerprint
+  of the uncommitted changes), so runs on several PCs can be tied to the exact code they ran.
+- Regression suite: `LAB_BLOCKED_SITES` makes NAVIGATE refuse given sites, for benchmark runs that must not read the
+  benchmark's own answer pages.
+
+### Fixed: parts of the objective dropping out of the answer
+- **The coverage check asks for the quote first.** "Does this discovery answer the item?" is now answered by first
+  copying the words that answer it, with a rule that words on the same topic but a different aspect don't count.
+  On 30 hand-labelled real cases, 25 right instead of 16 (no false "not found", 5 false "found" left).
+- **A part credited to several items only counts where the item's own words appear**, so one quote can't silently
+  cover three parts; such an item is sent back as unanswered.
+- **Missing amounts and concrete missing terms send the agent back.** When an item asks for amounts and no discovery
+  for it gives one, or the check names a concrete missing term ("O2") that no discovery contains, the agent is told
+  to find it, instead of finishing without it (an explanation once finished in 4 turns without oxygen).
+- **Each fact stays under its own part.** A discovery keeps the page heading it came from, including headings inside
+  a discovery that crosses page sections; a heading the discovery starts with isn't replaced by the one above;
+  bylines, very long "headings" (numbered sentences) and the prompt's own section names don't count as headings or
+  names.
+- **Names no discovery gives are cut,** with a note: a short cited heading's name is checked too, word forms need
+  an 80% shared start ("PhotoScape" over a PhotoSplice source), and words in a source's web address count as given.
+- **A sentence under one label copied from another subject's sentence is cut** (SQLite's sentence under
+  PostgreSQL).
+- **Numbers in words are checked like digits** ("two weeks" needs a 2 in a source, "one cup" equals "1 cup"), unit
+  abbreviations count as their words ("5,895 m (19,341 ft)" answers "in meters" and "in feet"), and a time needs
+  the same number as a time in a source ("2 weeks" was taken from a "2L jar").
+- **Citations:** citation lines on their own are joined to the line they cite; citations are moved out of code, and
+  unclosed code blocks closed, so correct commands aren't cut as invented.
+- A **"Recommendation" section is labelled as the model's view**, like "My view:".
+- **NAVIGATE refuses text that isn't a web address** (a run lost 5 turns on it).
+
+### Fixed: the regression suite's scoring
+- The code's own "Left out, because…" notes are no longer read as the result's claims (commands and uncited lines).
+  4 of 78 earlier run folders were over-counted.
+- A bold label ("**1. Creating a virtual environment:**") is a heading, not an uncited claim.
+
+### Measured
+- 7B on two PCs: 7 of 7 on both PCs with v11 (297.8 and 269.6 model minutes), with no outside knowledge, invented
+  numbers or invented commands in any finished result.
+- Four models, same code (v12, 90 minutes per task): Qwen2.5-7B 6 of 7, Llama 3.1 8B 7 of 7, Qwen2.5-14B 5 of 7,
+  Qwen2.5-3B 3 of 7. On 1.63 they were 6, 2, 2 and 0 of 7. No finished result used outside knowledge. Details in
+  docs/regression-history.md.
+
+## [1.63] - 2026-11-01
+### Added
+- A model call the server fails is tried again after a wait (5, 15, 30, 60 and 120 seconds, about 4 minutes in all),
+  instead of ending the run. LM Studio loads a model by itself when a request names one that is not loaded, and
+  unloads idle ones, so this can carry a run through an unloaded model (tested) or, if the app comes back within
+  about 4 minutes, a restarted app (not yet tested). Tested live: with the model
+  unloaded in the middle of turn 2, LM Studio refused that call ("400 - Model unloaded by user or API request"), the
+  agent waited 5 seconds, LM Studio loaded the model again, and the run finished with the right answer (before, the
+  same unload ended the run). Only server failures are retried; that refusal is the one 400 that is.
+- Regression suite: `--model` runs the tasks with another model LM Studio has; the model is loaded first and its
+  context length checked, so every model is measured under the same conditions, and the run's folder and summary
+  name it.
+- Regression suite: honesty measures for each finished result: what it says was not found, the code's caveats,
+  gaps filled from the model's own knowledge ("the text does not specify, but it is generally recommended ..."),
+  numbers no discovery contains, uncited lines and citations of missing discoveries, in a table in every summary.
+### Fixed (from the first tests on other models: Qwen2.5 3B, Llama 3.1 8B, Qwen2.5 14B)
+- A runaway reply no longer stalls a run. A 3B model kept writing one turn's reply for 10 minutes (2,500+ tokens,
+  where real replies are at most a few hundred) until the connection timed out, and the request was then sent again;
+  two of its tasks never got past one turn. A browsing reply is now capped at 600 tokens, a timeout is not retried
+  (the model is still writing, not lost), and the client's own silent retries are off.
+- The final result is asked for in a required field, `{"result": "..."}`, like every browsing reply. Asked for plain
+  text, Llama 3.1 8B answered 3 of 6 results with a made-up tool call (`{"name": "get_mountain_info", ...}`), and a 3B
+  model "finished" a how-to with its own prompt repeated back. A reply that is still JSON or repeats the prompt's
+  sections is asked for once more; if that fails too, the report says no result was written and points to the
+  findings, instead of showing the broken reply as a result.
+- A finding recorded on the search results page is refused: "NAVIGATE to the result that looks right and DISCOVER from
+  the page itself". A result's snippet is a summary written to be clicked, not a source; the 3B recorded snippets in 5
+  of 7 tasks, and every model did it at least once.
+### Fixed (from the full suite on four models, and an untimed suite on the 7B)
+- **Commands are checked as exact text.** A command in a finding must be on the page shown. That covers text in
+  backticks, every line of a code block, and any path or placeholder (a backslash, `<venv>`, two slashes); prose
+  slashes such as "macOS/Linux" and web addresses are not counted. Word-by-word matching cannot see a wrong
+  command:
+  - The 7B recorded `pip install package_name`, which the page never gives. It scored 60% against the page, above
+    the two lines it really copied, because its words repeat the objective.
+  - Copying the official venv activation table, it swapped `bin` and `Scripts` (`C:\> <venv>\bin\activate.bat`)
+    in two runs.
+
+  Across every earlier suite run, 21 of 525 findings fail this check, all on the documentation task and all real.
+  The refusal quotes the page's own line for the same command. With that, the 7B recorded only the page's commands
+  in 3 of 6 tries, and repeated the refused one in the other 3, which is refused again; told only to leave the
+  command out, it kept the invented command and dropped the real ones 6 times out of 6.
+- **The final result's commands must come from a finding.** If one doesn't, its sentence (or code-block line) is cut
+  and the command is named after the result: "Left out, because no discovery gives them". In earlier runs this
+  caught 5 of 119 results, including the 14B's `<venv>/Scripts/activate` and a code block with `pip install
+  package_name`. The report warns about such commands, and the suite's honesty table counts them.
+- **Fewer findings recorded from the wrong place.** A finding is refused, with what to do instead, when:
+  - it is empty;
+  - it copies the page's headings or outline;
+  - it describes the page ("this guide covers…") instead of giving its facts;
+  - it only asks questions;
+  - it repeats the objective or the searches instead of the page;
+  - it repeats the checklist's own wording ("covered by [2]").
+- **Cut-off replies are rescued.** A reply cut off inside its parameter is kept for SEARCH, NAVIGATE and READ_MORE.
+  It is cut where the words start repeating, and a search is limited to 20 words. A cut-off finding on the results
+  page is refused as usual.
+- **Coverage check:**
+  - When the model cites only discovery numbers twice, it is asked a yes-or-no question about the cited
+    discovery alone, and a "yes" counts only with words copied from it. A "no" still counts when the discovery has
+    words that belong to that item alone: the 7B said "no" 3 times out of 3 for "the light-dependent reactions,
+    which take place in the thylakoids". The question accepts a partial answer ("answer the item, or part of
+    it"); asked whether the discovery "says what the item asks for", the 7B said "no" 6 times out of 6 for that
+    same partial answer.
+  - Things are counted only when a finding names them.
+  - The words that answered each item are shown to the final result, which had said "the discoveries do not
+    specify what goes into photosynthesis" over a discovery naming water, sunlight and carbon dioxide.
+- **Final result:**
+  - "Not specified …, but <something from the model>" is cut back to the gap.
+  - Items whose findings give no amounts are named, so none are supplied.
+  - A "My view" paragraph that repeats the result is dropped.
+- Refusal hints no longer quote a sentence where the page describes itself, or a question.
+- The 14B's model calls took longer than the client library's 10-minute default, which ended its runs. The timeout is
+  now 30 minutes.
+- A run stopped while its final result was being written says so in the report, and shows the unchecked draft; the
+  suite no longer counts it as finished.
+
+### Changed
+- The README says near the top which goals work and which can't yet: anything behind a form, a login or a button,
+  such as flight prices. An automated write-up of the project, built from the README, had suggested "the three
+  cheapest flights to Denver" as an example goal. The privacy line also says that searches, written from your
+  objective, go to DuckDuckGo and pages come from the web; the objective itself, the model and the report stay on your
+  computer.
+- Two real reports from the 1.62 suite run, unedited, in `docs/examples/`, with a note on what each got right and
+  wrong (a path slip in one; in the other, an opinion that leans on the one part it found nothing about). Linked
+  from the README.
+- `llms.txt`, a short plain-text summary for AI tools that read the repository (the convention described at
+  llmstxt.org), saying the same as the README: good goals, what it can't do yet, speed, privacy. Nothing in it is
+  hidden or addressed to a model as an instruction.
+- A project site (`docs/`, served by GitHub Pages at https://snowfxxx.github.io/LocalAgentBrowser/) with the
+  findings, a quick start and the challenge, plus two write-ups of this round: an article, "We Can Do Better: A
+  Small AI That Can't Cover Up", and a technical paper, "Verification over Trust". Every regression run so far is
+  attached to the release as a zip.
+- Corrected: the README and the regression history had counted the 14B's results that the time limit cut off while
+  they were being written. Of its five finishes, two count (2 of 7).
+
 ## [1.62] - 2026-10-25
 No change in what the agent does: the same prompts, decisions and results as 1.61.
 ### Changed

{localagentbrowser-1.62/localagentbrowser.egg-info → localagentbrowser-1.64}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: localagentbrowser
-Version: 1.62
+Version: 1.64
 Summary: An autonomous web agent driven by a local LLM (LM Studio) and Playwright. Runs entirely on your own machine.
 Author: SnowFX
 License-Expression: MIT
@@ -36,6 +36,7 @@ Dynamic: license-file
 decides when it's done, all on your own computer.**
 
 [![Tests](https://github.com/SnowFXXX/LocalAgentBrowser/actions/workflows/tests.yml/badge.svg)](https://github.com/SnowFXXX/LocalAgentBrowser/actions/workflows/tests.yml)
+[![PyPI](https://img.shields.io/pypi/v/localagentbrowser.svg)](https://pypi.org/project/localagentbrowser/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/SnowFXXX/LocalAgentBrowser/blob/main/LICENSE)
 ![Python 3.10–3.13](https://img.shields.io/badge/python-3.10%E2%80%933.13-blue.svg)
 ![Runs locally](https://img.shields.io/badge/LLM-local%20(LM%20Studio)-green.svg)
@@ -48,7 +49,9 @@ A local language model served by [LM Studio](https://lmstudio.ai) decides what t
 [Playwright](https://playwright.dev/python/) carries it out in a visible Chromium window you can watch. Built
 and tuned to work well with a **small 7B model on a CPU, no GPU required**.
 
-- 🔒 **No cloud AI and no API keys.** The model runs in your own LM Studio.
+- 🔒 **No cloud AI and no API keys.** The model runs in your own LM Studio. (Its searches, written from your
+  objective, go to DuckDuckGo, and it reads pages from the web like any browser; the objective itself, the model and
+  the report stay on your computer.)
 - 🧭 **Goes from objective to findings unattended.** It searches, opens pages, reads long pages part by part,
   records what's relevant and finishes when the objective is met.
 - 🛡️ **Keeps a small model on track.** Rules enforced in code stop loops, repeats and drifting, and the model
@@ -56,9 +59,37 @@ and tuned to work well with a **small 7B model on a CPU, no GPU required**.
 - ⚡ **Built for slow hardware.** It reuses LM Studio's prompt cache, so most turns process only what changed.
 - 📝 **A readable report every time.** `research_report.md` holds the final result and every finding with its
   source, written however the run ends; `research_log.txt` keeps every action and reason.
-- ✅ **Tested.** 165 tests on Linux and Windows, Python 3.10–3.13, run in a few seconds with no model or browser,
+- ✅ **Tested.** 324 tests on Linux and Windows, Python 3.10–3.13, run in a few seconds with no model or browser,
   plus a [regression suite](#regression-suite) of live runs on seven different kinds of task.
 
+**New with 1.64:** parts of the objective no longer drop out of the answer unannounced as often (the coverage check asks for the words first; missing amounts and terms send the agent back), measured on two PCs and then four models with identical code: Qwen2.5-7B 6 of 7, Llama 3.1 8B 7 of 7, Qwen2.5-14B 5 of 7 and Qwen2.5-3B 3 of 7 tasks, with no outside knowledge in any finished result ([regression history](docs/regression-history.md)). On Windows, a launcher with Start-menu and desktop shortcuts opens the report by itself when a run ends.
+
+**New with 1.63:** the [project site](https://snowfxxx.github.io/LocalAgentBrowser/), an article,
+[We Can Do Better: A Small AI That Can't Cover Up](https://snowfxxx.github.io/LocalAgentBrowser/article.html), and a
+technical paper, [Verification over Trust](https://snowfxxx.github.io/LocalAgentBrowser/paper.html).
+
+### What it's good for, and what it can't do yet
+
+- **Good goals:** questions you would research by reading pages. Examples:
+  - a fact ("How tall is Kilimanjaro, in meters and feet?")
+  - a list with conditions ("Three free, open-source alternatives to Microsoft Office, and what each is best at")
+  - a comparison with a recommendation
+  - a how-to
+  - an explanation of a topic
+  - views from several sources
+  - looking something up in official documentation
+
+  These are the seven kinds of task the [regression suite](#regression-suite) measures.
+- **Not yet:**
+  - **Anything behind a form, a login or a button.** It reads pages; it does not click, fill in forms or sign
+    in. So goals like "the three cheapest flights to Denver next month" won't work: flight and hotel prices,
+    shopping and booking sites need searches it cannot type into.
+  - **Fast answers.** On a CPU a run takes about 10 to 45 minutes, and a 7B model sometimes returns thin findings,
+    so check the result against its listed sources.
+
+See [two example reports](https://github.com/SnowFXXX/LocalAgentBrowser/blob/main/docs/examples/README.md), unedited,
+with what each got right and wrong.
+
 📖 **The story behind it:** [Teaching a 7B model on a CPU to finish what it starts](https://dev.to/snowfxxx/teaching-a-7b-model-on-a-cpu-to-finish-what-it-starts-1amc),
 six lessons from 30 versions of keeping a small local model on track.
 
@@ -107,6 +138,11 @@ six lessons from 30 versions of keeping a small local model on track.
    When it finishes, read `research_report.md` in the folder you ran it from. Press Ctrl+C at any time to
    stop; the report is still written with everything found so far.
 
+**On Windows**, from a clone of the repository (below), `powershell -ExecutionPolicy Bypass -File
+tools\windows\make_shortcuts.ps1` adds LocalAgentBrowser to the Start menu and the desktop. The launcher checks that
+LM Studio is answering, runs each objective in its own folder under `Documents\LocalAgentBrowser runs`, and opens the
+report as a page in your browser when the run ends.
+
 To work on the code instead, clone the repository and install from it:
 
 ```bash
@@ -203,15 +239,15 @@ turns up, so the suite compares versions on all seven at once. 1.54 to 1.56 and
 measured this way and released with 1.57 and 1.61; [docs/regression-history.md](https://github.com/SnowFXXX/LocalAgentBrowser/blob/main/docs/regression-history.md)
 has every step's results and what each one showed.
 
-| Task | 1.52 | 1.53 (objective checklist) | 1.57 (coverage check) | 1.61 (verified send-backs, outline pointer) |
-|---|---|---|---|---|
-| Factual question | Finished, 12 min | Finished, 14 min | Finished, 13 min | Finished, 17 min |
-| List of three office suites | Time limit | Finished, 30 min; but WPS Office is not open source | Time limit: three correct open-source suites by turn 7, sent back because no words showed them to be "free" | Finished, 21 min: LibreOffice, WPS Office and OnlyOffice, each marked "not shown in the pages read to be free and open-source" |
-| Comparison and recommendation | Time limit | Finished, 39 min | Time limit, after a send-back for "official documentation", which the objective does not ask for | Finished, 26 min, setup and administration from the part of a long page the outline pointer chose |
-| How-to (cold brew from 1.55; repotting before) | Time limit | Time limit | Finished, 36 min; but "how long it keeps" answered with the steeping time | Finished, 33 min: every step, the 16-hour steep and the shelf life, cited |
-| Explanation (photosynthesis) | Time limit | Finished, 13 min, but thin: no oxygen or water | Finished, 59 min: inputs, outputs and both stages, cited | Finished, 39 min |
-| Opinion (owls) | Time limit | Finished, 38 min | Time limit: its first page covered two of the three cultures asked for | Finished, 42 min: four cultures, cited, with a view |
-| Documentation (venv, official docs) | Finished, 15 min, from a tutorial site | Time limit (30 min) | Finished, 37 min, all from python.org; but the Windows line repeats the create command instead of the activate one | Time limit: all four items found from python.org, then it read on |
+| Task | 1.52 | 1.53 (objective checklist) | 1.57 (coverage check) | 1.61 (verified send-backs, outline pointer) | 1.64 (quote-first coverage, send-backs for missing amounts and terms) |
+|---|---|---|---|---|---|
+| Factual question | Finished, 12 min | Finished, 14 min | Finished, 13 min | Finished, 17 min | Finished, 17 min |
+| List of three office suites | Time limit | Finished, 30 min; but WPS Office is not open source | Time limit: three correct open-source suites by turn 7, sent back because no words showed them to be "free" | Finished, 21 min: LibreOffice, WPS Office and OnlyOffice, each marked "not shown in the pages read to be free and open-source" | Finished, 34 min: GIMP, Krita and Darktable (the list task asks for Photoshop alternatives from 1.64) |
+| Comparison and recommendation | Time limit | Finished, 39 min | Time limit, after a send-back for "official documentation", which the objective does not ask for | Finished, 26 min, setup and administration from the part of a long page the outline pointer chose | Finished, 49 min |
+| How-to (cold brew from 1.55; repotting before) | Time limit | Time limit | Finished, 36 min; but "how long it keeps" answered with the steeping time | Finished, 33 min: every step, the 16-hour steep and the shelf life, cited | Finished, 50 min |
+| Explanation (photosynthesis) | Time limit | Finished, 13 min, but thin: no oxygen or water | Finished, 59 min: inputs, outputs and both stages, cited | Finished, 39 min | Finished, 73 min, oxygen included |
+| Opinion (owls) | Time limit | Finished, 38 min | Time limit: its first page covered two of the three cultures asked for | Finished, 42 min: four cultures, cited, with a view | Finished, 49 min |
+| Documentation (venv, official docs) | Finished, 15 min, from a tutorial site | Time limit (30 min) | Finished, 37 min, all from python.org; but the Windows line repeats the create command instead of the activate one | Time limit: all four items found from python.org, then it read on | Time limit: read the reference page part after part without recording (finished in 34 and 42 min in the two v11 runs) |
 
 ## How it works
 
@@ -222,7 +258,7 @@ current page, and answers with exactly one JSON action:
 |---|---|
 | `SEARCH` | Searches DuckDuckGo (HTML version) and shows a numbered list of results with their real URLs |
 | `NAVIGATE` | Opens a URL taken from the results or from the current page's links |
-| `READ_MORE` | Shows the next part of a long page (pages are read in parts of about 2,500 characters) |
+| `READ_MORE` | Shows the next part of a long page (pages are read in parts of about 2,500 characters), or the unread part whose outline label names an item still missing |
 | `DISCOVER` | Records information from the current page as a finding |
 | `FINISH` | Ends the run (after checking each part of the objective is covered); the final result is then written in a separate step |
 
@@ -272,6 +308,11 @@ The code is in the `localagentbrowser` package, one file per job, each depending
   its own knowledge when a page says "follow these 6 steps" and the steps come later. The refusal points the way:
   it quotes the one page sentence closest to what the model wrote, to record instead, or names what the parts
   shown never mention ("Japan, Navajo"), so the model reads on.
+- **Commands must be on the page, exactly.** Commands and paths in a finding (in backticks, in a code block, or
+  any word with a backslash, a `<placeholder>` or two slashes) must appear on the page as written, however often
+  the finding is repeated. Word matching cannot catch a wrong command: a 7B recorded `pip install package_name`,
+  which the official page never gives, and swapped `bin` and `Scripts` when copying its activation table. The same
+  goes for the final result: a command no finding gives is cut and named after the result.
 - **An objective checklist.** At the start, one short model call splits the objective into a fixed checklist of
   what the result must contain, keeping conditions like "free, open-source" or "three". Each finding names the
   items it serves, the checklist with its findings is in every prompt, and every check goes through these items
@@ -283,8 +324,10 @@ The code is in the `localagentbrowser` package, one file per job, each depending
   its own, and the code checks the verdict instead of taking it on trust: an item nothing was copied for, fewer
   things than asked for ("only 1 of 3 found"), and things not shown, in words copied from a finding about them,
   to meet a condition such as "open-source". A claim that something is missing is dropped when every word of it is
-  already in the findings. With gaps, the FINISH is sent back once, with the next step named ("SEARCH for
-  'OnlyOffice free and open-source' to check it, or replace it"), and the next turn starts from a fresh prompt.
+  already in the findings. Only gaps the code verifies send the FINISH back, once: an item nothing was copied for,
+  fewer things than asked for ("only 1 of 3 found"), or nothing recorded. The model's other verdicts, and things
+  not shown to meet a condition, only shape the final result. The send-back names the next step, and the next turn
+  starts from a fresh prompt.
 - **Numbered findings.** Findings are numbered in the prompt and listed with every check, and the model names
   the finding that covers each part of the objective ("covered by [2]") instead of just saying "covered".
 - **Knowing when it's done.** After every new finding the model is asked whether the objective is now fully
@@ -299,9 +342,10 @@ The code is in the `localagentbrowser` package, one file per job, each depending
   cite findings by number, give as many items as asked for (never repeating one, and saying how many were not
   found), and add a "My view:" paragraph only when the objective asks for an opinion. A small model writes far
   better when that is its only job, and every claim stays traceable to a recorded finding. What no finding
-  answers is not left to the model: those checklist items, and things not shown to meet a condition, are left out
-  of what it is asked to write, and the code adds them after the result as "Not found in the pages read" and
-  "Left out, not shown to meet the conditions". The report flags any number in the final result that appears in
+  answers is not left to the model: those checklist items are left out of what it is asked to write, and the code
+  adds them after the result as "Not found in the pages read". A thing not shown to meet a condition stays in the
+  result, and the code marks it where it is first named ("not shown in the pages read to be open-source"), or lists
+  it after the result if the result does not name it. The report flags any number in the final result that appears in
   none of the findings, the most harmful kind of slip in a how-to.
 - **Error pages are recognised.** A page answering 404, 403 and so on counts as a failed visit, not content.
 - **Readable pages.** Navigation menus, footers, cookie banners and site notices such as fundraising banners are
@@ -356,13 +400,32 @@ It was developed and tested with Qwen2.5-7B-Instruct (Q4_K_M). Any instruction-t
 serves should work, as long as the server supports `response_format` with a JSON schema (LM Studio 0.3 and
 later). Set `LMSTUDIO_MODEL` and `LMSTUDIO_CONTEXT_LENGTH` to match. Reports with other models are very welcome.
 
+The full suite, run on four models with the same code, 90 minutes per task, CPU only (October 2026):
+
+| Model | Tasks finished | Model time | Gaps filled from the model's own knowledge |
+|---|---|---|---|
+| Qwen2.5-7B-Instruct | 6 of 7 | 197 min | 0 |
+| Qwen2.5-14B-Instruct | 2 of 7 | 535 min | 0 |
+| Llama-3.1-8B-Instruct | 2 of 7 | 278 min | 0 |
+| Qwen2.5-3B-Instruct | 0 of 7 | 302 min | — |
+
+Where they finished, every model's results were honest by the suite's measures. What separates them is speed and
+follow-through:
+- The 14B was about 2.5 times slower than the 7B on a CPU. It reached FINISH on five tasks, but on three of them
+  the time limit hit while the final result was being written, so only two count.
+- Llama found most items but rarely finished in time.
+- The 3B recorded findings on every task but never finished one.
+
+With no time limit, the 7B finished 7 of 7 (277 minutes). These measures are word checks: the suite counted a
+documentation result with an invented command as clean, which led to the command check above.
+
 **Can I use another OpenAI-compatible server instead of LM Studio?**
 Probably: point `LMSTUDIO_BASE_URL` at it. It has only been tested with LM Studio, and the speed tricks rely
 on the server reusing cached prompt prefixes (llama.cpp-based servers do).
 
 **Is anything sent to the internet?**
 No AI service is involved, and nothing about your run is uploaded. The agent does browse the real web,
-though: search queries go to DuckDuckGo, and pages it opens load normally, as if you visited them yourself.
+though: search queries, written from your objective, go to DuckDuckGo, and pages it opens load normally, as if you visited them yourself.
 
 **Where are the results?**
 In the folder you ran it from: `research_report.md` has the final result and every finding with its source,
@@ -400,7 +463,8 @@ turn that brings in a new page takes a few minutes. See [Performance](#performan
 - **Very strict objectives.** An objective that forbids finishing until everything is "fully validated" used
   to keep the agent looking for something the web may not contain. On 1.50 it finished such an objective in 25
   minutes, but with thin findings and without naming what it could not find, so check the result against the
-  objective.
+  objective. Since 1.57 the code lists checklist items no finding answers after the result; this has not been
+  tested on a strict objective yet.
 - **Check the citations.** The final result is told to use only recorded findings, but a 7B model sometimes
   puts the wrong number on a citation, adds a line no finding supports, leaves out a step or misreads a part of
   the question (on 1.57 it answered "how long it keeps in the fridge" with the steeping time), so verify a claim
@@ -418,7 +482,7 @@ turn that brings in a new page takes a few minutes. See [Performance](#performan
 - [x] Write the final result in a separate, rule-bound step and flag unsupported numbers (1.32–1.38)
 - [x] Finish as soon as the objective is covered (1.35)
 - [x] Get the model to weigh each part of a long page before reading on (1.42)
-- [ ] Let very strict objectives end with the gaps named instead of searching indefinitely (they finish since 1.50; naming the gaps is still open)
+- [ ] Let very strict objectives end with the gaps named instead of searching indefinitely (they finish since 1.50; since 1.57 the code names checklist items no finding answers, not yet tested on a strict objective)
 - [x] Measure every change on a regression suite of seven kinds of task (1.53)
 - [x] Keep a fixed checklist of the objective's parts instead of re-splitting it at every check (1.53)
 - [ ] Check that a finding really covers its checklist item, including conditions like "open source" (partly, since
@@ -430,8 +494,13 @@ turn that brings in a new page takes a few minutes. See [Performance](#performan
 - [ ] Rewind dead ends: save the conversation before an expensive step such as reading a long page, and if it
   leads nowhere, return to that point with a short note instead of carrying the unused page text in every later
   prompt (LM Studio reuses its cache up to the saved point)
+- [ ] Catch an objective too ambiguous to research before the run starts, and only then ask one question: never
+  during a run, and only for extreme ambiguity, so the run can still be left unattended but isn't spent chasing an
+  objective it can't pin down
+- [ ] A small project site on GitHub Pages: the README's essentials, the example reports and `llms.txt` at the
+  site's root, where AI tools look for it
 - [ ] Try more kinds of objectives (seven so far) and adjust the rules that don't generalise
-- [ ] Make fresh prompts (the slowest kind of turn) cheaper still (31% cheaper since 1.34)
+- [ ] Make fresh prompts (the slowest kind of turn) cheaper still (31% cheaper since 1.34; about 2,500 tokens since 1.61, from 3,500–4,900)
 - [ ] Interacting with pages (clicking, forms) for tasks beyond reading
 
 Ideas and requests are welcome in [issues](https://github.com/SnowFXXX/LocalAgentBrowser/issues).
@@ -466,6 +535,10 @@ To measure a change on all seven kinds of task (about four hours on a CPU), comp
 python tools/regression.py --baseline regression_runs/<earlier run>/summary.json
 ```
 
+To measure another model on the same tasks, name it as LM Studio lists it, for example
+`--model meta-llama-3.1-8b-instruct`. The model is loaded before the first task and its context length checked, so
+every run is measured with the same context length; the run's folder and summary name the model.
+
 Each run gets a folder in `regression_runs/` with every task's report, journal, console output and prompt
 trace, and a `summary.md` that is updated after each task: whether it finished, turns, model time, how many
 checklist items got findings, which checks fired, and problems found by simple word checks (missing answers,