@mirsella/browser-mcp 2.1.0 → 2.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "browser",
3
- "version": "2.1.0",
3
+ "version": "2.1.2",
4
4
  "description": "Shared Chromium profile with isolated conversation tabs for Codex and OpenCode.",
5
5
  "author": {
6
6
  "name": "mirsella"
package/CONTRIBUTING.md CHANGED
@@ -93,29 +93,71 @@ checkout that fails its first valid submission. No money or external account
93
93
  is involved. `bun test tests/model-eval.test.ts` checks the fixture's rejection
94
94
  and retry behavior.
95
95
 
96
- Use a fresh `.scratch/` directory per experiment. Build the baseline from a
97
- recorded commit in a separate source directory and copy the two release
98
- binaries to `<root>/bin/before` and `<root>/bin/after`. Keep each build's own
99
- browser skill. Start `bun scripts/browser-model-eval.ts site --root=<root>`
100
- under a known PID or held lock; stop and collect it when the experiment ends.
101
-
102
- Generate temporary project MCP configuration with `config --root=<root>
103
- --condition=before` (then `after`) and merge it without overwriting existing
104
- configuration. Generate each prompt with `prompt --run=before-1
105
- --skill=<baseline-skill-path>`, repeating for `before-2`, `after-1`, and
106
- `after-2`. Use separate model conversations with the same provider, model and
107
- effort. Run both baseline cases before switching configuration. The adapter
108
- preserves trusted OpenCode session metadata; the model never supplies routing.
96
+ Use a fresh `.scratch/` directory per experiment. Record the source commit,
97
+ binary checksum, browser version, provider, exact model, and reasoning
98
+ setting. For source comparisons, give each checkout its own Cargo build
99
+ directory and reuse the shared Kache cache. Keep each condition's browser
100
+ skill.
101
+
102
+ Start `bun scripts/browser-model-eval.ts site --root=<root> --port=0` under a
103
+ known PID or held lock. Its `site.json` records the listening URL and PID. The
104
+ browser origin exposes no scoring endpoint. The host scorer reads order state
105
+ from files in the experiment directory; agents must not read those files.
106
+
107
+ Run names have a condition label and seed, such as `ours-current-1`,
108
+ `ours-guidance-2`, or `browser-use-1`. Seeds 1 and 2 select paired fixture
109
+ cases. Generate temporary project configuration with `config --root=<root>
110
+ --labels=ours-current,ours-guidance --condition=ours-guidance --seed=1
111
+ --ours-current=<binary> --ours-guidance=<binary>`. Merge it without
112
+ overwriting existing configuration. Each run gets its own profile and an
113
+ allocated CDP port. Restore the configuration and collect the fixture process
114
+ when finished.
115
+
116
+ For browser-use 1.0.0, install `browser-use==1.0.0` in a task-local Python
117
+ virtual environment. Keep the interpreter's virtual-environment path; do not
118
+ resolve its symlink. Write a JSON argv array containing that Python
119
+ executable, `scripts/browser-use-eval.py`, `--profile`, a fresh profile path,
120
+ `--ipc`, and a short task-local IPC path. Pass its path as
121
+ `--command=<argv.json> --backend=browser-use` to `config` or `mcp`.
122
+ External-command configurations require `--seed=1` or `--seed=2` to avoid
123
+ sharing one profile between active trials. The wrapper runs the official MCP
124
+ implementation with local headless Chromium and explicitly stops its browser.
125
+ Use a separate command file and profile for each seed. This dependency is for
126
+ evaluation only and is not part of the published server.
127
+
128
+ Generate each task with `prompt --root=<root> --run=ours-guidance-1
129
+ --port=<fixture-port> --skill=<condition-skill>`. Omit `--skill` for a server
130
+ that supplies its own packaged instructions. Page JavaScript, DOM operations
131
+ and normal page requests through the assigned browser MCP are permitted for
132
+ every condition. Host file/network access, fixture implementation (including
133
+ inline script contents), scoring data and other tools are forbidden. A
134
+ code-based server may use host code to orchestrate its browser helpers and
135
+ process observed page data, but cannot use it as a shortcut to the fixture's
136
+ private state.
137
+
138
+ Use separate model conversations with the same provider, model, reasoning,
139
+ harness and budgets. Counterbalance condition order across seeds. Compare the
140
+ current and candidate guidance on the same server binary before attributing
141
+ any difference to a runtime change. Test multi-harness routing separately from
142
+ single-agent task quality.
109
143
 
110
144
  Save final answers to `<root>/answers.json` as objects containing `run`,
111
- `answer`, `elapsedSeconds`, and `thread`, then run `score --root=<root>`.
112
- Scoring checks one correct order and all reported facts and confirmation data.
113
- Traces retain tool calls, errors, text bytes, structured bytes, and total MCP
114
- response bytes. These byte counts are not tokenizer counts or provider billing.
115
- Inspect the agent traces for forbidden tools or source access before accepting
116
- a result. Exclude setup failures explicitly. Restore project configuration and
117
- stop only the experiment's adapters, workers and browsers afterward. Two cases
118
- per version are a smoke test, not a statistical reliability estimate.
145
+ `answer`, `elapsedSeconds`, `thread`, optional provider `usage`, and `valid`.
146
+ Set `valid: true` only after inspecting the agent trace for forbidden tools,
147
+ source or oracle access. Run `score --root=<root>
148
+ --runs=ours-current-1,ours-guidance-1,browser-use-1`. A passing score requires
149
+ that audit, a complete trace of issued tool calls within the 50-call and
150
+ 10-minute budgets, one correct order, all facts, and matching receipt data.
151
+ Setup failures and unaudited trials do not establish reliability.
152
+
153
+ The adapter records run identities independently of tool names and navigation,
154
+ so Python `browser_exec` calls and compact `dispatch` calls use the same
155
+ scorer. Reports include tool responses, client request bytes, initialization
156
+ and catalog response bytes, images, errors and supplied provider usage. Byte
157
+ counts are not tokens or billing. Include task/skill inputs and provider
158
+ token/cache/reasoning usage when available; do not infer total context or cost
159
+ from response bytes. Two paired cases are a smoke test, not a statistical
160
+ reliability estimate.
119
161
 
120
162
  ## Releases
121
163
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mirsella/browser-mcp",
3
- "version": "2.1.0",
3
+ "version": "2.1.2",
4
4
  "description": "Session-isolated browser automation over MCP and Chrome DevTools Protocol",
5
5
  "type": "module",
6
6
  "exports": {
Binary file