@yibie/pi-jev-browser 0.1.0

package/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Cline Bot Inc.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

package/README.md

@@ -0,0 +1,137 @@
+# pi-jev-browser
+
+An isolated Playwright Chromium browser for pi, driven by [Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) — TypeSafe AI's System One model — called directly through the TypeSafe API, or by the model pi already has configured.
+
+> **Ported from Cline.** This package is a port of [`cline/plugins` → `plugins/jev-browser`](https://github.com/cline/plugins/tree/main/plugins/jev-browser) (v0.2.2, by Bee / Cline Bot Inc., Apache-2.0) to the pi extension API. The observation layer, decision loop, action executor, configuration, live stream, recording overlay, and browser setup are upstream code, largely unchanged. The host adapter, the direct TypeSafe transport in place of Vercel AI Gateway, and the `pi` decision policy are new. See [Porting notes](#porting-notes) for the full list.
+
+Jev does not see screenshots. The plugin hands it a structured DOM observation and one multiple-choice question per step, and Jev answers with a concrete operation plus a probability distribution over the offered options. That removes the screenshot round trip and the reasoning round trip from every browser step.
+
+**This is not a replacement for `agent_browser`.** It is the other trade: no login state, no extensions, no host environment, every step recorded with its probability, and a much cheaper fast loop. Use it for narrowly scoped goals on public pages. Use a profile-based browser tool when you need the user's session.
+
+## Decision policies
+
+Each step offers the same enumerated choices — every concrete action compared directly against scrolling, waiting, and stopping — and a policy answers which one to take. Only the answerer differs.
+
+| `policy` | Who decides | Needs | Trade-off |
+| --- | --- | --- | --- |
+| `pi` (default) | The model pi has configured | Nothing extra | `probability` is whatever the model claims, and completion discipline follows that model |
+| `typesafe` | [Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) via the TypeSafe API | `TYPESAFE_API_KEY` or `typesafe.apiKey` | Better at checking that a requirement is really visible before declaring done; every step is one TypeSafe request |
+
+`pi` is the default because it works with no second credential and no extra API quota. Its weakness is the mirror image: measured on one goal (open a category, then a detail page, stop when the UPC and availability are visible), both `deepseek-flash` and `deepseek-v4-pro` declared `DONE` after two clicks without ever scrolling to the Product Information table, where Jev scrolled twice and then stopped. Clarifying the `DONE` criterion did not change that. What kept the outcome honest was the evidence: `done_unverified` plus a viewport-scoped page text let the calling agent see that the UPC had never been read, and it said so instead of reporting success.
+
+On the same goal, policy `typesafe` completed it in four executed steps (7.1 s) with both required values in the returned page text, and five consecutive direct API calls showed no throttling at all — the Gateway free tier in the same position stopped after five or six requests.
+
+## Related work
+
+[`pi-jev-browser`](https://github.com/laihenyi/pi-Jev-browser) (npm `pi-jev-browser`) is a sibling port of the same upstream plugin, and it is the more capable of the two: eight tools, including a deterministic extractor and a macOS accessibility-tree driver behind a surface-agnostic loop, plus stuck detection and a 22-scenario benchmark across local, model, live, and desktop tiers. It requires a TypeSafe API key to run at all.
+
+This package differs in two ways worth choosing it for: policy `pi` runs with no external credential, and the measurement above compares the decision layers instead of assuming one. If you want the broader tool surface, use the sibling.
+
+## Install
+
+```bash
+pi install /absolute/path/to/pi-jev-browser        # local checkout
+pi install git:github.com/yibie/pi-jev-browser     # from git
+pi install npm:@yibie/pi-jev-browser               # from npm
+```
+
+Installing pulls in `playwright`, whose own install step downloads Chromium — roughly 150 MB, so the first install takes a moment. `ensureChromium()` covers the case where that step was skipped: the first `jev_run` runs `playwright install chromium` if no matching build is cached, bounded to two minutes. Nothing is downloaded at pi startup. On Linux the system browser libraries remain an administrator-managed prerequisite; this package never runs sudo.
+
+## Configuration
+
+Optional. Without a config file the plugin allows all HTTP and HTTPS origins, runs headless at 1280×720, records WebM video, and writes artifacts to `~/.pi/agent/data/jev-browser/`.
+
+```bash
+cp pi-jev-browser.config.example.json ~/.pi/agent/pi-jev-browser.config.json
+```
+
+| Key | Default | Notes |
+| --- | --- | --- |
+| `policy` | `"pi"` | `pi` uses the model pi has configured; `typesafe` calls the TypeSafe API directly. See [Decision policies](#decision-policies). |
+| `allowedOrigins` | `["http://*", "https://*"]` | `*` wildcards, matched against the origin. Narrow this for sensitive work. |
+| `headless` | `true` | On macOS a rejected headless launch falls back to a visible window. |
+| `recordVideo` | `true` | Finalized by `jev_stop`. |
+| `showCursor`, `showClickIndicators` | `true` | Overlay for screenshots, stream, and recordings. |
+| `viewport` | `1280×720` | Clamped to 640–2560 × 480–1600. |
+| `outputDir` | `~/.pi/agent/data/jev-browser` | One directory per browser session. |
+| `stream` | `{enabled:false, intervalMs:1000}` | `jev_stream` can start it on demand. |
+| `typesafe.apiKey` | — | Used when `TYPESAFE_API_KEY` is not set. |
+| `typesafe.model` | `jev-latest` | TypeSafe model alias for the decision step. |
+
+Credentials resolve in this order: `TYPESAFE_API_KEY`, then `typesafe.apiKey`; `TYPESAFE_MODEL`, then `typesafe.model`. `PI_JEV_BROWSER_CONFIG` overrides the config path. Credentials are read on every run, never written into the browser's environment, and never returned in tool results. Both settings apply to policy `typesafe` only; policy `pi` resolves its model through pi's own provider configuration. Field values are always filled by pi's configured model, because Jev generates no text.
+
+With policy `typesafe`, **every step costs one request**, so a 20-step run makes up to 20 of them. TypeSafe documents `429 Too Many Requests` and `529 Overloaded` as back-off-and-retry. The loop does not retry: retrying inside the loop would spend the step budget on requests that keep failing. Those responses end the run as `interrupted` with failure category `rate_limited` or `overloaded`, take no action for the step being decided, and tell the caller to wait.
+
+## Tools
+
+| Tool | Purpose |
+| --- | --- |
+| `jev_run` | Start or reuse the browser, capture before/after screenshots, and run the Jev loop toward one goal. |
+| `jev_actions` | Manual click/type/scroll/drag batch. **Does not call Jev.** Escape hatch only. |
+| `jev_state` | URLs, titles, tabs, viewport, start time. |
+| `jev_logs` | Console messages, page errors, failed requests, navigations, blocked downloads, security blocks. |
+| `jev_stream` | Tokenized live screenshot + log viewer bound to `127.0.0.1`. |
+| `jev_stop` | Cancel any in-flight run, close the browser, finalize video. |
+
+A run is bounded to 20 steps by default (60 max) and 100 seconds. `jev_run` returns a status, the executed step count, `elapsedMs`, a JSONL trace path, screenshots on disk, and text evidence of the page the run stopped on: URL, title, an excerpt of the visible text, and the number of actionable targets observed.
+
+**Verification is text-first.** The final screenshot is attached as an image content block only when the active model declares image input. pi also strips images when `images.blockImages` is set, and extensions cannot read that setting — so the result always carries text evidence, and the verification line states which path applies, names the `Image reading is disabled` symptom, and forbids reporting success from the status alone.
+
+Statuses are `done_unverified`, `blocked`, `needs_review`, `uncertain`, `step_limit`, `evaluation_limit`, and `interrupted`. There is deliberately no `done`: `done_unverified` means Jev believes the goal is complete and the calling agent must verify independently before reporting success.
+
+## Safety
+
+Three layers exist, and only the first one is enforcement:
+
+1. **Mechanical.** Navigation allowlist enforced in the request router; downloads refused and logged; service workers blocked; extensions and file-system access disabled; the browser process gets an empty environment; password, file, and hidden inputs are never observed; the model can only select from server-generated target IDs, so its output can never become a selector, URL, or code.
+2. **Model guidance.** Jev is instructed to return `REVIEW` before messages, posts, orders, payments, bookings, deletions, permission changes, sensitive-data entry, CAPTCHAs, or security warnings. This is guidance, not a deterministic boundary.
+3. **Agent instructions.** The tool guidelines tell pi to treat page content as untrusted, to ask before consequential actions, to never type secrets, and to verify `done_unverified` independently. These are instructions to another model, not guarantees.
+
+Page text, visible field values, and the goal are sent to the decision model — the TypeSafe API under policy `typesafe`, your configured provider under policy `pi` — and field values are sent to pi's model to be filled. Password and file fields are excluded, but other sensitive content is **not** automatically redacted. Delegate only narrowly scoped tasks.
+
+## Scope and limits
+
+- **Not a vision agent.** No screenshots are sent to Jev. Screenshots exist for the calling agent's verification and for the user.
+- **DOM coverage.** Frames, shadow DOM, canvas controls, nested scrolling, uploads, and arbitrary keyboard widgets are outside the observation loop. Use `jev_actions` there.
+- **Observation caps.** 200 action targets, 6,000 visible characters, 50 selected options, and up to 50 offscreen control labels per direction. Dense pages can lose controls.
+- **No persistent state.** Every browser start creates a fresh context: no cookies, no logins, no profiles.
+- **Not desktop control.** This is a browser harness. Full desktop control would need a VM/container backend and an OS input adapter.
+- **Unbenchmarked.** End-to-end speed and live-model reliability have not been measured.
+- **Memory is narrow.** The last ten actions are retained in memory per goal within one browser session, and are dropped when the goal changes. Nothing is persisted.
+
+The loop retries only reads invalidated by a document replacement, up to five times. A browser mutation is **never** retried: a failed run may still have applied an action, so inspect the page before continuing. Three non-wait actions without observable progress end the run as `blocked`.
+
+## Porting notes
+
+Ported from [`cline/plugins` → `plugins/jev-browser`](https://github.com/cline/plugins/tree/main/plugins/jev-browser) (v0.2.2). The observation layer, decision loop, action executor, config, stream, overlay, and browser setup are the upstream code, unchanged apart from names. What the host boundary required:
+
+- **Cancellation.** Cline passes tool context over JSON IPC, so the upstream plugin could not receive a live `AbortSignal` and managed cancellation itself — Escape did not stop a run. Pi passes a real `signal` into `execute()`, so host cancellation now works and `jev_stop` is cleanup rather than the only stop button.
+- **Screenshots.** Upstream returned a host-specific result array; here the final image becomes a Pi `{type:"image"}` content block, so verification no longer depends on the client rendering an artifact path.
+- **One browser, not a map.** Upstream keyed sessions by Cline session id. A pi extension instance is one session, so the manager holds one browser and `session_shutdown` closes it.
+- **Rules → guidelines.** The upstream global safety rule became per-tool `promptGuidelines`, each naming its tool, plus `executionMode: "sequential"` on the tools that drive the shared page.
+- **No dashboard events.** Upstream emitted seven `jev_browser_update` events for the Cline UI. Pi has no equivalent surface; artifacts, the trace, and tool results carry the same information.
+- **Dependencies.** `zod` was unused and was dropped. Upstream reached Jev through Vercel AI Gateway with `ai` and `@ai-sdk/gateway`; both are gone, because TypeSafe's own API takes the same `state` + `questions` body the loop already builds. The validation those packages provided moved into `parseChoiceAnswer`, narrowed to what the loop actually needs: only an answer that names no offered option is fatal, because a doubtful probability distribution must never kill a run that has already clicked things. Its own value is reported when it is a usable number and marked unknown otherwise.
+- **Throttling is named, and failures explain themselves.** Upstream reported any evaluation failure as an unexplained interruption. HTTP 429 and 529 now carry their own failure categories, `rate_limited` and `overloaded`, and every failure records a bounded single-line `detail`. That last part is not cosmetic: an unexplained four-step failure is what prompted this change, and the detail line is what makes the next one diagnosable.
+- **Input validation.** `jev_actions` now validates every action in the batch before executing any of it, so a malformed action can no longer leave earlier actions half-applied.
+- **Pluggable decision policy.** The decision step became the `JevPolicy` seam the upstream interface hinted at: `policy: "pi"` answers it with the model pi already has configured, so the loop needs no second credential, while `policy: "typesafe"` calls the TypeSafe API directly. Vercel AI Gateway is no longer a dependency of any kind.
+- **Verification without vision.** Upstream handed back screenshots, so a text-only model could not check a `done_unverified` claim at all. Runs now also return the stopped page's URL, title, and visible-text excerpt, produced by the same observation layer, and the image is attached only when the model declares image input. Measured on one goal: the payload dropped from 1.38 MB to 164 KB with a text-only model, while the agent's verification went from "claim is unverified" to naming the book title and price.
+
+## Development
+
+```bash
+bun install
+bun run check   # tsc --noEmit
+bun run test    # node --test; browser tests need Chromium and a display
+```
+
+Tests use local HTML and mocked model responses, including the TypeSafe transport against a fake HTTP response. They make no paid model calls.
+
+To exercise the loop for real, run it: the default `pi` policy needs no key at all, and `policy: "typesafe"` with `TYPESAFE_API_KEY` set uses Jev.
+
+```bash
+pi -e ./extensions/jev-browser.ts -p "Use jev_run with url https://books.toscrape.com and goal: open the Travel category and stop when the first book's title is visible."
+```
+
+## License
+
+Apache-2.0. Upstream `cline/plugins` is Apache-2.0 (its plugin `package.json` says MIT, but the repository ships no separate plugin license, so the repository license is followed here). Upstream author: Bee, Cline Bot Inc.