visus-web 0.2.0__tar.gz → 0.3.2__tar.gz

{visus_web-0.2.0 → visus_web-0.3.2/.claude}/skills/developing-visus-web/SKILL.md

@@ -58,17 +58,66 @@ The clean-room `bundle.js` (`window.__visus`) is OUR code (NOT Playwright). It c
 3. Real-browser smoke test under that engine if available in CI.
 
 ### Add an MCP tool
-`src/visus/web/mcp/server.py`: a `@mcp.tool()` thin wrapper over the public API, targeting via the shared `locator(page, role=, name=, text=, selector=, frame=)` helper. Add it to the registration assertion in `tests/test_mcp_server.py` and drive it in `tests/test_e2e_mcp.py`.
+`src/visus/web/mcp/server.py`: a `@mcp.tool()` thin wrapper over the public API, targeting via the shared `make_locator(page, *, selector=, role=, name=, text=, exact=, frame=)` helper (`mcp/session.py:142-160`; **keyword-only** after `page`). Add the name to `_EXPECTED_TOOLS` in `tests/test_mcp_server.py` and drive the tool in `tests/test_e2e_mcp.py`.
 
 ### Add a CLI command
-`src/visus/web/cli/main.py`: an `@app.command()` Typer function over the public API. Test with `typer.testing.CliRunner` (real browser for browser-touching commands).
+
+There are two kinds of command, and only the first is a one-file change.
+
+**Daemon-free command** — 11 of the 54 registered commands: `version`, `translate`,
+`doctor`, `install`, `install-skills`, `screenshot`, `pdf`, `open`, `run`, `mcp`,
+`codegen`. These launch (at most) their own throwaway browser and never talk to the
+session daemon. For these, `src/visus/web/cli/main.py` really is the whole change:
+an `@app.command()` Typer function over the public API. Test with
+`typer.testing.CliRunner` (real browser for the browser-touching ones).
+
+**Session command** — the other 43. It drives the running daemon (started by
+`visus session start`) instead of launching its own browser. Following only the
+one-file recipe above for one of these ships a command that always fails at
+runtime with `error: ValueError: unknown op '<name>'`, because nothing ever
+answers the dispatch in step 1. Places to touch, in the order you'll hit them:
+
+1. **`src/visus/web/cli/session_server.py`** — a `_op_<name>(self, args)` method.
+   `SessionHandler.handle()` dispatches with `getattr(self, f"_op_{op}", None)`
+   and raises `unknown op '<name>'` if it's missing — this is the one contributors
+   skip, and it's the only one that fails silently at authoring time (ruff/mypy
+   are clean; it only breaks when someone runs the command). Required.
+2. **`src/visus/web/cli/main.py`** — an `@app.command()` that sends the op via
+   `_send("<name>", args)`, never talking to the daemon directly. Required (this
+   is the one place the old recipe named).
+3. **`src/visus/web/skills/visus-web/references/cli-commands.md`** — a row in the
+   matching table. `tests/test_skill_docs.py::test_every_cli_command_is_documented`
+   walks `app.registered_commands` and fails, listing anything undocumented here.
+   Required to pass the build.
+4. **`README.md`**'s CLI command tables — a row in the matching table. Not
+   test-gated (no test reads `README.md`), but it's the first thing a human
+   reads; keep it in sync by hand.
+5. **`src/visus/web/cli/console.py`** — a `dispatch()` branch plus a line in
+   `_HELP`, so the interactive REPL can reach it too. Nice-to-have, not
+   required: the REPL is a convenience wrapper over the same `_send`, and it is
+   currently *incomplete* on purpose/by-omission — `drag`, `upload`,
+   `add-cookies`, `clear-cookies`, `read-text`, `solve-captcha`, and
+   `find-image` all ship as working CLI commands with no `console.py` branch at
+   all. Add one when it's cheap; don't treat its absence as broken.
+
+Test the `_op_*` handler with a real-browser test, and the CLI command with
+`typer.testing.CliRunner` asserting the exact op name and payload sent to the
+daemon (pattern: `tests/test_cli_session.py:1152-1642`).
 
 ## Test conventions (enforced)
 
 - **Real browser, no fakes.** Coverage is driven by real-browser integration tests (`@pytest.mark.browser`) against local fixture pages in `tests/fixtures/`, served by the session-scoped `base_url` fixture (an ephemeral `http.server`). Pure-logic units (errors/enum/config/exception-translation/step-encoding/vision-on-generated-images) are the only non-browser tests. Test doubles isolate a unit ONLY when its real behavior is also covered by an integration test — never as the sole proof of a feature. No `skip`/`xfail`/mock-to-pass.
-- **Parallel:** the suite runs under `pytest-xdist` (`-n auto`, `--dist loadfile`). Run a subset with `--no-cov` (the `--cov-fail-under=90` gate only makes sense on the full run).
-- **Gates:** `uv run ruff check src tests`, `uv run mypy` (`--strict`, no weakening — cast `execute_script`/CDP `Any` returns; selenium types only inside backend files), `uv run pytest -q` (≥90% coverage on `visus.web`).
+- **Parallel:** the suite runs under `pytest-xdist` (`-n auto`, `--dist loadfile`). The 90% gate lives in `[tool.coverage.report] fail_under`, not `addopts`, so it is no longer implicit — pass `--cov-fail-under=90` to enforce it, or `--cov-fail-under=0` on a filtered subset (e.g. `-m "not browser"`) to keep coverage measured without failing on a partial run; `--no-cov` drops measurement entirely.
+- **Gates:** `uv run ruff check src tests`, `uv run mypy` (`--strict`, no weakening — cast `execute_script`/CDP `Any` returns; selenium types only inside backend files), `uv run pytest -q --cov-fail-under=90` (≥90% coverage on `visus.web`).
 - **Commits:** small, TDD (failing test → minimal impl → green → commit).
 
+### Change the shipped agent skill
+
+The skill lives at `src/visus/web/skills/visus-web/` and ships inside the wheel.
+Its frontmatter `version` must equal `visus.web.__version__` — bump both together.
+`tests/test_skill_docs.py` gates command coverage, the MCP tool count, and link
+integrity. The contributor guide you are reading is **not** shipped; it lives in
+`.claude/skills/developing-visus-web/`.
+
 ## Layout & build
-`src/`-layout namespace package (`visus.web`; no `src/visus/__init__.py`). Build: `uv` + `hatchling`. Optional extras: `[dev]`, `[vision]` (rapidocr/opencv/pillow/numpy), `[mcp]` (mcp), `[cli]` (typer). Entry points: `visus`, `visus-web-mcp`.
+`src/`-layout namespace package (`visus.web`; no `src/visus/__init__.py`). Build: `uv` + `hatchling`. Optional extras: `[dev]`, `[vision]` (rapidocr/opencv/pillow/numpy), `[mcp]` (mcp), `[cli]` (typer + rich + click). Entry points: `visus`, `visus-web-mcp`.

visus_web-0.3.2/.github/dependabot.yml

@@ -0,0 +1,32 @@
+version: 2
+
+updates:
+  # Python dependencies are declared in pyproject.toml (PEP 621 + optional-dependencies).
+  # NOTE: `selenium>=4.38,<5` is a range, so Dependabot will only open a PR when a
+  # release falls OUTSIDE it. Rolling Selenium within the range is a manual step —
+  # see RELEASING.md, "Rolling Selenium".
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+      day: "monday"
+    open-pull-requests-limit: 5
+    groups:
+      dev-tooling:
+        patterns:
+          - "pytest*"
+          - "ruff"
+          - "mypy"
+          - "pre-commit"
+          - "build"
+          - "hatchling"
+          - "tomli"
+          - "pyyaml"
+
+  # Pinned action majors in ci.yml and publish.yml.
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+      day: "monday"
+    open-pull-requests-limit: 5

visus_web-0.3.2/.github/workflows/ci.yml

@@ -0,0 +1,171 @@
+name: CI
+
+# Runs the real gates on every push and PR, and is callable from publish.yml so a
+# release cannot be built before these jobs go green.
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  workflow_call: {}
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  lint:
+    name: Lint + types
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Install (mirrors TESTING.md)
+        shell: bash
+        run: |
+          uv venv --python "3.12"
+          uv pip install -e ".[dev,vision,mcp,cli]"
+      - name: Put the venv on PATH
+        shell: bash
+        run: |
+          if [ -d .venv/bin ]; then
+            echo "$PWD/.venv/bin" >> "$GITHUB_PATH"
+          else
+            # Windows: $PWD is a bash path (/d/a/...) that the pwsh steps cannot
+            # resolve -- `visus`/`pytest` were "not recognized"
+            cygpath -w "$PWD/.venv/Scripts" >> "$GITHUB_PATH"
+          fi
+      - name: ruff
+        run: ruff check src tests scripts
+      - name: mypy --strict
+        run: mypy
+
+  fast-tests:
+    name: Tests (no browser) ${{ matrix.os }} py${{ matrix.python }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+        python: ["3.10", "3.12"]
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Install (mirrors TESTING.md)
+        shell: bash
+        run: |
+          uv venv --python "${{ matrix.python }}"
+          uv pip install -e ".[dev,vision,mcp,cli]"
+      - name: Put the venv on PATH
+        shell: bash
+        run: |
+          if [ -d .venv/bin ]; then
+            echo "$PWD/.venv/bin" >> "$GITHUB_PATH"
+          else
+            # Windows: $PWD is a bash path (/d/a/...) that the pwsh steps cannot
+            # resolve -- `visus`/`pytest` were "not recognized"
+            cygpath -w "$PWD/.venv/Scripts" >> "$GITHUB_PATH"
+          fi
+      - name: pytest (no browser, no packaging)
+        run: pytest -q -m "not browser and not packaging" --cov-fail-under=0
+
+  packaging-tests:
+    name: Installed console scripts (${{ matrix.os }})
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Install (mirrors TESTING.md)
+        shell: bash
+        run: |
+          uv venv --python "3.12"
+          uv pip install -e ".[dev,vision,mcp,cli]"
+      - name: Put the venv on PATH
+        shell: bash
+        run: |
+          if [ -d .venv/bin ]; then
+            echo "$PWD/.venv/bin" >> "$GITHUB_PATH"
+          else
+            # Windows: $PWD is a bash path (/d/a/...) that the pwsh steps cannot
+            # resolve -- `visus`/`pytest` were "not recognized"
+            cygpath -w "$PWD/.venv/Scripts" >> "$GITHUB_PATH"
+          fi
+      - name: pytest -m packaging
+        run: pytest -q -m packaging --cov-fail-under=0
+
+  browser-tests:
+    name: Full suite + coverage gates (windows-latest)
+    runs-on: windows-latest
+    timeout-minutes: 75
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Install (mirrors TESTING.md)
+        shell: bash
+        run: |
+          uv venv --python "3.12"
+          uv pip install -e ".[dev,vision,mcp,cli]"
+      - name: Put the venv on PATH
+        shell: bash
+        run: |
+          if [ -d .venv/bin ]; then
+            echo "$PWD/.venv/bin" >> "$GITHUB_PATH"
+          else
+            # Windows: $PWD is a bash path (/d/a/...) that the pwsh steps cannot
+            # resolve -- `visus`/`pytest` were "not recognized"
+            cygpath -w "$PWD/.venv/Scripts" >> "$GITHUB_PATH"
+          fi
+      # Chrome ships with the runner image; Selenium Manager resolves chromedriver
+      # on first launch (backends/browsers/chrome.py). Fail loudly here rather than
+      # 298 times below.
+      - name: Smoke-test browser + driver
+        run: visus doctor
+      # -n 4 (not the addopts default -n auto): a measured full-suite run on a
+      # many-core dev machine showed -n auto spawning one xdist worker per core,
+      # each loading its own ONNX model for the vision tests (test_e2e_vision.py
+      # is `browser`-marked and runs here), and workers dying with
+      # `_ArrayMemoryError`. -n 4 was the exact configuration measured safe:
+      # 935 passed, 7 skipped, 0 failed in 14m52s. GitHub-hosted windows-latest
+      # runners are not guaranteed to stay small-core forever, so this is pinned
+      # explicitly rather than left to whatever `os.cpu_count()` reports.
+      - name: pytest (full suite, real browsers) + global coverage gate
+        run: pytest -q -n 4 -m "not packaging" --cov-fail-under=90 --cov-report=json
+      - name: Per-package coverage floors
+        run: python scripts/check_coverage.py
+      - name: Upload coverage.json
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-json
+          path: coverage.json
+          if-no-files-found: warn

visus_web-0.3.2/.github/workflows/publish.yml

@@ -0,0 +1,109 @@
+name: Publish to PyPI
+
+# Publishes to PyPI when you cut a GitHub Release (tag vX.Y.Z).
+#
+# A plain `git push` does NOT publish. Neither does `workflow_dispatch`: EVERY job
+# below is guarded, so a manual run from the Actions tab — on any branch — builds
+# nothing and publishes nothing. The dispatch trigger is kept only so the guard
+# stays visible and testable.
+#
+# The guard has two halves:
+#   github.event_name == 'release'                     -> not a dispatch
+#   github.event.release.target_commitish == 'main'    -> cut from main
+#
+# It is `target_commitish`, NOT `github.ref`: on a release event GITHUB_REF is the
+# TAG ref (refs/tags/vX.Y.Z), so `github.ref == 'refs/heads/main'` is never true
+# and would silently block every release. `target_commitish` carries the branch
+# the release was created from.
+#
+# Caveat: if a Release is cut from a pre-existing tag rather than a branch, GitHub
+# sets target_commitish to a commit SHA and this guard rejects it. Cut releases
+# from the `main` branch, as RELEASING.md describes.
+#
+# Nothing is built until the full CI matrix (lint, types, unit, packaging and the
+# real-browser suite with both coverage gates) is green on the released commit.
+#
+# Auth is via PyPI Trusted Publishing (OIDC): no tokens, no secrets.
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch: # deliberately inert — see the guard on every job
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    name: CI gates
+    if: github.event_name == 'release' &&
+        github.event.release.target_commitish == 'main'
+    uses: ./.github/workflows/ci.yml
+
+  build:
+    name: Build distribution
+    if: github.event_name == 'release' &&
+        github.event.release.target_commitish == 'main'
+    needs: [test]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Build sdist + wheel and validate metadata
+        run: |
+          python -m pip install --upgrade pip build twine
+          python -m build
+          twine check dist/*
+
+      # `twine check` validates metadata and never opens the wheel's payload.
+      # The agent skill and the Selenium JS bundle both live inside the package
+      # directory, so a packaging regression would silently ship a CLI that
+      # cannot install its own skill or drive a browser.
+      - name: Assert the wheel ships the agent skill and the runtime data files
+        run: |
+          python - "$(ls dist/*.whl)" <<'PY'
+          import sys, zipfile
+          wheel = sys.argv[1]
+          names = set(zipfile.ZipFile(wheel).namelist())
+          required = [
+              "visus/web/skills/visus-web/SKILL.md",
+              "visus/web/backends/selenium/js/bundle.js",
+              "visus/web/py.typed",
+          ]
+          print(f"{wheel}: {len(names)} entries")
+          for item in required:
+              print(("  OK   " if item in names else "  MISS ") + item)
+          missing = [item for item in required if item not in names]
+          if missing:
+              sys.exit(f"wheel is missing required files: {missing}")
+          PY
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    name: Publish to PyPI
+    if: github.event_name == 'release' &&
+        github.event.release.target_commitish == 'main'
+    needs: [build]
+    runs-on: ubuntu-latest
+    # The environment name MUST match what you set on PyPI's trusted publisher.
+    environment:
+      name: pypi
+      url: https://pypi.org/p/visus-web
+    permissions:
+      id-token: write # required for OIDC / trusted publishing
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

{visus_web-0.2.0 → visus_web-0.3.2}/.gitignore

@@ -4,7 +4,10 @@
 /botcity-framework-web-python/
 
 # ---- Local-only planning docs (specs/plans kept on disk, NOT tracked) ----
-/docs/
+# `/docs/*` (not `/docs/`) so a single tracked page can be re-included below —
+# git cannot un-ignore a file whose parent directory is excluded.
+/docs/*
+!/docs/agent-skill.md
 
 # ---- Workflow research temp files ----
 _synthesis.tmp.md
@@ -37,6 +40,9 @@ uv.lock
 .coverage.*
 htmlcov/
 coverage.xml
+# machine-readable report consumed by scripts/check_coverage.py — regenerated,
+# never committed (coverage-floors.json IS committed)
+coverage.json
 .tox/
 .nox/
 
@@ -58,3 +64,10 @@ IEDriverServer.exe
 *.swp
 .DS_Store
 Thumbs.db
+
+# ---- Agent skills installed into this repo (build artifacts) ----
+# `visus install --skills` writes the shipped skill here; the source of truth is
+# src/visus/web/skills/visus-web/. The contributor skill under
+# .claude/skills/developing-visus-web/ IS tracked — do not widen this to `.claude/`.
+.claude/skills/visus-web/
+.agents/

visus_web-0.3.2/.pre-commit-config.yaml

@@ -0,0 +1,30 @@
+# Local-only hooks. Every hook shells out to the exact command CI runs, so
+# `pre-commit run --all-files` and the `lint` job in .github/workflows/ci.yml
+# cannot disagree, and no hook needs a network clone or a second pinned copy of
+# ruff/mypy.
+#
+# Setup (see TESTING.md):
+#   uv venv && uv pip install -e ".[dev,vision,mcp,cli]"
+#   uv run pre-commit install
+#
+# Requires `uv` on PATH.
+repos:
+  - repo: local
+    hooks:
+      - id: ruff
+        name: ruff check (src, tests, scripts)
+        entry: uv run ruff check --force-exclude
+        language: system
+        types_or: [python, pyi]
+        files: ^(src|tests|scripts)/
+        require_serial: true
+
+      - id: mypy
+        name: mypy --strict (visus.web)
+        entry: uv run mypy
+        language: system
+        types_or: [python, pyi]
+        files: ^src/
+        # [tool.mypy] packages = ["visus.web"] — mypy selects its own inputs.
+        pass_filenames: false
+        require_serial: true

visus_web-0.3.2/CONTRIBUTING.md

@@ -0,0 +1,74 @@
+# Contributing to visus.web
+
+## Setup
+
+```bash
+uv venv && uv pip install -e ".[dev,vision,mcp,cli]"
+uv run pre-commit install
+```
+
+`[vision]` (RapidOCR + OpenCV) is required for the OCR/image tests; `[mcp]` and `[cli]`
+for the MCP/CLI tests. See `TESTING.md` for what the suite covers and how it is split.
+
+## The gate
+
+Everything below must pass before a change is proposed. CI runs the same commands
+(`.github/workflows/ci.yml`), so a green local run is a green CI run.
+
+```bash
+uv run ruff check src tests scripts
+uv run mypy
+uv run pytest -q --cov-fail-under=90 --cov-report=json
+uv run python scripts/check_coverage.py
+```
+
+- The **global** 90% floor lives in `[tool.coverage.report] fail_under` and is passed
+  explicitly on the pytest command. It is deliberately *not* in `addopts`: a filtered
+  run (`-m "not browser"` selects most, but not all, of the suite) can never reach it.
+- `scripts/check_coverage.py` adds **per-package** floors from `coverage-floors.json`,
+  so `cli/` and `mcp/` — roughly 3,200 of 11,300 lines — cannot rot behind a healthy
+  total. Floors ratchet up only: after coverage improves, re-run
+  `uv run python scripts/check_coverage.py --freeze` and commit the new numbers.
+
+While iterating on one file, drop the gate but keep the table:
+
+```bash
+uv run pytest tests/test_e2e_locators.py -m browser --cov-fail-under=0
+```
+
+The suite runs under `pytest-xdist` and takes roughly 15 minutes end to end with the
+real-browser lane (currently 965+ collected items — it grows, so treat any specific
+count as a snapshot, not a target); markers are `browser` and `packaging`.
+
+## How to write a change
+
+**TDD is not optional.** Write the failing test, run it and watch it fail for the right
+reason, write the smallest implementation that makes it pass, run it again, commit.
+
+**Real browser, no fakes.** Anything that touches a page is marked `@pytest.mark.browser`
+and drives a real headless Chromium against a local fixture page in `tests/fixtures/*.html`,
+served by the session-scoped `base_url` fixture in `tests/conftest.py`. A test double may
+isolate a unit only when the same behaviour is *also* proven by an integration test.
+
+**No `skip`, no `xfail`, no mock-to-pass.** `tests/test_repo_hygiene.py` enforces this;
+the two pre-existing conditional skips are allowlisted there by name with their reason.
+
+**The hidden-engine rule.** No `selenium.*` type may appear in any public signature,
+return type, exception or docstring. Selenium imports are allowed only in
+`src/visus/web/backends/browsers/*.py`, `src/visus/web/backends/selenium/*.py` and
+`src/visus/web/backends/selenium_backend.py`.
+
+## Commits
+
+Small and semantic: `label(scope): description`, where `label` is one of
+`fix`, `feat`, `chore`, `docs`, `test`, `devops`.
+
+```
+fix(session): invalidate cached field indexes on navigation
+```
+
+Never add `Co-Authored-By` trailers.
+
+## Releasing
+
+See `RELEASING.md`.