vrex-flow-engine 0.3.2__tar.gz → 0.3.4__tar.gz

{vrex_flow_engine-0.3.2 → vrex_flow_engine-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: vrex-flow-engine
-Version: 0.3.2
+Version: 0.3.4
 Summary: Launcher + supervisor and OpenAI-compatible image/video generation service (Vrex Flow Engine).
 Author: Vrex
 License: Proprietary

vrex_flow_engine-0.3.4/README.md

@@ -0,0 +1,373 @@
+# flow-engine
+
+Host a **Google Flow browser-extension instance** and expose **OpenAI-compatible
+image/video generation endpoints** backed by Google Flow (flow.google.com).
+
+flow-engine drives a Chrome extension that borrows your signed-in Flow session
+(Google cookies + the page's XSRF token + reCAPTCHA solving) and proxies Flow's
+private `batchexecute` RPCs, mapping the results onto familiar OpenAI
+request/response shapes. PyPI package `vrex-flow-engine`, import module
+`flow_engine`, console commands `vrex-flow-engine` / `flow-engine` (subcommands:
+`start`, `serve`, `setup`, `doctor`).
+
+> **Transport status (2026-09).** Flow moved from the Next.js app at
+> `labs.google/fx/tools/flow` (Bearer `ya29` tokens against
+> `aisandbox-pa.googleapis.com`) to an Angular app at `flow.google.com` whose
+> only backend is `/_/AiSandboxAngularFrontend/data/batchexecute`. The old
+> transport now fails with *"Request had invalid authentication credentials"*.
+> flow-engine speaks the new protocol for everything the recorded sessions
+> exercised (images, uploads, text/image/reference-to-video); the few modes
+> whose payload has not been captured yet answer **HTTP 501** — see "Porting
+> more Flow features" below.
+
+```
+OpenAI client ──HTTP :8101──> flow-engine ──WS :9223──> Chrome extension ──fetch──> Google Flow
+                               (this repo)               (extension/, in your browser)
+```
+
+In production the HTTP surface is published at `https://flow.getvrex.com`
+(a `cloudflared` tunnel → `127.0.0.1:8101`); the Next.js app reaches it via
+`FLOW_ENGINE_URL`. See "Expose to production" below.
+
+## Endpoints
+
+- `POST /v1/videos/generations` + `GET /v1/videos/{id}` — async video job + poll
+  (Veo 3.1 lite, low-priority queue): text-to-video, image-to-video
+  (`start_images`), reference-to-video (`ref_images`), frames-to-video
+  (`end_images` / `loop`) and 1080p upsample of a generated clip
+  (`mode: "upsample"` + `source_media_id`). `mode: extend|edit` and r2v
+  `reference_audio` answer `501` until captured.
+- `GET /v1/videos/{id}/debug` — raw `{dispatch, last_poll}` Google payloads for a job (troubleshooting).
+- `POST /v1/images/generations` — synchronous image (Nano Banana 2), optional `ref_images`.
+- `POST /v1/flow/uploads` — image **or video** bytes → Flow media id (routes by mime).
+- `GET /v1/models` — model catalog (cached on disk, refresh with `?refresh=1`);
+  each mode reports `dispatchable`.
+- `GET /v1/audio/voices` — Flow's preset voices (read from the default project).
+- `GET /media/{id}` — cached generated bytes at a stable URL.
+- `GET /api/health` — liveness + extension-connected status (ungated).
+
+`/v1/*` and `/api/pool/status` require `Authorization: Bearer <FLOW_ENGINE_API_KEY>`.
+`/api/health` and `/media/{id}` are ungated.
+
+## Requirements
+
+- Python 3.10+
+- Google Chrome signed in to **flow.google.com** with a Flow tab open, and the
+  extension in `extension/` loaded (chrome://extensions → Load unpacked).
+- A Flow account (Pro or Ultra — the paygate tier is read from your session;
+  the pinned `veo_3_1_lite_low_priority` family is only enabled on Ultra).
+- The engine generates into a project titled **`flow_engine`** — found by
+  title, created on first use when missing; `FLOW_DEFAULT_PROJECT_ID` pins one.
+- `cloudflared` (only for exposing to production).
+
+> The extension's `background.js` hardcodes `ws://127.0.0.1:9223` and
+> `http://127.0.0.1:8101/api/ext/callback`, so flow-engine defaults to those
+> ports. It therefore **cannot run alongside the legacy flowboard agent on
+> defaults** — change both the env vars and the extension URLs to run both.
+
+## Quickstart — one command
+
+`flow-engine start` is a supervisor: it stores your secrets, launches the engine
+**and** the Cloudflare tunnel as child processes, restarts either on crash, and
+renders a live status dashboard. The package is published to PyPI as
+**`vrex-flow-engine`**. Run it with [uv](https://docs.astral.sh/uv/) (the Python
+equivalent of `npx` — no manual venv):
+
+```bash
+uvx vrex-flow-engine start
+```
+
+`uvx` ships with `uv`. If you get `uvx: command not found`, install uv first
+(then re-run the line above):
+
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh   # then: exec $SHELL   (adds ~/.local/bin to PATH)
+# or on macOS: brew install uv
+```
+
+No uv? It's a normal PyPI package — `pipx run vrex-flow-engine start`, or
+`pip install --user vrex-flow-engine && flow-engine start`.
+
+First run prompts for the **engine API key** (must match the value the Next.js
+app sends — `FLOW_ENGINE_API_KEY` on Vrex Multimodal / `.env.prd`) and the
+**Cloudflare tunnel token** (dashboard → Zero Trust → Networks → Tunnels → your
+tunnel → "run a connector"). Both are saved to `~/.vrex-flow/config.json` (0600)
+so later runs need no input. Then it brings up the engine + tunnel and shows:
+
+```
+🎬 Vrex Flow Engine   ● running   uptime 3m02s
+┌ engine ──────────────┐ ┌ tunnel ───────────────┐
+│ status     ● UP       │ │ status  ● connected    │
+│ extension  ● connected│ │ public  flow.getvrex.com│
+│ instances  1          │ │ →       127.0.0.1:8101  │
+└──────────────────────┘ └───────────────────────┘
+```
+
+The one thing the CLI can't automate: open a `https://flow.google.com` tab in
+Chrome with the `extension/` loaded and signed into a Flow Pro/Ultra account.
+Until you do, the dashboard shows `extension ● OPEN A FLOW TAB`. Ctrl-C stops
+both processes cleanly.
+
+### CLI commands
+
+| Command | What it does |
+|---|---|
+| `flow-engine start` | Supervise engine + tunnel + live dashboard (the one-liner). `--no-tunnel` runs the engine only; `--tunnel-config <path>` runs a named tunnel from a cloudflared config.yml instead of a token; `--non-interactive` fails instead of prompting. |
+| `flow-engine setup` | (Re)store the engine key + tunnel token in `~/.vrex-flow/config.json`. |
+| `flow-engine doctor` | Check prerequisites (python, cloudflared, config) and probe a running engine. |
+| `flow-engine serve` | Run just the engine in the foreground (for systemd / your own supervisor). |
+
+Secrets can also come from env (`FLOW_ENGINE_API_KEY`, `CLOUDFLARE_TUNNEL_TOKEN`)
+or flags (`--engine-key`, `--tunnel-token`) — flags > env > stored config.
+
+## Run manually (without the CLI)
+
+The engine reads config from the environment directly — **no `.env` auto-load**.
+`FLOW_ENGINE_API_KEY` is **required**; the server refuses to boot without it.
+
+```bash
+cd flow-engine
+python -m venv .venv && source .venv/bin/activate
+pip install -e .
+export FLOW_ENGINE_API_KEY=<shared-secret>     # must match the caller's key
+flow-engine serve                              # serves :8101 (HTTP) + :9223 (WS)
+# equivalent: python -m flow_engine.main
+```
+
+Load `extension/` in Chrome, open a `https://flow.google.com` tab, then check:
+
+```bash
+curl localhost:8101/api/health
+# {"ok":true,"extension_connected":true,"instances":1}
+curl localhost:8101/api/pool/status -H "authorization: Bearer $FLOW_ENGINE_API_KEY"
+# {"instances":[{"account_id":"acct-001","connected":true,"session_present":true,
+#   "email":"…","paygate_tier":3,"credits":25050,"in_flight":0,"capacity":4}]}
+```
+
+`extension_connected:false` means no extension is bridged yet — reload the
+extension. `session_present:false` / `paygate_tier:null` means the extension is
+bridged but Chrome is not signed in to flow.google.com (a generation request
+answers `503 account_unresolved`).
+
+### Tests
+
+```bash
+cd flow-engine
+uv run --with pytest python -m pytest tests -q     # parsers, catalog resolution
+node --test tests/batchexecute-helpers.test.mjs    # extension batchexecute helpers
+```
+
+## Expose to production (Cloudflare Tunnel)
+
+The public origin `https://flow.getvrex.com` is a `cloudflared` tunnel to the
+local `:8101`. **`flow-engine start` runs this tunnel for you** (via the stored
+token), so normally you don't touch `cloudflared` directly. To run it standalone
+— e.g. the `serve` path — start the tunnel as a separate process on the same
+machine:
+
+```bash
+cloudflared tunnel run --token <your-tunnel-token>   # flow.getvrex.com → 127.0.0.1:8101
+# or, with a named-tunnel config file:
+cloudflared tunnel run --config cloudflared.yml
+```
+
+If `https://flow.getvrex.com` returns **HTTP 530 / `error code: 1033`**, the
+tunnel is down (Cloudflare has no active connection to the origin) — restart it
+(or `flow-engine start`). A `502`/`1016` instead means the tunnel is up but the
+engine on `:8101` is not — restart the engine.
+
+## Examples
+
+Image (`model` must be `NANO_BANANA_2`, or omit to force it; `size` picks the
+aspect — `1024x1024` square, `1280x720` landscape, `720x1280` portrait):
+
+```bash
+curl localhost:8101/v1/images/generations \
+  -H "authorization: Bearer $FLOW_ENGINE_API_KEY" \
+  -H 'content-type: application/json' \
+  -d '{"prompt": "a lighthouse at dusk", "n": 2, "size": "1280x720"}'
+# {"created":…,"data":[{"url":"http://127.0.0.1:8101/media/<id>"}, …]}
+```
+
+Text-to-video (`model` must be `veo_3_1_lite_low_priority`):
+
+```bash
+curl localhost:8101/v1/videos/generations \
+  -H "authorization: Bearer $FLOW_ENGINE_API_KEY" \
+  -H 'content-type: application/json' \
+  -d '{"prompt": "a cat playing", "model": "veo_3_1_lite_low_priority", "size": "1280x720"}'
+# {"id":"video_…","status":"in_progress","mode":"t2v","model_key":"veo_3_1_t2v_lite_low_priority",…}
+curl localhost:8101/v1/videos/video_… \
+  -H "authorization: Bearer $FLOW_ENGINE_API_KEY"   # poll until status == "completed"
+```
+
+Image-to-video — pass the start frame inline; flow-engine uploads it for you.
+Reference-to-video ("Ingredients", up to 3 images) works the same way with
+`ref_images` / `ref_media_ids`:
+
+```bash
+curl localhost:8101/v1/videos/generations \
+  -H "authorization: Bearer $FLOW_ENGINE_API_KEY" \
+  -H 'content-type: application/json' -d "{
+    \"prompt\": \"the subject turns to camera\", \"model\": \"veo_3_1_lite_low_priority\",
+    \"start_images\": [\"$(base64 < frame.png)\"], \"size\": \"1280x720\"
+  }"
+```
+
+End frames (`end_images` / `end_media_ids`) turn i2v into frames-to-video —
+motion between two stills. `loop: true` reuses the start frame as the end
+frame, which yields a clip finishing on the frame it began with, so it tiles
+seamlessly behind narration:
+
+```bash
+curl localhost:8101/v1/videos/generations \
+  -H "authorization: Bearer $FLOW_ENGINE_API_KEY" \
+  -H 'content-type: application/json' \
+  -d '{"prompt": "subtle calm motion", "model": "veo_3_1_lite_low_priority",
+       "start_media_ids": ["<still media id>"], "loop": true, "size": "1280x720"}'
+# {"id":"video_…","status":"in_progress","mode":"interpolation",…}
+```
+
+`start_images` / `ref_images` / `end_images` accept a data URL, an `http(s)`
+URL, or bare base64. Already have a Flow media id (e.g. from `/v1/flow/uploads`)? Pass it in
+`start_media_ids` / `ref_media_ids` instead — the two forms merge. Uploading a
+reference **video** works via `/v1/flow/uploads -F file=@clip.mp4`, which
+returns a `"kind":"video"` media id (Flow registers it as PENDING and flips it
+to SUCCESSFUL a few seconds later).
+
+The completed job carries stable `/media/{id}` URLs; the bytes are fetched from
+Flow's short-lived signed CDN link once and cached locally.
+
+A finished clip can be upsampled to 1080p. No prompt is needed; `size` only
+tells Flow the clip's orientation (landscape by default). The job's media id
+is the source id with `_upsampled` appended, and Flow charges credits for it
+(25050 on the capturing account, where the 720p generations were free):
+
+```bash
+curl localhost:8101/v1/videos/generations \
+  -H "authorization: Bearer $FLOW_ENGINE_API_KEY" \
+  -H 'content-type: application/json' \
+  -d '{"mode": "upsample", "model": "veo_3_1_lite_low_priority",
+       "source_media_id": "<clip media id>", "size": "1280x720"}'
+# {"id":"video_…","status":"in_progress","mode":"upsample","model_key":"veo_3_1_upsampler_1080p",…}
+```
+
+The upsample item names the source clip's workflow. The engine fills it from
+its own registry when it generated the clip in this run, else it reads it from
+Flow's media entry; pass `workflow_id` to skip that lookup.
+
+Not yet available on flow.google.com (each answers **`501 Not Implemented`**
+naming the missing capture; the request contract is unchanged):
+`mode: "extend"`, `mode: "edit"`, r2v `reference_audio`.
+
+## Porting more Flow features
+
+Everything flow-engine knows about the flow.google.com wire format lives in
+`flow_engine/bridge/flow_rpc.py` (RPC ids, payload builders, response parsers)
+and was derived from recorded web sessions. To add a feature:
+
+1. In Chrome DevTools (Network, filter `batchexecute`) perform the action once
+   in the Flow web app — e.g. generate an image, upload a reference image, run
+   image-to-video — and export the HAR.
+2. For each new call note the `rpcids` query param, the decoded `f.req`
+   payload (a positional JSON array), and the `wrb.fr` response row.
+3. Add the RPC id + a payload builder + a parser to `flow_rpc.py` with a test
+   in `tests/test_flow_rpc.py` against the recorded response, then wire it in
+   `flow_sdk.py` and lift the mode out of the `501` guard (`SUPPORTED_MODES`
+   in `openai/videos.py`, `_DISPATCHABLE_MODES` in `catalog.py`, the image /
+   upload routes).
+
+### RPC ids can rotate — the extension re-discovers them
+
+The ids are Wiz RPC hashes; Google may change them in a future frontend
+build. Flow's JS declares every RPC with a descriptor that carries the stable
+backend method path next to the id (`new _.kx("nzlxg", …,
+"/VideoFxService.GetCredits")`), so the extension rebuilds a `method → id`
+map from the public bundle (base script + every lazy module, ~45 MB, fetched
+once per build label and cached in `chrome.storage`). Every `rpc_request`
+names its method (`flow_rpc.RPC_METHOD_BY_ID`); the extension calls the
+discovered id when it differs from the one the engine sent, and pushes the
+map to the engine, which logs a warning and lists the drift in
+`/api/pool/status` (`flow_build`, `rpc_map_size`, `rpc_ids_rotated`). A
+rotation therefore keeps working unattended; update the constants in
+`flow_rpc.py` when convenient. Payload *shapes* are not covered — a method
+whose request proto changes still needs a fresh capture.
+
+Known RPC ids so far: `YhhmEf` / `eb1hJf` / `MZZa6b` / `nprQif` generate
+video (t2v / i2v / r2v / frames-to-video; reCAPTCHA token at
+`payload[1][10][0]`), `p0UkFb` upsample a clip to 1080p (same context, batch
+id without the trailing `2`; the model key `veo_3_1_upsampler_1080p` is not in
+the catalog RPC), `fZytfe` / `jIps6` extend / edit (ids known, payloads not),
+`ogiZ0b` generate
+image (sync, token in both clientContext copies), `maseQ` upload image (bytes
+inline), `jwpduf` batch media status, `as29s` media detail with signed URLs,
+`UpteDb` list projects (page token at index 2), `jHPbke` create project,
+`Zzl0ze` project contents (+ preset voices), `HTrJv` model catalog, `nzlxg`
+credits + paygate tier. Video upload is not an RPC: a cookie-authed resumable
+upload against `/upload/v1/flow/upload/video/<projectId>` (see
+`extension/background.js` → `handleUploadRequest`).
+
+One open question: the reCAPTCHA *action* name the web client uses for image
+uploads is not visible in a capture (reCAPTCHA bodies are opaque). The engine
+sends `IMAGE_UPLOAD`; if Flow rejects uploads with a captcha error, override
+it with `FLOW_UPLOAD_CAPTCHA_ACTION`.
+
+## Configuration
+
+These configure the **engine** process (read from its environment). The
+`flow-engine start` launcher injects `FLOW_ENGINE_API_KEY` + ports for you from
+`~/.vrex-flow/config.json`; set these directly only for the manual `serve` path.
+All optional except the API key. Port/host/storage vars keep the `FLOWPROXY_`
+prefix for extension compatibility; the API key and pool size use the
+`FLOW_ENGINE_` / `FLOW_POOL_` names.
+
+| Env var | Default | Purpose |
+|---|---|---|
+| `FLOW_ENGINE_API_KEY` | — (**required**) | Bearer key gating `/v1/*`. Legacy `FLOWPROXY_API_KEY` still accepted. |
+| `FLOWPROXY_HTTP_PORT` | `8101` | HTTP surface (OpenAI endpoints + `/media` + callback). |
+| `FLOWPROXY_WS_HOST` | `127.0.0.1` | Extension WS bind — **must be loopback** (unauthenticated by design). |
+| `FLOWPROXY_EXT_WS_PORT` | `9223` | Extension WebSocket port. |
+| `FLOWPROXY_PUBLIC_BASE_URL` | `http://127.0.0.1:8101` | Base used to build absolute `/media` URLs returned to clients. |
+| `FLOW_POOL_CONCURRENCY` | `4` | Max concurrent in-flight Flow calls per instance (back-pressure, no 503). |
+| `FLOWPROXY_STORAGE` | `./storage` | Local byte cache dir. |
+| `FLOWPROXY_CATALOG_TTL` | `86400` | Model-catalog cache TTL (seconds). |
+| `FLOWPROXY_PROJECT_TITLE` | `flow_engine` | Title of the Flow project to generate into (found or created). |
+| `FLOW_DEFAULT_PROJECT_ID` | — | Pin the Flow project id instead of looking it up by title. |
+| `FLOW_UPLOAD_CAPTCHA_ACTION` | `IMAGE_UPLOAD` | reCAPTCHA action solved before an image upload (see "Porting more Flow features"). |
+| `VIDEO_POLL_MAX_CYCLES` / `VIDEO_POLL_INTERVAL_S` | `72` / `10` | Video poll budget (default 12 min). |
+
+## Layout
+
+```
+extension/            Chrome MV3 bridge (open a flow.google.com tab);
+                      batchexecute.js = shared request/response helpers
+flow_engine/
+  bridge/             WS server + flow_client (transport) + flow_rpc (wire
+                      format: ids, builders, parsers) + flow_sdk (high level)
+  media.py            DB-free media cache (registry + on-disk bytes)
+  jobs.py             in-memory job store + async video poller
+  session.py          readiness gate + default project + pool seam
+  catalog.py          model catalog (Flow catalog RPC), disk-cached
+  openai/             /v1/images, /v1/videos, /v1/flow/uploads, /v1/models
+  main.py             FastAPI app, /api/ext/callback, /media/{id}, lifespan
+  cli/                launcher: app (argparse), supervisor, dashboard,
+                      processes, health, config (~/.vrex-flow)
+```
+
+## Notes & caveats
+
+- **Single account.** One extension instance = one Google account = one tier.
+  Multi-tenant needs a browser-instance pool routed by API key — the
+  `session.get_bridge(api_key)` indirection is where that goes.
+- **Live Flow tab required.** Every dispatch solves an enterprise reCAPTCHA in
+  an open Flow tab; hosting needs a headful/virtual-display Chrome kept warm.
+  Polling and metadata calls only need the cookies + XSRF token, which the
+  extension reads from the flow.google.com page shell and refreshes every
+  30 min (and on the first rejected call).
+- **Signed URLs expire.** flow-engine caches bytes locally and serves stable
+  `/media/{id}` URLs.
+- **Localhost WS is unauthenticated** by design — never bind it to a network
+  interface. Gate the HTTP `/v1/*` surface with `FLOW_ENGINE_API_KEY` when exposed.
+- **No `.env` auto-load.** The process reads `os.environ` directly — export vars
+  in the shell (or a wrapper/launchd unit), don't rely on a `.env` file.
+```

{vrex_flow_engine-0.3.2 → vrex_flow_engine-0.3.4}/flow_engine/bridge/flow_client.py

@@ -12,8 +12,11 @@ Control flow:
    separate ``upload_request`` command (resumable upload, not batchexecute).
 4. That HTTP handler resolves the pending future by id.
 5. Inbound WS messages (``extension_ready``, ``session_captured``,
-   ``session_lost``, ``pong``) update per-instance state. A fresh session
-   triggers an account refresh (credits + paygate tier) through the bridge.
+   ``session_lost``, ``rpc_map``, ``pong``) update per-instance state. A fresh
+   session triggers an account refresh (credits + paygate tier) through the
+   bridge. ``rpc_map`` carries the ``method → rpcId`` table the extension
+   rebuilt from Flow's JS bundle; every ``rpc_request`` names its method so
+   the extension can substitute a rotated id.
 """
 from __future__ import annotations
 
@@ -61,6 +64,12 @@ class FlowClient:
         self._paygate_tier: Optional[int] = None
         self._credits: Optional[int] = None
         self._plan_code: Optional[int] = None
+        # RPC-id discovery pushed by the extension: Flow build label, the
+        # method → id map it read from the bundle, and the ids that differ
+        # from flow_rpc's constants (empty = everything still matches).
+        self._flow_build: Optional[str] = None
+        self._rpc_map_size: int = 0
+        self._rpc_ids_rotated: dict[str, dict[str, str]] = {}
         self._request_count = 0
         self._success_count = 0
         self._failed_count = 0
@@ -109,6 +118,18 @@ class FlowClient:
     def session_present(self) -> bool:
         return self._session_present
 
+    @property
+    def flow_build(self) -> Optional[str]:
+        return self._flow_build
+
+    @property
+    def rpc_map_size(self) -> int:
+        return self._rpc_map_size
+
+    @property
+    def rpc_ids_rotated(self) -> dict[str, dict[str, str]]:
+        return dict(self._rpc_ids_rotated)
+
     async def refresh_account(self) -> bool:
         """Resolve credits + paygate tier via the credits RPC. Returns True when
         the tier is cached. Runs through the extension, so it needs a live
@@ -153,6 +174,28 @@ class FlowClient:
             logger.warning("session_lost: %s", data.get("reason"))
             self._clear_session()
             return
+        if t == "rpc_map":
+            self._flow_build = data.get("build") if isinstance(data.get("build"), str) else None
+            discovered = data.get("map") if isinstance(data.get("map"), dict) else {}
+            self._rpc_map_size = len(discovered)
+            rotated: dict[str, dict[str, str]] = {}
+            for method, expected in flow_rpc.RPC_ID_BY_METHOD.items():
+                actual = discovered.get(method)
+                if isinstance(actual, str) and actual and actual != expected:
+                    rotated[method] = {"expected": expected, "actual": actual}
+            self._rpc_ids_rotated = rotated
+            if rotated:
+                logger.warning(
+                    "Flow build %s rotated %d rpc id(s); the extension resolves them by "
+                    "method name, update flow_rpc.py when convenient: %s",
+                    self._flow_build, len(rotated), rotated,
+                )
+            else:
+                logger.info(
+                    "rpc_map from Flow build %s: %d methods, all known ids match",
+                    self._flow_build, self._rpc_map_size,
+                )
+            return
         if t == "pong":
             return
         # Inbound response (legacy path; production flow uses HTTP callback)
@@ -248,6 +291,9 @@ class FlowClient:
         ``captcha_path`` is the single-path shorthand).
         """
         params: dict[str, Any] = {"rpcId": rpc_id, "payload": payload}
+        method = flow_rpc.RPC_METHOD_BY_ID.get(rpc_id)
+        if method:
+            params["method"] = method
         if captcha_action:
             params["captchaAction"] = captcha_action
             params["captchaPaths"] = captcha_paths or ([captcha_path] if captcha_path else [])

{vrex_flow_engine-0.3.2 → vrex_flow_engine-0.3.4}/flow_engine/bridge/flow_rpc.py

@@ -30,10 +30,42 @@ RPC_GENERATE_VIDEO_T2V = "YhhmEf"
 RPC_GENERATE_VIDEO_I2V = "eb1hJf"
 RPC_GENERATE_VIDEO_R2V = "MZZa6b"
 RPC_GENERATE_VIDEO_INTERPOLATION = "nprQif"
+RPC_GENERATE_VIDEO_UPSAMPLE = "p0UkFb"
 RPC_GENERATE_VIDEO = RPC_GENERATE_VIDEO_T2V  # historical alias
 RPC_CHECK_MEDIA = "jwpduf"
 RPC_GET_MEDIA = "as29s"
 
+# Known ids for methods whose payload has NOT been captured yet (kept so the
+# rotation check covers them and porting them later starts from a known id).
+RPC_GENERATE_VIDEO_EXTEND = "fZytfe"
+RPC_GENERATE_VIDEO_EDIT = "jIps6"
+
+# Backend method path behind each id, as declared in Flow's JS bundle
+# (``new _.kx("nzlxg", …, "/VideoFxService.GetCredits")``). The ids are Wiz
+# hashes Google may rotate; the method names are the stable anchor. The
+# extension rebuilds ``method → id`` from the live bundle on every build
+# change and resolves calls by method, so the ids here are the fallback.
+RPC_METHOD_BY_ID: dict[str, str] = {
+    RPC_LIST_PROJECTS: "FlowService.GetProjects",
+    RPC_CREATE_PROJECT: "AiSandbox.CreateProject",
+    RPC_PROJECT_INFO: "AiSandbox.GetProject",
+    RPC_PROJECT_CONTENTS: "FlowService.GetProjectContents",
+    RPC_MODEL_CATALOG: "FlowService.GetModels",
+    RPC_ACCOUNT_CREDITS: "VideoFxService.GetCredits",
+    RPC_UPLOAD_IMAGE: "FlowService.UploadImage",
+    RPC_GENERATE_IMAGE: "FlowService.BatchGenerateImages",
+    RPC_GENERATE_VIDEO_T2V: "VideoFxService.BatchAsyncGenerateVideoText",
+    RPC_GENERATE_VIDEO_I2V: "VideoFxService.BatchAsyncGenerateVideoStartImage",
+    RPC_GENERATE_VIDEO_R2V: "VideoFxService.BatchAsyncGenerateVideoReferenceImages",
+    RPC_GENERATE_VIDEO_INTERPOLATION: "VideoFxService.BatchAsyncGenerateVideoStartAndEndImage",
+    RPC_GENERATE_VIDEO_EXTEND: "VideoFxService.BatchAsyncGenerateVideoExtendVideo",
+    RPC_GENERATE_VIDEO_EDIT: "VideoFxService.BatchAsyncGenerateVideoEditVideo",
+    RPC_GENERATE_VIDEO_UPSAMPLE: "VideoFxService.BatchAsyncGenerateVideoUpsampleVideo",
+    RPC_CHECK_MEDIA: "VideoFxService.BatchCheckAsyncVideoGenerationStatus",
+    RPC_GET_MEDIA: "FlowService.GetMedia",
+}
+RPC_ID_BY_METHOD: dict[str, str] = {m: i for i, m in RPC_METHOD_BY_ID.items()}
+
 # Video upload is NOT a batchexecute call: a cookie-authenticated resumable
 # upload (``x-goog-upload-*`` headers) against this path, project id appended.
 UPLOAD_VIDEO_PATH = "/upload/v1/flow/upload/video/{project_id}"
@@ -45,6 +77,13 @@ TOOL_PINHOLE = 22
 CAPTCHA_APPLICATION_WEB = 1
 # Trailing int in the mediaGenerationContext ``[batchId, 2]`` pair.
 GENERATION_CONTEXT_FLAG = 2
+# The 1080p upsampler's model key. Flow's catalog RPC does not list the video
+# upsamplers (they are not a family the account picks a tier variant of), so
+# the key comes from the capture, not from ``catalog.resolve_video``.
+VIDEO_UPSAMPLER_1080P_KEY = "veo_3_1_upsampler_1080p"
+# Second int in an upsample item (index 6). Observed as 2 alongside a landscape
+# source; its meaning is not visible in the capture.
+UPSAMPLE_ITEM_FLAG = 2
 # Where the extension writes the solved reCAPTCHA token. A clientContext is
 # ``[null, 22, null×3, projectId, null×4, [token, 1]]`` so the token sits at
 # ``[10][0]`` of wherever the context lives in a payload.
@@ -206,12 +245,38 @@ def video_request_item_r2v(
     ]
 
 
+def video_request_item_upsample(
+    source_media_id: str,
+    workflow_id: str,
+    aspect_enum: int,
+    scene_id: Optional[str] = None,
+    model_key: str = VIDEO_UPSAMPLER_1080P_KEY,
+) -> list[Any]:
+    """Upsample item (``p0UkFb``): ``[[null, sourceMediaId], null, aspect,
+    null, [null, workflowId, null×2, sceneId], null, 2, null×24, modelKey]``.
+    The web client sends the source clip's own workflow id — the one the
+    generate / get-media responses carry — and a fresh scene id. The result is
+    a new media whose id is the source id with ``_upsampled`` appended."""
+    return [
+        [None, source_media_id], None, aspect_enum, None,
+        [None, workflow_id, None, None, scene_id or uuid_upper()],
+        None, UPSAMPLE_ITEM_FLAG, *([None] * 24), model_key,
+    ]
+
+
 def generate_video_payload(
-    items: list[list[Any]], project_id: str, batch_id: Optional[str] = None
+    items: list[list[Any]],
+    project_id: str,
+    batch_id: Optional[str] = None,
+    with_context_flag: bool = True,
 ) -> list[Any]:
-    """Body shared by the three video RPCs: ``[items, clientContext,
-    [batchId, 2]]``. Token goes at ``GENERATE_VIDEO_CAPTCHA_PATH``."""
-    return [items, client_context(project_id), [batch_id or uuid_upper(), GENERATION_CONTEXT_FLAG]]
+    """Body shared by the video RPCs: ``[items, clientContext, [batchId, 2]]``.
+    Token goes at ``GENERATE_VIDEO_CAPTCHA_PATH``. The upsample RPC sends the
+    batch id alone (``[batchId]``) — pass ``with_context_flag=False``."""
+    context = [batch_id or uuid_upper()]
+    if with_context_flag:
+        context.append(GENERATION_CONTEXT_FLAG)
+    return [items, client_context(project_id), context]
 
 
 def image_request_item(
@@ -351,17 +416,27 @@ def parse_media_entry(entry: Any) -> dict[str, Any]:
 def parse_generate_video(data: Any) -> dict[str, Any]:
     """``YhhmEf`` → ``{credits, workflows:[{name, primary_media_id}], media}``.
 
-    ``data[2]`` lists the new workflows as ``[workflowId, null, null,
+    ``data[2]`` lists the workflows as ``[workflowId, null, null,
     [title, createTime, null, null, mediaId, batchId, updateTime], projectId]``;
     ``data[3]`` lists the media rows (see ``parse_media_entry``).
+
+    The media to poll is taken from the media row that names the workflow,
+    not from the workflow's own ``mediaId`` slot: for a fresh generation the
+    two agree, but a derived generation (upsample) reuses the source clip's
+    workflow, whose slot still points at the source. Polling that would
+    report the 720p clip as the finished 1080p one.
     """
+    media = [parse_media_entry(m) for m in _list(_at(data, 3))]
+    new_media_by_workflow = {
+        m["workflow_id"]: m["media_id"]
+        for m in media if m.get("workflow_id") and m.get("media_id")
+    }
     workflows: list[dict[str, str]] = []
     for wf in _list(_at(data, 2)):
         wf_id = _str(_at(wf, 0))
-        media_id = _str(_at(wf, 3, 4))
+        media_id = new_media_by_workflow.get(wf_id) or _str(_at(wf, 3, 4))
         if wf_id and media_id:
             workflows.append({"name": wf_id, "primary_media_id": media_id})
-    media = [parse_media_entry(m) for m in _list(_at(data, 3))]
     if not workflows:
         # Fall back to the media rows, which also carry their workflow id.
         for m in media: