dv-pipecat-flows 0.0.23.dev9__tar.gz → 0.0.23.dev10__tar.gz

{dv_pipecat_flows-0.0.23.dev9 → dv_pipecat_flows-0.0.23.dev10}/.gitignore

@@ -187,3 +187,5 @@ MANIFEST
 
 # Asterisk configs
 remote-asterisk-code/pjsip.conf
+.venvringg1
+.venvringg2

{dv_pipecat_flows-0.0.23.dev9 → dv_pipecat_flows-0.0.23.dev10}/AGENTS.md

@@ -125,6 +125,23 @@ Example `call_config`:
 - `tests/test_manager.py`, `tests/test_actions.py`, etc., cover FlowManager and ActionManager edge cases (context resets, action ordering, function registration). Extend these tests when adding new features.
 - Logging: see the dedicated section below for Loguru usage and the custom logger configuration.
 
+## Metrics & Observability
+
+- **SDK**: `ringg-telemetry` (metrics-only facade over OpenTelemetry). Initialised once per process in the FastAPI lifespan — `ringg-bot/server.py` — gated on `ENABLE_METRICS`. Export path is OTLP/gRPC → the in-cluster `otel-gateway` → **Google Managed Prometheus**. Tracing is a *separate* stack (Langfuse, `ringg-bot/utils/tracing.py`); the two are unrelated.
+- **Declare instruments in one place**: the registry block near the bottom of `ringg-bot/utils/metrics_collector.py`. Module-level handles (`ringg_metrics.histogram/counter/gauge`), created at import, resolved lazily. Keep the cardinality justification in a comment next to each — the `labels=(...)` tuple is a hard allowlist, and the SDK **silently drops** any label not in it.
+- **Record from a lazily-importing, swallow-all helper** when the call site is on a per-turn hot path — see `_record_redis_latency` in `ringg-bot/ringg_processors/llm_cache.py` and `_record_router_metrics` in `ringg-bot/utils/language_detection.py`. Telemetry must never raise into a live call, and the import must not drag the metrics stack into eval scripts.
+- **Names**: the SDK prefixes everything with `ringg.` and GMP rewrites it for PromQL — `ringg.language.router.duration` + `unit="s"` becomes `ringg_language_router_duration_seconds_{bucket,count,sum}`; a counter with `unit="1"` becomes `..._total`. A non-standard unit gets appended as a suffix, so use `"1"` even for money (put "USD" in the description).
+- **Dashboards as code**: `k8s/monitoring/dashboards/` — one JSON per dashboard plus a README with the `gcloud monitoring dashboards create/update` commands and the assigned resource ids.
+- **Tests**: `ringg_telemetry.testing.install_test_provider()` / `uninstall_test_provider()` route handles to an in-memory reader. See `ringg-bot/tests/test_metrics_bridge.py` and `tests/test_language_router_metrics.py`.
+
+## Data Residency
+
+- Some regions forbid their call data leaving regional storage. The single definition of that rule is `is_data_residency_restricted(region)` in `ringg-bot/utils/artifact_store.py` (currently `{"ksa"}`). **Consult it before any new write to the long-lived artifact bucket** — `RUNTIME_CONFIG_BUCKET` is US-located.
+- Restricted regions **skip** the write entirely: there is no fallback bucket and no re-routing. Applies to the runtime-config snapshot, the tool-call snapshot and the language-router samples (`utils/generic_functions/cleanup.py`, `utils/language_sample_store.py`). The data still reaches the region's own backend via the `call_completion` POST, and transcripts/recordings still go to the regional bucket via `utils/transcript._get_gcs_bucket`.
+- `region` is a **per-call** value (`RunConfig.region`, set by the backend), not a pod env var. On the two artifact uploaders it is keyword-only and *required* precisely so a new call site cannot become a violation by omission.
+- Skipping the upload must not skip local cleanup: `LOCAL_STORAGE_DIR` is the pod's `/tmp` and the pod outlives thousands of calls, so anything buffered on disk still has to be unlinked on the skip path.
+- Where feasible, gate at *capture* time too, not just upload time — e.g. `bot.py` passes `collect_samples=not is_data_residency_restricted(_region)` so raw user utterances are never written to `/tmp` for a restricted call.
+
 ## Logging
 
 - We use Loguru for all application logging. Do not use the Python `logging` module in app code; always `from loguru import logger`.
@@ -399,6 +416,7 @@ curl -X POST https://api.ringg.ai/call \
 
 - `provider_metadata.asterisk_channel_id`, `provider_metadata.asterisk_sip_provider` — set by the bridge for Asterisk calls.
 - `provider_metadata.transfer.type`, `provider_metadata.transfer.sip_domain` — may be injected from dialplan variables (`TRANSFER_TYPE`, `TRANSFER_SIP_DOMAIN`) or ARI variables for outbound calls. Agents typically do not need to set these.
+- `call_config.transfer.include_call_numbers` — when true, `dialplan_context` transfers add this call's `to_number` / `from_number` to the SIP MESSAGE body. Defaults off.
 
 ## Next Steps
 

{dv_pipecat_flows-0.0.23.dev9/src/dv_pipecat_flows.egg-info → dv_pipecat_flows-0.0.23.dev10}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dv-pipecat-flows
-Version: 0.0.23.dev9
+Version: 0.0.23.dev10
 Summary: Conversation Flow management for Pipecat AI applications
 License: BSD 2-Clause License
 Project-URL: Source, https://github.com/pipecat-ai/pipecat-flows

dv_pipecat_flows-0.0.23.dev10/browser_copilot/.env.example

@@ -0,0 +1,13 @@
+# Browser Copilot eval harness — environment template (copy to .env, fill in)
+# NO real secrets in this file.
+#
+# Only the eval/ benchmark harness reads this file. It needs Azure OpenAI creds
+# (COPILOT_AZURE_API_KEY / COPILOT_AZURE_ENDPOINT — see eval/_env.py). The model
+# and API version are pinned in eval/bench.py; the values below just document
+# what the harness runs against.
+
+# --- LLM provider (Azure OpenAI) — used by the eval harness ---
+COPILOT_AZURE_ENDPOINT=https://<your-azure-openai>.openai.azure.com
+COPILOT_AZURE_API_KEY=<azure-openai-api-key>
+COPILOT_AZURE_API_VERSION=2024-10-21
+COPILOT_MODEL=gpt-4.1-mini

dv_pipecat_flows-0.0.23.dev10/browser_copilot/.gitignore

@@ -0,0 +1,5 @@
+.venv/
+.env
+__pycache__/
+*.pyc
+.venv-stagehand/

dv_pipecat_flows-0.0.23.dev10/browser_copilot/README.md

@@ -0,0 +1,71 @@
+# Browser Copilot (beta)
+
+An embeddable, screen-aware AI copilot for web apps. The agent **sees the current
+page** (structured DOM perception — no screenshots, no screen share), **answers
+questions about it**, and **acts on it** (highlight / click / type / …) under a
+configurable confirmation policy.
+
+**One brain, two I/O modes.** Chat and voice are BOTH LiveKit sessions of the
+real Ringg agent (pipecat runtime) — they differ only in I/O: **voice** =
+mic + captions (`media_type=audio`), **chat** = typed text over `lk.chat`
+(`media_type=text`). Both drive the widget's `cp_action` executor. There is no
+separate chat server anymore (the old gpt-4.1-mini `/plan` orchestrator was
+retired — see `docs/METHODS.md`); this package now holds the **eval** harness +
+**docs** only.
+
+```mermaid
+flowchart LR
+    W["Widget<br/>perceive (values redacted) · execute · confirm<br/>chat⇄voice toggle"]
+    B["Voice runtime (pipecat)<br/>DomContextInjector · browser_actions"]
+
+    W <-->|"dom_summary ↓ · cp_action ↑ (both modes)"| B
+    W -->|"audio | text"| B
+```
+
+Full diagrams (component graph, chat/voice sequences, enablement):
+[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
+
+## Layout
+
+```
+eval/      benchmark harness + fixture pages + committed baseline results
+docs/      METHODS (all approaches + numbers), ARCHITECTURE, CONTRACTS, CONFIG,
+           DISTRIBUTION, VOICE_E2E, CODE_MAP (the system file-by-file, all 4 repos)
+scripts/   dev-stack.sh — one-shot local stack (backend/runtime/ngrok/frontend)
+```
+
+The copilot BRAIN is the real agent in the pipecat runtime
+(`../ringg-bot/`) — this package no longer ships a server. The FE widget lives
+in **`dv-frontend-v2/apps/agents-cdn`** (`src/copilot/`, branch
+`feat/browser-copilot`) — the embeddable CDN widget that already owns the
+LiveKit room + chat/voice UI; activation is backend-driven (`copilot_enabled`
+in the webcall response — no widget env). The runtime bridge lives in
+`ringg-bot/ringg_processors/` + `utils/flow_tools/browser_actions.py`.
+(The earlier `fe-ringg-v2-app` widget was removed — one FE home.)
+
+## Run
+
+**The whole local stack in one go** (backend + voice runtime + ngrok +
+frontend, with branch/env/Krisp preflight):
+
+```bash
+scripts/dev-stack.sh up        # start everything (idempotent, 4 services)
+scripts/dev-stack.sh status    # health + branch matrix
+scripts/dev-stack.sh logs runtime   # tail a service (runtime|frontend|ngrok)
+scripts/dev-stack.sh down      # stop everything it started
+```
+
+## Eval (the numbers gate)
+
+Any change to perception/prompts must re-run the benchmark before docs cite numbers:
+
+```bash
+uv run python eval/bench.py               # hybrid v1/v2 + screenshot, QA + action funnel
+```
+
+Results land in `eval/results_*.json`; summaries in `eval/RESULTS.md` and `docs/METHODS.md`.
+
+The FE widget lives in `dv-frontend-v2/apps/agents-cdn` (`src/copilot/`,
+branch `feat/browser-copilot`; e2e in `packages/tests` — `pnpm --filter tests
+test:copilot`); the voice runtime bridge lives in `ringg-bot/ringg_processors/`
+on this repo's copilot branches.

dv_pipecat_flows-0.0.23.dev10/browser_copilot/docs/ARCHITECTURE.md

@@ -0,0 +1,190 @@
+# Architecture (as-built, beta)
+
+Original HLD (design rationale, roadmap phases): [HLD.md](HLD.md). This doc is
+the shorter as-built picture. Contracts: [CONTRACTS.md](CONTRACTS.md) · modes:
+[CONFIG.md](CONFIG.md) · method evidence: [METHODS.md](METHODS.md).
+
+## The whole system at a glance
+
+```mermaid
+flowchart TB
+    subgraph BROWSER["🖥️ Browser — the user's page"]
+        UI["Widget UI<br/>mini chat · chat⇄voice toggle · confirm bar"]
+        PERC["Perception (src/copilot/perception.ts)<br/>data-cp tagging · text/rows/headings ·<br/>state flags · VALUES NEVER LEAVE"]
+        EVT["Events (events.ts)<br/>focus / typing-length / selection / click"]
+        EXEC["Executor (executor.ts)<br/>runCpAction: click · type(native setter) ·<br/>select · scroll · highlight — confirm bar"]
+    end
+
+    subgraph VOICE["🎙️ The agent — one brain, two I/O modes"]
+        BE["backend /calling/webcall<br/>copilot_support: tools + copilot_config"]
+        LK[("LiveKit room<br/>audio | text (lk.chat) + RPC")]
+        subgraph BOTS["pipecat runtime (ringg-bot)"]
+            BWF["bot_with_flows.py<br/>multi-node"]
+            BOT["bot.py<br/>single-node"]
+            INJ["DomContextInjector<br/>ONE [Browser context] msg"]
+            TOOLS["browser_actions tools<br/>guardrails + act-then-wait"]
+        end
+    end
+
+    UI -->|"startWebCall (media_type: audio | text)"| BE --> LK
+    PERC -->|receive_dom_summary RPC| INJ
+    EVT -->|receive_user_event RPC| INJ
+    BWF --- INJ
+    BOT --- INJ
+    BWF --- TOOLS
+    BOT --- TOOLS
+    TOOLS -->|cp_action RPC| EXEC
+    EXEC -->|"{ok, result}"| TOOLS
+```
+
+## Two planes
+
+```
+┌─────────────────────── BROWSER (the user's page) ────────────────────────┐
+│ dv-frontend-v2 agents-cdn widget  (src/copilot/)                         │
+│  • Perception: id-contract collector (data-cp registry, label/value      │
+│    pairing, tables, viewport, redaction) + MutationObserver + SPA hooks  │
+│  • Execution: runCpAction — full human-action vocabulary + confirm bar   │
+│  • UI: floating mini chat (ChatWindowFrame) — chat⇄voice toggle; both    │
+│    are LiveKit sessions of the agent (text vs audio I/O)                  │
+└───────────────────────────────────▲──────────────────────────────────────┘
+        startWebCall(media_type)     │ receive_dom_summary ↓ · cp_action ↑
+                                     ▼
+              ┌────────────── VOICE RUNTIME (ringg-bot) ──────────────┐
+              │ LiveKit bot = the REAL agent (STT/TTS or lk.chat text)│
+              │ DomContextInjector ← receive_dom_summary RPC;         │
+              │ browser_actions flow tools → cp_action RPC; captions  │
+              │ via native publish_transcription. bot.py (single-node)│
+              │ + bot_with_flows.py (multi-node) share the tools.     │
+              └───────────────────────────────────────────────────────┘
+```
+
+## The loop (both modes)
+
+**perceive → the agent decides → guardrail → execute → ack → re-perceive**
+
+- Perception is *structured text*, never pixels: the collector emits addressable
+  elements (`[cp:e11] textbox: Coupon code`) with values REDACTED in-browser,
+  pushed to the bot as `receive_dom_summary` → ONE `[Browser context]` message.
+- The agent references those `[cp:…]` handles in a `cp_action` tool call.
+- Guardrails on the bot (`copilot_config`: origin allowlist / kill-switch /
+  high_risk_confirm) + the widget's one-tap confirm bar.
+- Every execution returns `{ok, result}` over the RPC — the agent learns what
+  happened; mutating actions then WAIT (act-then-wait) for the fresh page.
+
+### Chat turn (media_type = text)
+
+```mermaid
+sequenceDiagram
+    autonumber
+    actor U as User
+    participant W as Widget
+    participant LK as LiveKit (lk.chat)
+    participant B as Bot (pipecat)
+    participant LLM as Agent LLM
+
+    Note over W,B: on connect + every page change
+    W->>LK: receive_dom_summary (DomSummary v2)
+    LK->>B: → DomContextInjector → [Browser context]
+    U->>W: type "open Campaigns"
+    W->>LK: lk.chat send
+    LK->>B: typed message
+    B->>LLM: [Browser context] + chat history
+    LLM-->>B: tool call click_element(cp="e12", say="Opening Campaigns")
+    B->>B: guardrails: click is confirm-exempt → auto-run<br/>(only a destructive type would set require_confirm)
+    B->>LK: cp_action RPC (require_confirm:false)
+    LK->>W: → executor.runCpAction
+    W->>W: runs click directly (no confirm bar)
+    W-->>B: {ok:true}
+    B->>W: lk.chat: "Opening Campaigns — take a look."
+    Note over B: act-then-wait: no follow-up completion; next page pushes a fresh summary
+```
+
+### Voice turn (media_type = audio)
+
+```mermaid
+sequenceDiagram
+    autonumber
+    actor U as User
+    participant W as Widget
+    participant LK as LiveKit
+    participant B as Bot (pipecat)
+    participant LLM as Agent LLM
+
+    Note over W,B: on connect + every page change
+    W->>LK: receive_dom_summary (DomSummary v2)
+    LK->>B: → DomContextInjector
+    B->>B: render ONE "[Browser context]" msg<br/>(actions+flags · page content · focused · viewport)
+
+    U->>LK: 🎤 "open Campaigns"
+    LK->>B: audio → STT
+    B->>LLM: context (incl. [Browser context]) + transcript
+    LLM-->>B: tool call click_element(cp="e12", say="Opening Campaigns")
+    B->>B: guardrails: kill-switch · origin allowlist ·<br/>click is confirm-exempt → auto-run
+    B->>LK: cp_action RPC {action, target, require_confirm:false}
+    LK->>W: → executor
+    W->>W: runs click directly (no confirm bar)
+    W-->>B: {ok: true}
+    B->>U: 🔊 speaks "say" ack
+    Note over B: act-then-wait: _suppress_followup_llm →<br/>NO follow-up completion (page is changing)
+    W->>B: fresh receive_dom_summary (new page)
+    U->>B: next utterance sees the NEW [Browser context]
+```
+
+### Enablement (who turns the copilot on)
+
+```mermaid
+flowchart LR
+    CFG["agent_versions.call_config<br/>copilot_config {enabled, high_risk_confirm}"]
+    BE["backend copilot_support.py"]
+    MN["multi-node: inject 8 tools into<br/>every flow node's predefined_tools"]
+    SN["single-node: bot.py<br/>register_single_node_browser_tools"]
+    RESP["webcall response<br/>copilot_enabled: true"]
+    WID["widget activates<br/>(DOM push + cp_action handler)"]
+
+    CFG --> BE
+    BE -->|flow_config| MN
+    CFG -->|call_config forwarded| SN
+    BE --> RESP --> WID
+    MN --> BOT1["bot_with_flows.py"]
+    SN --> BOT2["bot.py"]
+```
+
+## Voice specifics
+
+- Captions: `LiveKitCaptionObserver` already publishes user STT + agent speech
+  as native LiveKit transcription for every audio call — the widget renders
+  them with the app's existing `WebcallChatUI` (`useTrackTranscription`).
+- DOM freshness: widget pushes `receive_dom_summary` on connect and on
+  debounced mutations, `receive_page_change` on SPA routes; the injector keeps
+  exactly ONE `[Browser context]` message in the LLM context.
+- Actions: the agent's `browser_actions` flow tools (attached per agent config)
+  call `cp_action`; the widget executes under the same confirm policy; the
+  runtime's `copilot_config` (allowed_origins / high_risk_confirm / disabled)
+  is the server-side guardrail.
+
+## Related: the runtime's OTHER DOM path
+
+`cp_action` (this copilot) is not the only DOM mechanism — widget authors have
+a separate one-way `execute_dom_action` path (preconfigured `action_id` →
+host-page custom event). They are complementary by design; boundary + table:
+[CONTRACTS.md → "Two DOM paths"](CONTRACTS.md#two-dom-paths--complementary-by-design-do-not-consolidate).
+
+## Security invariants
+
+1. Input **values never leave the browser** (masked flags only; server rejects
+   `value` keys).
+2. Mutating actions are **origin-allowlisted** (runtime
+   `copilot_config.allowed_origins`) and **policy-gated** on both sides.
+3. Widget activation is domain-gated (agent `whitelisted_domains` for voice;
+   CORS + Origin check for chat).
+4. Kill switches: `copilot_config.disabled` (agent) / `COPILOT_DISABLED`
+   (runtime env).
+
+## Repos & branches (beta)
+
+| repo | branch | contents |
+|---|---|---|
+| dv-pipecat-flows | `feat/browser-copilot-beta` | this package (eval/docs) + runtime `copilot_config` field |
+| dv-pipecat-flows (runtime) | `feat/browser-copilot` lineage | copilot bridge: contract, injector, RPC bridge, browser_actions |
+| dv-frontend-v2 | `feat/browser-copilot` | widget: apps/agents-cdn/src/copilot (perception/executor/hook/confirm) + packages/tests copilot e2e |

dv_pipecat_flows-0.0.23.dev10/browser_copilot/docs/CODE_MAP.md

@@ -0,0 +1,254 @@
+# CODE_MAP — the browser copilot, file by file, across four repos
+
+One agent brain (the pipecat bot) + one embeddable widget. Chat and voice are both
+LiveKit webcalls (`media_type` text|audio) of the same agent. The widget perceives the
+host page (hybrid-v2 DOM walk, `data-cp` handles), pushes a DomSummary v2 over LiveKit
+RPC; the bot keeps exactly ONE `[Browser context]` message in the LLM context; the LLM
+calls browser_actions tools; the bot RPCs `cp_action` back; the widget executes the DOM
+action and answers `{ok, result, error}`. Activation is backend-driven: the webcall
+response's `copilot_enabled` comes from the agent's `call_config.copilot_config.enabled`.
+
+Repo/branch matrix: `dv-frontend-v2 @ feat/browser-copilot` (widget),
+`fe-ringg-v2-app @ feat/browser-copilot-widget-beta` (dashboard embed),
+`new_calling_agent_backend @ feat/browser-copilot` (webcall mint),
+`dv-pipecat-flows @ feat/browser-copilot-beta` (runtime brain, `ringg-bot/`).
+
+Quick index:
+
+| Concern | File |
+|---|---|
+| Wire types/constants (TS) | `dv-frontend-v2/apps/agents-cdn/src/copilot/contract.ts` |
+| Perception (DOM walk, handles) | `…/src/copilot/perception.ts` |
+| User-event tracking | `…/src/copilot/events.ts` |
+| Action execution | `…/src/copilot/executor.ts` |
+| Widget↔bot stitch (hook) | `…/src/copilot/use-copilot.ts` |
+| Confirm UI | `…/src/copilot/ActionConfirmBar.tsx` |
+| Webcall start + Room + wiring | `…/src/App.tsx` (Room born in `AppWrapper.tsx`) |
+| Embed entry / env / harness | `…/src/main.tsx`, `…/src/lib/env-helpers.ts`, `…/copilot-test.html` |
+| Dashboard embed | `fe-ringg-v2-app/components/RinggWidgetEmbed.tsx` |
+| Copilot enablement policy | `new_calling_agent_backend/calling/copilot_support.py` |
+| Webcall mint | `new_calling_agent_backend/calling/router.py` → `calling/service.py` |
+| Webcall auth | `new_calling_agent_backend/middleware/auth_middleware.py` |
+| Wire constants (bot) + scrub | `ringg-bot/ringg_processors/copilot_contract.py` |
+| RPC bridge | `ringg-bot/ringg_processors/livekit_rpc_bridge.py` |
+| [Browser context] injector | `ringg-bot/ringg_processors/dom_context_injector.py` |
+| Tools + guardrails | `ringg-bot/utils/flow_tools/browser_actions.py` |
+| Bot wiring | `ringg-bot/bot_with_flows.py` (flows) / `ringg-bot/bot.py` (single-node) |
+| Local stack | `browser_copilot/scripts/dev-stack.sh` |
+
+## Wire diagram (files on every hop)
+
+```
+HOST PAGE (customer site / copilot-test.html / fe-ringg-v2-app dashboard)
+  └─ RinggWidgetEmbed.tsx / <script> loader ──► window.loadAgent()  [main.tsx]
+       └─ AppWrapper.tsx (new Room) ──► App.tsx
+
+App.tsx ── POST {backend}/calling/webcall (getAuthHeaders, env-helpers.ts) ──► router.py
+  router.py /webcall ── get_webcall_identity [auth_middleware.py] ──► service.py
+  service.py handle_initiate_webcall_request
+    ├─ origin_allowed_for_webcall / resolve_copilot_enabled /
+    │  build_copilot_session_config / enable_browser_copilot_tools_in_flow
+    │                                                   [copilot_support.py]
+    └─ session → bot (agent dispatch, or POST /webcall/livekit/start when
+       WEBCALL_LIVEKIT_VIA_BOT=1) ──► {user_token, call_id, copilot_enabled} ──► App.tsx
+
+App.tsx room.connect(user_token) ◄── LiveKit room ──► bot_with_flows.py / bot.py
+  (identity "user")                                     (identity "agent")
+
+PERCEPTION  widget ──► bot
+  use-copilot.ts (MutationObserver/scroll/popstate + cheapHash)
+    └─ pushSummary → buildDomSummary [perception.ts]
+       → performRpc "receive_dynamic_data" {receive_dom_summary | receive_user_event
+         [events.ts] | receive_page_change}
+       ──► livekit_rpc_bridge.py _on_receive_dynamic_data → copilot_sink
+       ──► dom_context_injector.py absorb → _sync_to_context → ONE "[Browser context]"
+
+ACTION  bot ──► widget
+  LLM tool call → browser_actions.py handler → _run (guardrails)
+    → livekit_rpc_bridge.py send_dom_action → performRpc "cp_action"
+    ──► use-copilot.ts handler (confirm gate → ActionConfirmBar.tsx)
+    ──► executor.ts runCpAction (resolveTarget → click/type/select/…)
+    ──► {ok,result,error} back ──► _run: say-ack (_speak, scrub_handles
+        [copilot_contract.py]) + _suppress_followup_llm (act-then-wait)
+
+CHAT TEXT  bot ──► widget: send_chat_text (topic "lk.chat", scrub_handles)
+```
+
+## Cross-repo sync points (change one side, check the other)
+
+- `contract.ts` ↔ `copilot_contract.py` — RPC names, component types, action kinds,
+  identities. Mapping table + invariants: `docs/CONTRACTS.md`.
+- `perception.ts:buildDomSummary()` ↔ `dom_context_injector.py:_render_summary()` —
+  the DomSummary v2 shape. All v2 keys optional; legacy `headings` kept for old injectors.
+- `copilot_support.py:BROWSER_COPILOT_ACTION_TOOLS` ↔
+  `browser_actions.py:BROWSER_ACTION_SCHEMAS` — the 9 tool names the backend injects
+  must resolve to schemas the bot registers.
+- Timeouts: widget `AUTO_DENY_MS=15s` (use-copilot.ts) must stay under the bot's
+  `send_dom_action` 20s RPC timeout (livekit_rpc_bridge.py) — a stuck confirm bar
+  auto-denies before the bot's RPC gives up.
+- Identities: widget token minted as `"user"`, bot joins as `"agent"` — hardcoded on
+  both sides (`USER_IDENTITY` in livekit_rpc_bridge.py / `AGENT_IDENTITY` in contract.ts;
+  App.tsx discovers the agent by `identity.includes("agent")`).
+
+## A. Widget — `dv-frontend-v2/apps/agents-cdn/` (the deep end)
+
+### `src/copilot/contract.ts`
+- TypeScript mirror of `ringg-bot/ringg_processors/copilot_contract.py`; kept in lock-step by hand.
+- Transport constants: `AGENT_IDENTITY="agent"`, `RPC_CP_ACTION="cp_action"`, `RPC_RECEIVE_DYNAMIC_DATA="receive_dynamic_data"`, component types `CT_DOM_SUMMARY="receive_dom_summary"`, `CT_USER_EVENT`, `CT_PAGE_CHANGE`.
+- Types: `PageState`/`ElementState` (chat-era addressable view, still feeds `find_elements`), `UserEvent`, `CpAction`/`CpTarget`/`CpResult`, `DomSummary` + `DomSummaryAction`/`DomSummaryContent` (v2: `content`, `focused`, `viewport`, `truncated`, omit-if-default flags `filled/disabled/sensitive/off/options`; legacy `headings` kept for old injectors).
+- Load-bearing redaction invariant in the types themselves: an element NEVER carries an input `value` — only `value_masked` / `sensitive` flags.
+
+### `src/copilot/perception.ts`
+- How the copilot SEES the page — production port of the benchmarked hybrid-v2 collector (`browser_copilot/eval/bench.py::COLLECTOR_JS`).
+- `CP_ATTR="data-cp"`, `UI_MARKER="data-copilot-ui"` (the widget's own subtree is never perceived); `INTERACTIVE_SEL` covers buttons/links/inputs/selects + ARIA roles incl. combobox/listbox/option.
+- `collectTagged()` — the single tagging walk: clears stale tags, then (1) interactive elements → `e1..eN` handles stamped as `data-cp`, (2) table/grid rows → one `t*` line each (cap 40), (3) headings/small text containers → `t*` lines.
+- Helpers: `roleOf` (explicit role → tag map → input-type map), `nameOf` (aria-label → label → aria-labelledby → placeholder → title → text → name, ≤90 chars), `optionsOf` (native `<select>` choices, ≤12, skips placeholders), `isSensitive` (password type, `cc-*` autocomplete, `SENSITIVE_RE` on name/id/testid), `isFilled` (drives `value_masked` — the flag, never the value).
+- `prioritize()` — budget buckets: in-viewport interactive > off-viewport interactive > in-viewport text > rest (stable within rank).
+- `collectPageState()` → `PageState` (≤300 elements + `recent_events` from events.ts) — consumed locally by `findElements`/`cheapHash`.
+- `buildDomSummary()` → `DomSummary` v2 (≤120 actions, ≤70 content lines — keeps the RPC under LiveKit's ~15KiB cap), forms grouped under their `<form>`, landmarks, `focused`, `viewport`, `truncated`. Shape MUST match `DomContextInjector._render_summary`.
+- `resolveById(id)` — live `[data-cp="…"]` lookup; the handle lives in the DOM, so resolution is always fresh.
+
+### `src/copilot/events.ts`
+- Behavioral perception: what the user is DOING (structure says what exists). Pure module (no React) — shared by the hook and the e2e bundle.
+- Ring buffer of 10 `UserEvent`s (oldest→newest), consumed by `collectPageState()`; `onUserEvent(cb)` push-subscribes (the voice session forwards each event to the bot).
+- `startEventTracking()` wires capture-phase listeners: `focusin` (focus), `input` (typing, 400ms debounce — records `text_len` ONLY, never the text), `selectionchange` (300ms debounce, the one kind that carries text: visible page content the user highlighted, ≤200 chars), `click`. Everything inside `[data-copilot-ui]` is ignored; targets resolve to the nearest `data-cp` ancestor via `targetIdOf`.
+- `stopEventTracking()` removes listeners and clears the ring.
+
+### `src/copilot/executor.ts`
+- The agent's hands: `runCpAction(action, ctx)` runs a parsed `CpAction`, always resolving to `{ok, result, error}` (never throws).
+- `resolveTarget()` chain: explicit `targetId` → `target.cp` → `target.ref` (shares the cp namespace; the bot no longer advertises `ref` but stray ones still resolve) → CSS `selector` → visible `text` (substring match over tagged + button/link candidates) → `role`(+name).
+- `normalizeCp()` maps the wire action to an op; `click`/`type`/`select`/`highlight`/`read_region` require a resolved target ("target not found" otherwise); `scroll` only requires one when the bot asked for an element scroll — else `scrollPage(direction, amount)` (default step 0.8×viewport). `screenshot` → "unsupported in beta widget".
+- `select`'s value comes from `params.value` (the bot's `select_option.value` schema), falling back to `params.text`.
+- `typeInto`/`setNativeValue`: writes via the native value setter + dispatched `input`/`change` events so React controlled inputs actually update (a plain `.value=` is ignored by React).
+- `selectOption()` (async): native `<select>` by value/text; else custom dropdown — click the trigger (`[role=combobox]`/`[aria-haspopup]`/button), then `waitForOption(want, 1200ms)` polls the WHOLE document every 40ms (Radix/shadcn portal the listbox to `<body>`), clicks the match, or re-clicks the trigger to close if none found.
+- `findElements(query, role, limit)` — query-scored (exact id/name=3, contains=2), capped ≤50, returns `{matches:[{cp,role,name}], total_considered}`; never the full page state (RPC size). `read_dom` returns `buildDomSummary()`; `read_region` returns the target's `innerText` ≤2000 chars; `highlight` = smooth scroll + 3px outline for 2s.
+
+### `src/copilot/use-copilot.ts` — THE stitch point
+- `useCopilot({room, agentIdentity, enabled, hostEl, onLog})` — App owns the Room; this hook adds the copilot on top when `enabled` (webcall said `copilot_enabled` && call connected). Stamps `hostEl` with `UI_MARKER` on mount, removes on teardown.
+- Bot→widget: registers `room.localParticipant.registerRpcMethod("cp_action", handler)`. Handler parses the `CpAction`; if `params.require_confirm` → `askConfirm()` surfaces a `pending` (rendered by ActionConfirmBar), auto-DENIES after `AUTO_DENY_MS=15s` (beats the bot's 20s RPC timeout); denial returns `{ok:false, error:"user denied"}`.
+- Otherwise → `runCpAction()`; the result JSON is the RPC response. Confirmation POLICY is the bot's (the widget just honors the flag; the confirm label is `target.text ?? target.cp ?? action`). Every action is logged to the widget's SystemLog via `onLog`.
+- Widget→bot: `pushSummary()` — `buildDomSummary()`; if the URL changed since the last push, first sends `CT_PAGE_CHANGE {url,title}`, then `CT_DOM_SUMMARY` via `performRpc("receive_dynamic_data")` to the agent identity. Guards `room.state === Connected`. `pushSummaryWithRetry()` — up to 6 attempts, backoff 700ms×n, for the agent-connect race (the bot registers its handler a beat after `ParticipantConnected`).
+- Perception loop: debounced (400ms) MutationObserver on `document.body` (childList + subtree + attrs `aria-label/disabled/aria-checked/value/hidden`), ignoring its own `data-cp` churn and the widget subtree; `scroll`/`resize` throttled 500ms; `popstate`/`hashchange` for pure history moves (pushState apps also mutate the DOM, which the observer catches).
+- `notify()` dedupes with `cheapHash()` = `route|elementCount|firstId|lastId|focused|scroll_y` before pushing — cheap enough to run on every debounce tick.
+- First summary fires when the agent participant appears (`RoomEvent.ParticipantConnected`, or immediately if already present). `onUserEvent` subscription forwards each tracked event as `CT_USER_EVENT` (mapped by `toRuntimeEvent`: `target_id`→`target_ref`, typing carries `text_len` only; fire-and-forget).
+- Teardown: unregister RPC, disconnect observer/listeners, stop tracking, resolve any pending confirm with `false` (never leave the bot's RPC hanging), reset hash/url.
+
+### `src/copilot/ActionConfirmBar.tsx`
+- One-tap confirm for a `require_confirm` action; themed via `useWidgetTheme`, inline-styled (embeds in arbitrary host pages).
+- Verb map (`click`→"Click", `type`→"Type into", …); for `type` shows the text to be typed, else the target label. Buttons carry `data-ringg="copilot-confirm-approve"` / `"copilot-confirm-deny"`; the bar is `data-ringg="copilot-confirm-bar"`.
+- Purely presentational — approve/deny call the hook's `confirm()`/`deny()`, which resolve the pending promise inside the `cp_action` RPC handler.
+
+### `src/App.tsx` (copilot wiring only)
+- Owns the webcall + Room lifecycle (Room itself is created in `AppWrapper.tsx` and provided via `RoomContext`). `handleCallStart(mediaType)` → `fetch(`${getBackendUrl(mode)}/calling/webcall`)` with `{agent_id, custom_args_values, media_type}` and `getAuthHeaders(authorization, xApiKey)`; then `setCopilotEnabled(data?.copilot_enabled === true)` and `room.connect(getLivekitUrl(mode), data.user_token)` (mic enabled for audio).
+- `agentParticipant` = first remote participant whose identity includes `"agent"`; `copilotHostEl` = `#ringg_ai_container` (the hook stamps it `data-copilot-ui`).
+- `useCopilot({room, agentIdentity: agentParticipant?.identity, enabled: copilotEnabled && isCalling, hostEl, onLog: pushSystemLog})` → `{pending, confirm, deny}`.
+- Renders the confirm overlay when `pending`: fixed, bottom 96 / right 24, `zIndex 2147483000`, wrapping `<ActionConfirmBar/>` (`data-ringg="copilot-confirm-overlay"`).
+- `handleCallEnd()` resets `copilotEnabled` to false (the hook effect tears everything down).
+
+### `src/main.tsx`
+- The embed entry. Dev mode: auto-creates `#ringg_ai_container` and mounts `AppWrapper` with `DEV_APP_CONFIG` (env `VITE_X_API_KEY` / `VITE_DEV_AGENT_ID`).
+- Prod bundle: exposes `window.loadAgent(config)` — reuses an existing `#ringg_ai_container` (integrator-positioned) or appends one to `<body>`, then renders `AppWrapper config`.
+- Also injects the brand `@font-face` by resolving the woff2 next to the loaded `dv-agent.es.js` (classic-script-safe; no `import.meta`).
+
+### `src/lib/env-helpers.ts`
+- `getBackendUrl(mode)` / `getLivekitUrl(mode)`: `dev|stage|prod` → `VITE_BACKEND_URL_{DEV,STAGE,PROD}` / `VITE_LIVEKIT_SERVER_URL_*` (baked at build time).
+- `getAuthHeaders(authorization, xApiKey)`: exactly one header — `Authorization` wins over `X-API-KEY`; dev builds fall back to `VITE_AUTHORIZATION` / `VITE_X_API_KEY`.
+
+### `copilot-test.html`
+- Standalone host-page harness ("Acme Corp"): loads the LOCAL build (`./dist/dv-agent.es.js` + `style.css`), then `window.loadAgent({mode:"dev", agentId, xApiKey, defaultTab:"audio"})` (agent id + workspace key via inputs or `?agentId=…&key=…`).
+- Test surface: a sign-in form (email, password → flagged `sensitive`, plan `<select>`), and Subscribe / Contact sales / Delete account buttons wired to visible result text — demonstrates click auto-run vs the type-only confirm policy.
+
+### e2e — `packages/tests/tests/copilot/` + `packages/tests/copilot.playwright.config.ts`
+- `bundle.ts` — `buildCopilotBundle()` concatenates the REAL `contract/perception/events/executor` sources, strips module syntax, transpiles with the TypeScript compiler API, and exposes `window.__cp` — specs exercise shipped code, not a re-implementation.
+- 3 specs: `copilot-perception.spec.ts` (grounding/roles/ids resolve), `copilot-events.spec.ts` (typed text NEVER captured — length only), `copilot-summary-v2.spec.ts` (v2 content with resolvable `t*` handles).
+- Fixtures: static pages at `packages/tests/tests/fixtures/copilot-pages/{checkout,dashboard,signup}.html`, loaded over `file://` — no app server, no auth (`copilot.playwright.config.ts` deliberately omits the root config's webServer). Run: `pnpm --filter tests test:copilot`.
+
+## B. Host-page embed — `fe-ringg-v2-app` (@ feat/browser-copilot-widget-beta)
+
+### `components/RinggWidgetEmbed.tsx`
+- Embeds the agents-cdn widget into the dashboard EXACTLY like a customer site: appends `style.css` + `dv-agent.es.js` from `NEXT_PUBLIC_RINGG_WIDGET_URL` (default `/ringg-widget`, synced from the widget build), then `window.loadAgent({mode, agentId, xApiKey | authorization, defaultTab:"audio"})`.
+- Env-gated: `NEXT_PUBLIC_RINGG_WIDGET_ENABLED`, `_AGENT_ID`, `_API_KEY`, `_MODE`. Auth: workspace `X-API-KEY` if set, else the logged-in user's `Bearer` token (`useAuth`). Renders `null`; mounted once in `app/layout.tsx` (`<RinggWidgetEmbed />`).
+- All copilot brain/UI live in the widget — this file is ONLY the embed; activation stays backend-driven (`copilot_enabled`).
+
+## C. Backend — `new_calling_agent_backend` (@ feat/browser-copilot)
+
+### `calling/copilot_support.py`
+- `extract_copilot_config(agent_version)` — reads the RAW `call_config` dict (not the pydantic model, so unknown-key stripping elsewhere can't eat it) → `copilot_config` or `{}`.
+- `resolve_copilot_enabled()` — per-AGENT capability: `call_config.copilot_config.enabled == true` (no workspace switch). `build_copilot_session_config()` → `{enabled, high_risk_confirm}` handed to the bot session.
+- `origin_allowed_for_webcall(origin, whitelist)` — same rule for pages and extensions: origin must be in the agent's `whitelisted_domains` (no `chrome-extension://` bypass).
+- `BROWSER_COPILOT_ACTION_TOOLS` — the 8 canonical names (read_dom, find_elements, read_region, click_element, type_text, select_option, scroll_page, highlight_element), sync'd with the bot's `BROWSER_ACTION_SCHEMAS`. `enable_browser_copilot_tools_in_flow(flow_config)` appends `{"name": …}` to every flow node's `predefined_tools` (in place, idempotent).
+
+### webcall flow — `calling/router.py` → `calling/service.py`
+- `router.py` `POST /webcall` (router prefix `/calling`, mounted under `/ca/api/v0` in `main.py`) — requires an `Origin` header, authenticates via `Depends(get_webcall_identity)`, delegates to `handle_initiate_webcall_request(payload, current_user, db, origin)` in `service.py`.
+- `service.py`: agent + version lookup → `origin_allowed_for_webcall` (403 if not whitelisted) → `initiate_call_helper`. In the `runtime == "livekit"` branch: `copilot_enabled = resolve_copilot_enabled(agent_version)`; `build_copilot_session_config` is written into the session's `call_config.copilot_config`; when enabled, `enable_browser_copilot_tools_in_flow(flow_config)` expands the flag into concrete tools on every node.
+- Session delivery: `WEBCALL_LIVEKIT_VIA_BOT=1` (local dev) proxies the fully-assembled session to the bot's `POST {PIPECAT_URL}/webcall/livekit/start` (bot creates the room + mints tokens); otherwise `create_explicit_agent_dispatch` + a locally-minted user token. Response: `{user_token, call_id, enabled_slash_commands, copilot_enabled}`. Every other runtime branch hard-sets `copilot_enabled: False` (copilot is LiveKit-webcall-only).
+
+### `middleware/auth_middleware.py`
+- `get_webcall_identity` → `resolve_webcall_identity`: (1) `Authorization: Bearer pk_live_*` — the per-agent webcall public key, binds to exactly one agent (`auth_method="webcall_pk"`; can't reach admin endpoints, can't set callbacks); (2) user JWT via header or HttpOnly cookie (dashboard test calls); (3) deprecated fallback: `X-API-KEY` → `fetch_user_from_db` → `Workspace.api_key` lookup (`auth_method="api_key"`, logs a deprecation warning). Fails closed.
+
+## D. Runtime brain — `dv-pipecat-flows/ringg-bot/` (@ feat/browser-copilot-beta)
+
+### `ringg_processors/copilot_contract.py`
+- Bot-side constants: identities `AGENT_IDENTITY="agent"` / `USER_IDENTITY="user"`; `RPC_CP_ACTION`; inbound component types (`CT_DOM_SUMMARY`, `CT_USER_EVENT`, `CT_PAGE_CHANGE`, plus legacy `CT_DOM_SNAPSHOT`/`CT_SCREENSHOT`) collected in `COPILOT_INBOUND_TYPES`; the `ACTION_*` names; `MUTATING_ACTIONS = {click, type, select, scroll}`.
+- `scrub_handles(text)` — TTS/chat hygiene: strips `[cp:e48]`-style and bare `e48/t11/f3` tokens from user-facing text, tidies the gaps, never returns empty. Backstop for the paths we control (chat relay + spoken acks); the streamed voice narration relies on the prompt.
+
+### `ringg_processors/livekit_rpc_bridge.py`
+- `LiveKitRPCBridge(room, task, on_call_tools_config, bot_logger, copilot_sink)` — owns the LiveKit RPC surface for one webcall; node-agnostic (both bots).
+- `register_handlers()` registers `receive_dynamic_data` on `room.local_participant`. `_on_receive_dynamic_data`: copilot component types → `copilot_sink(component_type, component_data)` WITHOUT injecting a user turn (ambient observations); widget interactions (calendar/form/quick-reply/blocks) → tool API + `_inject_user_turn` (`LLMMessagesAppendFrame(run_llm=True)`).
+- `send_dom_action(action, params, timeout=20s)` — `perform_rpc(destination="user", method="cp_action")`; logs the full round-trip latency on every path (`grep cp_action`); ALWAYS returns `{ok, result, error}` (failures/timeouts become `ok:False`, never an exception).
+- `send_chat_text(text)` — assistant replies over the `lk.chat` text stream (text-mode webcalls), through `scrub_handles`. Plus non-copilot senders: `send_dynamic_data`, `send_blocks_payload` (`ringg.blocks` stream), `send_source_urls`, `publish_caption`.
+
+### `ringg_processors/dom_context_injector.py`
+- Absorbs copilot perception and keeps EXACTLY ONE `[Browser context]` user message in the LLM context (marker `_CTX_MARKER`); not a FrameProcessor — input arrives out-of-band over RPC, so it mutates the shared context via the user aggregator (`set_flow_manager` for flows / `set_context_aggregator` for single-node).
+- `absorb(component_type, data)`: `CT_DOM_SUMMARY` → store + `_sync_to_context`; `CT_PAGE_CHANGE` → keep url/title, DROP stale summary; `CT_USER_EVENT` → `_update_activity` ("typing in element e9 (12 chars)" — selection changes re-sync); legacy `CT_DOM_SNAPSHOT` supported.
+- `_render_summary()` — v2 renderer: Viewport line (px + scroll %), `Focused: [cp:…]`, on-screen actions (`[cp:e7] button "Subscribe" [filled] [sensitive] (disabled)` + `— options: …` suffix for dropdowns), "Off-screen (scroll_page to reach)" (≤20), Forms, "Page content" (`# heading` / `row:` / text lines with `[cp:tN]`, ≤60), Regions, legacy flat headings only when no v2 content, and the `interactive_count` + `find_elements` hint (+ "(list truncated)").
+- Legacy (pre-v2) summaries render byte-identical to before (`_action_flags` is empty for them); a raw input `value` key is never rendered on the summary path.
+- `_build_message_text()` = marker + URL/Title + summary + "User is currently: …"; hard tail-truncate at 12,000 chars (hostile summaries can't blow the context). `_sync_to_context()` filters old `[Browser context]` messages and appends the fresh one.
+- Also holds `_url` — read by `browser_actions._current_url` for the mutating-action origin guardrail.
+- Auto-continue chain: `arm_continue` (mutating action) → next LLM step fires when the fresh summary lands (`_maybe_continue`, capped `MAX_CHAIN`). Stale-summary watchdog (`CONTINUE_FALLBACK_S=3.5`): if no summary lands (a DOM change the digest can't see, e.g. an option toggle), fire the continue anyway with a `[Browser context]` re-check note (`_stale_note`) so the chain can't starve. `_action_flags` renders `[checked]`/`[unchecked]` (selection state) alongside `[filled]`/`[sensitive]`; long `— options: …` lists truncate with `…`.
+
+### `utils/flow_tools/browser_actions.py`
+- The LLM-callable hands: 9 handlers (`read_dom`, `find_elements`, `read_region`, `click_element`, `type_text`, `select_option`, `scroll_page`, `highlight_element`, `execute_plan`) + their `FlowsFunctionSchema`s in `BROWSER_ACTION_SCHEMAS`. Targeting: `cp` handle is the advertised, preferred key (`ref` deliberately NOT advertised — one handle namespace; strays still forwarded); `selector`/`text`/`role` fallbacks. Mutating schemas take an optional `say` ack.
+- `_run_step(flow_manager, action, params) -> (ok, body)` — the shared guardrails, in order: bridge present (`state["livekit_rpc"]`) → kill-switch `copilot_config.disabled` → for `MUTATING_ACTIONS`, `_origin_allowed(injector._url, cfg.allowed_origins)` (empty allowlist = allow all) → confirm policy → `bridge.send_dom_action`. `_run(...)` is the single-action wrapper (adds `say`, suppression, `arm_continue`).
+- `execute_plan({steps, say})` — one LLM call runs an ordered batch of steps on the CURRENT view via `_run_step` each (`MAX_PLAN_STEPS=8`, `PLAN_BATCH_BUDGET_S=60`), stops at the first failure/denial, then issues one `read_dom` and folds the fresh page into `[Browser context]` (`injector.absorb(CT_DOM_SUMMARY, …)`), returning ActionResult-v1-shaped `results` + a compact `page` digest. NO suppression sentinel / no `arm_continue` → the follow-up LLM runs and plans the next batch off the real page. `disarm_continue()`s at start (a batch supersedes the passive chain). Confirm-eligible steps use the blocking confirm bar mid-batch.
+- Confirm policy: `_CONFIRM_EXEMPT_ACTIONS = {click, scroll, select}` ALWAYS auto-run (product decision), even destructive-named, even under `high_risk_confirm`. Only `type` can confirm: `require_confirm=True` when `high_risk_confirm` OR `_is_destructive`.
+- `_is_destructive`: `_target_name` from `target.text`, else the cp handle looked up in the injector's live summary (actions/content/form fields); matched against the `_DESTRUCTIVE` lexicon (submit/pay/delete/cancel subscription/publish/…); fail-open when no name resolves (destructive intents are named explicitly).
+- Act-then-wait: successful MUTATING actions return `_suppress_followup_llm: True` (flows manager → `run_llm=False`; single-node adapter translates) — the fresh `[Browser context]` arrives async, so an immediate completion could only act on the STALE page. The optional `say` ack is spoken out-of-band via `_speak` (scrubbed; `state["_speak"]` shim or `task.queue_frame(TTSSpeakFrame)`) so the bot isn't silent. Failures never suppress.
+- Single-node adapter: `resolve_copilot_config(call_config)` (env fallbacks `COPILOT_ALLOWED_ORIGINS` / `COPILOT_HIGH_RISK_CONFIRM` / `COPILOT_DISABLED`); `register_single_node_browser_tools(...)` registers the same 9 schemas on bot.py when `copilot_config.enabled`, wrapping each flows handler in a `SimpleNamespace(state=…)` shim — same `_run` guardrails, zero duplication; suppression becomes `FunctionCallResultProperties(run_llm=False)`.
+
+### bot wiring (copilot-relevant only)
+- `bot_with_flows.py` — LiveKit channel, `on_connected`: builds `DomContextInjector` (+`set_flow_manager`), resolves `copilot_cfg` from `call_config.copilot_config` with the same env fallbacks, then `LiveKitRPCBridge(room, task, …, copilot_sink=dom_injector.absorb)` + `register_handlers()`; stores `dom_injector` / `copilot_config` / `livekit_rpc` in `flow_manager.state` (read by `_run`). Tools arrive pre-injected in the flow nodes (backend's `enable_browser_copilot_tools_in_flow`) and resolve to the `BROWSER_ACTION_SCHEMAS` handlers. Text mode: `transcript_handler.on_message_added = _relay_chat_reply` → `bridge.send_chat_text` (the widget's chat rendering path).
+- `bot.py` — single-node: `copilot_state_ref: dict` created before tool build; `register_single_node_browser_tools(call_config=…, state_ref=copilot_state_ref, …)` at tool-registration time (LiveKit channel only); `on_connected` fills the ref with `livekit_rpc` + `dom_injector` (`DomContextInjector.set_context_aggregator(context_aggregator.user())`) — handlers read them lazily through the shim.
+
+### tests
+- `tests/test_browser_action_suppression.py` — act-then-wait (mutating success suppresses; reads/highlight/failures never), kill-switch, origin allowlist, click/scroll/select NEVER confirm, destructive `type` confirms by default, `high_risk_confirm` forwarding, `scrub_handles` + say-ack scrubbing, destructive-name lookup via the injector summary, single-node registration (9 tools; skipped when disabled) and adapter `run_llm` translation.
+- `tests/test_execute_plan.py` — ordered batch runs then re-perceives, stop-on-failure / denied-confirm / invalid-step, per-step origin & confirm guardrails, kill-switch skips read_dom, `MAX_PLAN_STEPS` truncation, one scrubbed `say`, read_dom-failure `page_error`, single-node adapter runs the LLM (no suppression), `_step_to_action` mapping.
+- `tests/test_dom_context_injector.py` — legacy summary renders golden-identical, v2 flags/content/focused/viewport, dropdown options line + long-list truncation, `[checked]`/`[unchecked]` state, hostile `value` key never rendered, content/message caps, activity line, auto-continue chain + stale-summary watchdog (fires on no-summary, cancelled by summary/disarm, one-shot, respects `MAX_CHAIN`).
+
+## E. Ops
+
+### `browser_copilot/scripts/dev-stack.sh`
+- One-shot local stack, `up|down|status|logs [svc]` — 4 services: **backend** (new_calling_agent_backend, :8082, `uv run python main.py` against the dockerized deps), **runtime** (ringg-bot via `ringg-bot/.venv/bin/python server.py`, :8765), **ngrok** (tunnels :8765 to the reserved `PIPECAT_URL` domain so the backend reaches the local bot), **frontend** (the host page with the embedded widget: builds the agents-cdn bundle from dv-frontend-v2, syncs it via `sync:widget`, serves on :3000).
+- Preflight: branch matrix warning (the 5 expected branches), Docker/ngrok present, Krisp model is real (not an LFS pointer stub — deaf-bot trap), backend `WEBCALL_LIVEKIT_VIA_BOT=1`, widget/app env vars. Logs + pids in `/tmp/copilot-stack`.
+
+### `fe-ringg-v2-app/package.json` → `sync:widget`
+- `rm -rf public/ringg-widget && mkdir -p … && cp -r ../dv-frontend-v2/apps/agents-cdn/dist/. public/ringg-widget/` — copies the built widget bundle into the app's public assets, where `RinggWidgetEmbed` loads it from.
+
+## Trace 1 — voice: "click Subscribe"
+
+1. Mic → LiveKit room (widget joined as `user` via `App.tsx handleCallStart`; bot as `agent`) → pipecat STT transcribes the utterance.
+2. The LLM context already holds one `[Browser context]` message (from `dom_context_injector.py:_sync_to_context`) listing `[cp:e7] button "Subscribe"`. The LLM calls `click_element({cp:"e7", say:"Done — subscribed."})`.
+3. `browser_actions.py:click_element` → `_run(ACTION_CLICK, {target:{cp:"e7"}}, say)`: bridge present; kill-switch off; click is mutating → origin check `_origin_allowed(injector._url, copilot_config.allowed_origins)`; click ∈ `_CONFIRM_EXEMPT_ACTIONS` → never `require_confirm`.
+4. `livekit_rpc_bridge.py:send_dom_action("click", …)` → `perform_rpc(destination="user", method="cp_action", timeout=20s)`, latency-timed.
+5. Widget: the `use-copilot.ts` `cp_action` handler parses the `CpAction`; no `require_confirm` → straight to `executor.ts:runCpAction`.
+6. `executor.ts:normalizeCp` → `resolveTarget` → `perception.ts:resolveById("e7")` → op `click` → `highlight(el)` + `el.click()` → handler returns `JSON.stringify({ok:true, result:null, error:null})` as the RPC response (and logs "Co-pilot: click" via `onLog`).
+7. Bot: `send_dom_action` parses `{ok:true}` → `_run` success path: `_speak(flow_manager, "Done — subscribed.")` (through `copilot_contract.py:scrub_handles`, as a `TTSSpeakFrame` / single-node `params.llm.push_frame`) then returns the result with `_suppress_followup_llm: True` → NO follow-up LLM completion. The turn ends (act-then-wait).
+8. The click changed the page → widget `MutationObserver` fires → `schedule()` (400ms debounce) → `cheapHash()` differs → `pushSummary()` → `receive_dynamic_data(CT_DOM_SUMMARY)` → `LiveKitRPCBridge._on_receive_dynamic_data` → `DomContextInjector.absorb` → the ONE `[Browser context]` message is replaced. The user's next turn reasons over the FRESH page.
+
+## Trace 2 — the page changes
+
+1. The host app re-renders (SPA navigation, dialog opens, row added) → `use-copilot.ts` MutationObserver callback; records caused by our own `data-cp` tagging or inside `[data-copilot-ui]` are skipped → `schedule()` (400ms debounce). Scroll/resize (500ms throttle) and `popstate`/`hashchange` feed the same `schedule()`.
+2. `notify()` → `cheapHash()` (runs `perception.ts:collectPageState`) = `route_key|count|firstId|lastId|focused|scroll_y`; identical to the last push → dropped; different → `pushSummary()`.
+3. `pushSummary()` → `perception.ts:buildDomSummary()` (fresh tagging walk, new dense `e*/t*` ids). If `summary.url` differs from the last push it first RPCs `CT_PAGE_CHANGE {url,title}`; then `performRpc("receive_dynamic_data", {component_type:"receive_dom_summary", component_data: summary})` to the agent identity.
+4. Bot: `livekit_rpc_bridge.py:_on_receive_dynamic_data` → component_type ∈ `COPILOT_INBOUND_TYPES` → `copilot_sink` (NO user turn injected — ambient observation) → returns `"ok"`.
+5. `dom_context_injector.py:absorb(CT_DOM_SUMMARY)` stores the summary (a preceding `CT_PAGE_CHANGE` had dropped the stale one) → `_sync_to_context()` → `_build_message_text()` (`[Browser context]` + URL/Title + `_render_summary()` + activity line, ≤12k chars) → replaces the single copilot message in the shared `OpenAILLMContext`. The next LLM completion sees the new page.