@deadragdoll/tellymcp 0.0.12 → 0.0.14

package/TOOLS.md

@@ -1,6 +1,6 @@
 # Tools
 
-Version: `2026-05-22.1`
+Version: `2026-06-06.2`
 
 Gateway/client runtime compatibility:
 
@@ -16,10 +16,19 @@ This MCP server currently exposes the following tools.
 
 Browser tools:
 
+- `browser_list_attached_instances`
+- `browser_list_tabs`
+- `browser_attach_active_tab`
+- `browser_attach_tab`
+- `browser_detach_tab`
+- `browser_recording_start`
+- `browser_recording_stop`
+- `browser_recording_status`
 - `browser_open`
 - `browser_reload`
 - `browser_click`
 - `browser_fill`
+- `browser_inject_script`
 - `browser_press`
 - `browser_wait_for`
 - `browser_wait_for_url`
@@ -34,7 +43,17 @@ Browser tools:
 
 Browser runtime rule:
 
-- Browser tools use Playwright Chromium.
+- Browser tools now have two backends:
+  - attached browser tab backend for a real user Firefox or Chrome session
+  - isolated Playwright Chromium backend
+- `browser_open` is only for the isolated Playwright backend.
+- `browser_list_attached_instances`, `browser_list_tabs`, `browser_attach_active_tab`, `browser_attach_tab`, and `browser_detach_tab` are for the attached browser backend.
+- after a tab is attached for the current session, `browser_click`, `browser_fill`, `browser_inject_script`, `browser_press`, `browser_wait_for`, `browser_wait_for_url`, `browser_dom`, `browser_computed_style`, `browser_reload`, `browser_screenshot`, `browser_console`, `browser_errors`, `browser_network_failures`, `browser_clear_logs`, and `browser_close` use that attached browser tab without requiring `browser_open`
+- `browser_recording_start`, `browser_recording_stop`, and `browser_recording_status` are also for the attached browser backend
+- browser recordings are written under `.mcp-xchange/web/{tab-title-slug}-{timestamp}/`
+- each recording bundle contains `session.json`, `timeline.ndjson`, `pages/`, `network/`, and `console/`
+- inside `network/`, use `index.ndjson` as the thin search index and `requests/{seq}-{request_id}/` as the per-request artifact bundle
+- otherwise those tools use the Playwright backend after `browser_open`
 - If browser tools fail because the Playwright browser runtime is missing, install it with `tellymcp browser install`.
 - Do not stop at the installation error itself. Install the browser runtime first, then retry the browser tool.
 
@@ -64,13 +83,16 @@ Collaboration tools:
 
 Utility tools:
 
+- `get_file_list`
+- `get_file`
 - `refresh_tools_markdown`
 
 ## Console model
 
 Gateway-first runtime model:
 
-- agents register themselves on the gateway automatically through `GATEWAY_TOKEN`
+- agents authenticate the gateway HTTP/WS transport through `GATEWAY_AUTH_TOKEN`
+  and register into their gateway scope through `GATEWAY_TOKEN`
 - Telegram no longer links sessions with pair codes
 - `/menu` in the gateway bot shows available remote consoles directly
 - one running agent console is one logical session/console target
@@ -95,7 +117,18 @@ Preferred order for cross-console work:
    - `send_partner_note`
    - `send_partner_file`
    - browser tools
-   with explicit `session_id` for the current console or explicit target routing fields for another console.
+     with explicit `session_id` for the current console or explicit target routing fields for another console.
+
+Preferred file-return flow for MCP chat clients:
+
+1. Resolve the selected live console and keep its canonical `session_id`.
+2. If the user refers to an existing managed file without an exact path, call `get_file_list` and select the matching `file_path`.
+3. If creating a screenshot now, call `browser_screenshot` and keep its returned `file_path`.
+4. For an exact Markdown, source-code, or other UTF-8 project path, call `get_file(type = "text")`; arbitrary project files are not discovered by `get_file_list`.
+5. For binary files, call `get_file` with the selected or newly returned `file_path`; default to `type = "url"`.
+6. If the chat cannot fetch an image URL, repeat the same call with `type = "image"` for a native inline MCP image block.
+7. If a host such as Claude.ai replaces the native block with `[image]`, retry with `type = "base64"`; this host-independent fallback returns the complete JSON, including raw `data`, in a regular MCP text block.
+8. If the user asks for the most recent existing screenshot, `get_file(selector = "latest_screenshot")` is the direct shortcut.
 
 ## `set_session_context`
 
@@ -174,6 +207,99 @@ Output:
 - `terminal?`
   Terminal runtime metadata for the console.
 
+## `get_file_list`
+
+Purpose:
+
+- List managed files available in a selected console workspace before calling `get_file`.
+- Return exact `file_path` values accepted by `get_file`.
+
+Input:
+
+- `session_id`
+- `source?`
+  - `telegram-upload`
+  - `browser-screenshot`
+  - `partner-artifact`
+- `limit?`
+  - default: `50`
+  - maximum: `200`
+
+Output:
+
+- `total`
+- `files[]`
+  - `file_path`
+  - `filename`
+  - `mimetype`
+  - `source`
+  - `size_bytes?`
+  - `created_at`
+
+Behavior:
+
+- results are ordered newest first
+- `total` is the number of matching managed files before `limit` is applied
+- this tool lists TellyMCP-managed Telegram uploads, browser screenshots, and partner artifacts; it does not recursively enumerate arbitrary source files in the workspace
+- in gateway mode, always pass the canonical `session_id = client_uuid:local_session_id`
+- pass the chosen `files[].file_path` unchanged to `get_file`
+
+## `get_file`
+
+Purpose:
+
+- Return the actual content of a file from a selected console workspace to an MCP chat client.
+- URL mode creates a bounded temporary gateway copy; text and base64 modes relay content directly.
+
+Input:
+
+- `session_id`
+- `type?`
+  - `url` (default)
+  - `image` (native inline image plus download URL)
+  - `text` (native UTF-8 MCP text content)
+  - `base64` (fallback)
+- exactly one of:
+  - `file_path`
+  - `selector = "latest_screenshot"`
+
+Output:
+
+- `type`
+- `data`
+  - short-lived HTTPS URL when `type = "url"`
+  - short-lived HTTPS URL when `type = "image"`; the actual pixels are in the native MCP image content block
+  - decoded UTF-8 file content when `type = "text"`
+  - base64 payload when `type = "base64"`
+- `mimetype`
+- `filename`
+- `size_bytes`
+- `expires_at?`
+
+Behavior:
+
+- in gateway mode, always pass the canonical `session_id = client_uuid:local_session_id` returned by `list_gateway_sessions`
+- `file_path` may be the exact path returned by `browser_screenshot`, so the human does not need to know generated filenames
+- `selector = "latest_screenshot"` selects the newest file metadata with `source = "browser-screenshot"` for that session
+- file reads are limited to the selected console workspace, including after symlink resolution
+- exact project-relative paths such as `README.md` or `src/test.ts` are accepted even though `get_file_list` only discovers managed artifacts
+- text mode returns the decoded file directly as a native MCP text content block and rejects invalid UTF-8 or binary NUL content
+- source extensions such as `.ts` and `.tsx` use text MIME overrides instead of the generic MPEG/octet-stream mappings
+- sensitive paths including live `.env` files, credential stores, private-key extensions, and secret directories are rejected server-side; documented `.env.example`/`.env.sample`/`.env.template` files remain readable
+- URL mode uploads the file as a binary stream to `.tellymcp/tmp/file-links` on the gateway and returns a link valid for 10 minutes
+- image mode is explicit: `structuredContent.type` is `image`, `structuredContent.data` contains the URL, and the native top-level MCP `image` is the first content block without duplicating base64 into structured output
+- image mode accepts only `image/*` files up to the safe native-inline limit; use `url` for larger images
+- URL links allow up to three GET downloads; HEAD requests do not consume the limit
+- if URL fetching fails for an image, retry with `type = "image"`
+- base64 mode is the compatibility fallback for hosts that replace native images with `[image]` or hide `structuredContent`: it returns the complete JSON payload, including raw base64 in `data`, as a regular MCP text block and does not emit a native image
+- base64 mode is bounded by the final MCP response body limit
+- do not attempt to bypass the sensitive-file policy or retrieve renamed secrets
+
+Screenshot examples:
+
+- "take a screenshot and send it": call `browser_screenshot`, then pass its returned `file_path` to `get_file`
+- "send the latest screenshot": call `get_file` with `selector = "latest_screenshot"`
+
 ## `refresh_tools_markdown`
 
 Purpose:
@@ -431,7 +557,7 @@ Required agent practice:
   - "tell frontend what changed"
   - "send the error to your teammate"
   - "tell the other agent what's new"
-  then the correct path is `send_partner_note`
+    then the correct path is `send_partner_note`
 
 How to find the partner correctly:
 
@@ -491,7 +617,7 @@ Reply rule for project asks:
   - `Reply project_uuid: ...`
   - `Reply target_client_uuid: ...`
   - `Reply target_local_session_id: ...`
-  or a `Reply Params` section with the same data
+    or a `Reply Params` section with the same data
 - then reply with those exact values
 - if `in_reply_to` is available, prefer `Reply message_uuid`
 - if only the note `share_id` is available, gateway now also accepts that value in `in_reply_to`
@@ -592,7 +718,7 @@ How the receiving agent must react:
   - `проверь xchange records`
   - `telegram_message`
   - `partner_note`
-  then do not start with a legacy inbox-specific tool
+    then do not start with a legacy inbox-specific tool
 - instead:
   1. call `list_xchange_records`
   2. identify the newest relevant record by category:
@@ -758,7 +884,8 @@ Output:
 Notes:
 
 - each session gets its own isolated browser context and page
-- call this first before reading console, DOM, styles, or screenshots
+- call this first only when you want the isolated Playwright backend
+- do not call this just to work with an already selected Firefox attached tab
 - `url` may be an absolute URL, or a relative path when `BROWSER_ADDRESS` is configured
 - in headed mode the browser window is started maximized by default
 - in headed mode a fresh browser context is created without a fixed viewport unless you explicitly pass `width` and `height`
@@ -766,6 +893,206 @@ Notes:
 - if you pass viewport size, pass both `width` and `height` together
 - use explicit `width` and `height` when the task depends on a specific responsive breakpoint or working area
 
+## `browser_list_attached_instances`
+
+Purpose:
+
+- list browser instances currently connected through the local attach extension bridge
+
+Input:
+
+- `session_id?`
+
+Output:
+
+- `session_id?`
+- `total`
+- `instances[]`
+  - `instance_id`
+  - `browser`
+  - `extension_version`
+  - `profile_name?`
+  - `connected_at`
+  - `last_seen_at`
+  - `capabilities[]`
+  - `tab_count`
+  - `active_tab?`
+
+## `browser_list_tabs`
+
+Purpose:
+
+- list tabs from an attached browser instance
+
+Input:
+
+- `session_id?`
+- `instance_id?`
+
+Output:
+
+- `session_id?`
+- `instance_id`
+- `total`
+- `tabs[]`
+  - `tab_id`
+  - `window_id?`
+  - `active`
+  - `selected?`
+  - `title`
+  - `url`
+  - `status?`
+
+Notes:
+
+- if only one attached browser instance is connected, `instance_id` can be omitted
+- tab selection for the attach backend can be done either from the browser extension control panel or through `browser_attach_active_tab` / `browser_attach_tab`
+- `active=true` means the tab is active in its browser window; there may be more than one such tab across multiple windows
+- for the tab currently selected for this MCP session, prefer `selected=true`
+
+## `browser_attach_active_tab`
+
+Purpose:
+
+- attach the current MCP session to the active tab of a connected attached browser instance
+
+Input:
+
+- `session_id?`
+- `instance_id?`
+
+Output:
+
+- `session_id`
+- `backend`
+- `instance_id`
+- `tab_id`
+- `attached_at`
+- `title?`
+- `url?`
+
+## `browser_attach_tab`
+
+Purpose:
+
+- attach the current MCP session to a specific attached browser tab by `tab_id`
+
+Input:
+
+- `session_id?`
+- `instance_id?`
+- `tab_id`
+
+Output:
+
+- `session_id`
+- `backend`
+- `instance_id`
+- `tab_id`
+- `attached_at`
+- `title?`
+- `url?`
+
+## `browser_detach_tab`
+
+Purpose:
+
+- detach the current MCP session from the currently selected attached browser tab
+
+Input:
+
+- `session_id?`
+
+Output:
+
+- `session_id`
+- `detached`
+
+## `browser_recording_start`
+
+Purpose:
+
+- start a structured browser recording bundle for the attached browser tab currently selected for this MCP session
+
+Input:
+
+- `session_id?`
+- `instance_id?`
+
+Output:
+
+- `session_id`
+- `backend`
+- `started`
+- `recording_id`
+- `instance_id`
+- `tab_id`
+- `tab_title?`
+- `tab_url?`
+- `bundle_dir_name`
+- `bundle_relative_path`
+- `bundle_path`
+- `started_at`
+
+Notes:
+
+- this works only with the attached browser backend, not Playwright
+- select the target tab first from the browser extension control panel
+- the bundle is created under `.mcp-xchange/web/{tab-title-slug}-{timestamp}/`
+- the recording captures page snapshots, console events, request metadata, headers, cookies snapshots, and available request/response bodies
+- for network analysis, read `network/index.ndjson` first, then open the matching `network/requests/{seq}-{request_id}/meta.json`
+- use `request.json`, `response.json`, and optional body files only when deeper detail is needed
+
+## `browser_recording_stop`
+
+Purpose:
+
+- stop the active structured browser recording for this MCP session
+
+Input:
+
+- `session_id?`
+
+Output:
+
+- `session_id`
+- `stopped`
+- `recording_id?`
+- `bundle_dir_name?`
+- `bundle_relative_path?`
+- `bundle_path?`
+- `stopped_at?`
+
+## `browser_recording_status`
+
+Purpose:
+
+- inspect whether a structured browser recording is active and where its bundle is being written
+
+Input:
+
+- `session_id?`
+
+Output:
+
+- `session_id`
+- `active`
+- `recording?`
+  - `backend`
+  - `recording_id`
+  - `instance_id`
+  - `tab_id`
+  - `tab_title?`
+  - `tab_url?`
+  - `bundle_dir_name`
+  - `bundle_relative_path`
+  - `bundle_path`
+  - `started_at`
+  - `stopped_at?`
+  - `status`
+  - `event_count`
+  - `last_event_at?`
+
 ## `browser_console`
 
 Purpose:
@@ -806,6 +1133,8 @@ Output:
 Purpose:
 
 - click an element in the current session page
+- if the session has a selected attached browser tab, click there
+- otherwise click in the isolated Playwright page
 
 Input:
 
@@ -831,6 +1160,8 @@ Output:
 Purpose:
 
 - fill an input or textarea in the current session page
+- if the session has a selected attached browser tab, fill there
+- otherwise fill in the isolated Playwright page
 
 Input:
 
@@ -858,6 +1189,8 @@ Output:
 Purpose:
 
 - send a key press to the page or a targeted element
+- if the session has a selected attached browser tab, press there
+- otherwise press in the isolated Playwright page
 
 Input:
 
@@ -880,6 +1213,38 @@ Output:
 - `url`
 - `title?`
 
+## `browser_inject_script`
+
+Purpose:
+
+- inject JavaScript into the current session browser target
+- if the session has a selected attached browser tab, inject into that real browser tab
+- otherwise inject into the isolated Playwright page
+- the script is wrapped so `window[namespace]` exists; default namespace is `TELLY`
+
+Input:
+
+- `session_id?`
+- `source?`
+- `file_path?`
+- `namespace?`
+
+Rules:
+
+- provide either `source` or `file_path`
+- prefer `file_path` for local reusable scripts and larger payloads
+- injected scripts can use both `window.TELLY` and local `TELLY` when `namespace` is left at the default
+
+Output:
+
+- `session_id`
+- `injected`
+- `namespace`
+- `source_type`
+- `bytes`
+- `url`
+- `title?`
+
 ## `browser_wait_for`
 
 Purpose:
@@ -921,6 +1286,7 @@ Browser target rules for `browser_click`, `browser_fill`, `browser_press`, `brow
   - `div[data-testid="save"]`
 - use `text` only when there is no reliable selector
 - do not mix ambiguous hashed CSS classes with fuzzy text guessing when a stable selector exists
+- for the attached browser backend, prefer `selector` first; `ai_tag` and `text` still work, but the attach executor is intentionally lighter than full Playwright
 
 ## `browser_wait_for_url`
 
@@ -1002,6 +1368,8 @@ Output:
 Purpose:
 
 - inspect a DOM element in the session browser tab
+- if the session has a selected attached browser tab, inspect that real browser tab
+- otherwise inspect the isolated Playwright page
 
 Input:
 
@@ -1027,6 +1395,8 @@ Output:
 Purpose:
 
 - inspect computed styles and box metrics for a DOM element in the session browser tab
+- if the session has a selected attached browser tab, inspect that real browser tab
+- otherwise inspect the isolated Playwright page
 
 Input:
 
@@ -1050,6 +1420,8 @@ Output:
 Purpose:
 
 - capture a screenshot from the session browser tab
+- if the session has a selected attached browser tab, capture from that real browser tab
+- otherwise capture from the isolated Playwright page
 
 Input:
 
@@ -1076,12 +1448,15 @@ Notes:
 - they are tracked separately from Telegram-uploaded files
 - they appear under Telegram `Browser -> Screenshots`
 - if `send_to_telegram=true`, the saved screenshot is also sent into the bound Telegram chat for that session
+- for the attached browser backend, the screenshot comes from the selected tab in the user browser session
 
 ## `browser_close`
 
 Purpose:
 
-- close the isolated browser context for the current session
+- close the current browser target for the session
+- if the session has a selected attached browser tab, close that real browser tab and clear the attach state
+- otherwise close the isolated Playwright browser context
 
 Input:
 

package/config/templates/env.both.template

@@ -29,13 +29,27 @@ MCP_HTTP_PATH=/mcp
 MCP_HTTP_ENABLE_DEBUG_ROUTES=false
 MCP_HTTP_ENABLE_PRUNE_ROUTE=false
 
+# ChatGPT / Claude OAuth connector. Setting any OAuth value enables the facade.
+# TELLYMCP_PUBLIC_URL=https://your-domain.example/api
+# TELLYMCP_OAUTH_ISSUER=https://your-domain.example/api
+# TELLYMCP_OAUTH_AUDIENCE=https://your-domain.example/api
+# TELLYMCP_MAGIC_TOKEN=change_me_private_connector_token
+# TELLYMCP_MAGIC_TOKEN_HASH=sha256:<hex>
+# TELLYMCP_OAUTH_CLIENT_ID=tellymcp
+# TELLYMCP_OAUTH_CLIENT_SECRET=
+# TELLYMCP_ALLOWED_REDIRECT_URIS=https://chatgpt.com/connector/oauth/...,https://claude.ai/api/mcp/auth_callback
+# TELLYMCP_OAUTH_PRIVATE_KEY_PEM="-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n"
+# TELLYMCP_AUTH_CODE_TTL_SECONDS=300
+# TELLYMCP_OAUTH_SCOPES=tellymcp:read tellymcp:write
+# TELLYMCP_OAUTH_KEY_ID=tellymcp-oauth
+
 DISTRIBUTED_MODE=both
 GATEWAY_PUBLIC_URL=
 GATEWAY_WS_URL=
 GATEWAY_WS_PATH=/api/gateway/ws
 GATEWAY_TOKEN=
 GATEWAY_USER_UUID=
-# GATEWAY_AUTH_TOKEN=
+GATEWAY_AUTH_TOKEN=
 
 ROOT_PREFIX=/api
 PORT=8080
@@ -63,7 +77,6 @@ WEBAPP_INITDATA_TTL_SECONDS=300
 WEBAPP_SESSION_TTL_SECONDS=900
 WEBAPP_LAUNCH_MODE=fullscreen
 WEBAPP_VISIBLE_SCREENS=10
-WEBAPP_POLL_INTERVAL_MS=2000
 WEBAPP_ACTION_COOLDOWN_MS=150
 
 MCP_XCHANGE_DIR=.mcp-xchange

package/config/templates/env.client.template

@@ -24,7 +24,7 @@ GATEWAY_WS_URL=
 GATEWAY_WS_PATH=/api/gateway/ws
 GATEWAY_TOKEN=
 GATEWAY_USER_UUID=
-# GATEWAY_AUTH_TOKEN=
+GATEWAY_AUTH_TOKEN=
 
 TERMINAL_SHELL=bash
 TERMINAL_COLS=120
@@ -51,7 +51,6 @@ WEBAPP_INITDATA_TTL_SECONDS=300
 WEBAPP_SESSION_TTL_SECONDS=900
 WEBAPP_LAUNCH_MODE=fullscreen
 WEBAPP_VISIBLE_SCREENS=10
-WEBAPP_POLL_INTERVAL_MS=2000
 WEBAPP_ACTION_COOLDOWN_MS=150
 
 MCP_XCHANGE_DIR=.mcp-xchange
@@ -66,6 +65,10 @@ BROWSER_WAIT_UNTIL=load
 # BROWSER_EXECUTABLE_PATH=
 # BROWSER_CHANNEL=chrome
 BROWSER_SLOW_MO_MS=0
+# BROWSER_ATTACH_ENABLED=true
+# BROWSER_ATTACH_WS_HOST=127.0.0.1
+# BROWSER_ATTACH_WS_PORT=9999
+# BROWSER_ATTACH_WS_PATH=/browser-attach/ws
 
 NAMESPACE=mcp
 NODE_ID=agent

package/config/templates/env.gateway.template

@@ -25,12 +25,26 @@ MCP_HTTP_PATH=/mcp
 MCP_HTTP_ENABLE_DEBUG_ROUTES=false
 MCP_HTTP_ENABLE_PRUNE_ROUTE=false
 
+# ChatGPT / Claude OAuth connector. Setting any OAuth value enables the facade.
+# TELLYMCP_PUBLIC_URL=https://your-domain.example/api
+# TELLYMCP_OAUTH_ISSUER=https://your-domain.example/api
+# TELLYMCP_OAUTH_AUDIENCE=https://your-domain.example/api
+# TELLYMCP_MAGIC_TOKEN=change_me_private_connector_token
+# TELLYMCP_MAGIC_TOKEN_HASH=sha256:<hex>
+# TELLYMCP_OAUTH_CLIENT_ID=tellymcp
+# TELLYMCP_OAUTH_CLIENT_SECRET=
+# TELLYMCP_ALLOWED_REDIRECT_URIS=https://chatgpt.com/connector/oauth/...,https://claude.ai/api/mcp/auth_callback
+# TELLYMCP_OAUTH_PRIVATE_KEY_PEM="-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n"
+# TELLYMCP_AUTH_CODE_TTL_SECONDS=300
+# TELLYMCP_OAUTH_SCOPES=tellymcp:read tellymcp:write
+# TELLYMCP_OAUTH_KEY_ID=tellymcp-oauth
+
 DISTRIBUTED_MODE=gateway
 GATEWAY_PUBLIC_URL=
 GATEWAY_WS_URL=
 GATEWAY_WS_PATH=/api/gateway/ws
 GATEWAY_TOKEN=
-# GATEWAY_AUTH_TOKEN=
+GATEWAY_AUTH_TOKEN=
 
 ROOT_PREFIX=/api
 PORT=8080
@@ -58,7 +72,6 @@ WEBAPP_INITDATA_TTL_SECONDS=300
 WEBAPP_SESSION_TTL_SECONDS=900
 WEBAPP_LAUNCH_MODE=fullscreen
 WEBAPP_VISIBLE_SCREENS=10
-WEBAPP_POLL_INTERVAL_MS=2000
 WEBAPP_ACTION_COOLDOWN_MS=150
 
 MCP_XCHANGE_DIR=.mcp-xchange