crewly 1.13.0 → 1.14.0

package/config/skills/agent/remote-browser/SKILL.md

@@ -0,0 +1,178 @@
+---
+name: Remote Browser
+description: "Drive the user's real Chrome browser through the Crewly Chrome Extension. Navigate, click, fill forms, screenshot, read text, run JS, inspect cookies/console. Each agent gets its own bound tab, so concurrent agents never fight over the active tab. Plain HTTP + bash — usable by any agent runtime, not just Claude."
+version: 1.0.0
+category: browser
+skillType: claude-skill
+assignableRoles:
+  - developer
+  - qa
+  - researcher
+  - marketing
+triggers:
+  - remote browser
+  - drive chrome
+  - control my browser
+  - use my chrome
+  - browser automation
+  - fill the form
+  - take a screenshot of the page
+  - scrape the page
+tags:
+  - browser
+  - chrome
+  - extension
+  - automation
+  - scraping
+---
+
+# Remote Browser
+
+Control the user's **real, logged-in Chrome** — their sessions, their cookies,
+their extensions — through the Crewly Chrome Extension.
+
+```
+agent → bash execute.sh → POST /api/browser/* → WebSocket → Chrome Extension → real tab
+```
+
+The transport is plain HTTP from a bash script, so **any agent runtime can use
+it** (Claude Code, Codex, Gemini CLI, a cron job). There is no MCP dependency
+and nothing model-specific.
+
+## Why this and not a headless browser
+
+Use this when the task needs the user's **actual logged-in state** — an account
+only they are signed into, a site behind SSO, a page whose content differs for
+an authenticated user. For anything that works logged-out, a headless browser is
+cheaper and will not disturb the user's window.
+
+## Usage
+
+Two equivalent calling conventions:
+
+```bash
+# JSON argument
+bash execute.sh '{"action":"navigate","url":"https://example.com"}'
+
+# Flags
+bash execute.sh --action navigate --url https://example.com
+bash execute.sh -a click -s "#submit"
+```
+
+`bash execute.sh --help` prints the action list.
+
+## Actions
+
+### Page control
+| Action | Required | Optional |
+|---|---|---|
+| `navigate` | `--url` | |
+| `screenshot` | | |
+| `full-page-screenshot` | | scrolls and stitches the whole page |
+| `read-text` | | `--selector` (defaults to whole page) |
+| `scroll` | | `--params` e.g. `'{"direction":"down","amount":3}'` |
+| `scroll-in-element` | | `--params` with a selector + direction |
+
+### Interaction
+| Action | Required | Optional |
+|---|---|---|
+| `click` | `--selector` | or `--params` for coordinate clicks |
+| `fill` | `--selector` `--value` | replaces the field's contents |
+| `type` | `--selector` `--text` | types keystroke by keystroke |
+| `hover` | `--selector` | |
+| `press-key` | `--params` e.g. `'{"key":"Enter"}'` | |
+| `select-option` | `--selector` + one of `--value` / `--params` with `label` or `index` | resolution order: value → label → index |
+| `set-file-input` | `--params` with selector + file path | |
+
+### Reading the page
+| Action | Required | Optional |
+|---|---|---|
+| `get-element` | `--selector` | |
+| `get-interactive-elements` | | `--params` — buttons/links/inputs with refs |
+| `search-text` | `--text` | locate text on the page |
+| `list-options` | `--selector` | enumerate a `<select>` |
+| `wait-for-selector` | `--selector` | blocks until it appears |
+| `execute-js` | `--code` | alias: `execute`. Raw JS in page context |
+
+### Browser state
+| Action | Notes |
+|---|---|
+| `tabs` | list open tabs |
+| `cookies` | optional `?domain=` on the endpoint |
+| `console` | recent console messages; optional `?clear=true` |
+| `local-storage` | `--params` to scope the read |
+| `status` | extension connection status |
+| `instances` | connected browser instances |
+| `proxy-connect` | manually connect the proxy to Cloud Relay |
+
+### Tab binding
+| Action | Notes |
+|---|---|
+| `bind-tab` | bind a tab to this agent; `--active` foregrounds it (default: background) |
+| `unbind-tab` | release the tab and close it |
+
+## Per-agent tab binding
+
+This is what makes the skill safe to run from several agents at once.
+
+When `CREWLY_SESSION_NAME` is set in the agent's environment it is forwarded as
+the `X-Agent-Session` header, and the backend dispatches each command to **that
+agent's own tab** instead of whatever tab the user is looking at.
+
+- The first command from a fresh agent auto-binds a new background tab.
+- The resulting id is cached at
+  `~/.crewly/runtime/<agentSession>/browser-tab-id`, so later calls skip the
+  bind round-trip.
+- Concurrent agents therefore never steal each other's page — and never yank
+  the user's foreground tab out from under them.
+
+Escape hatches:
+
+| Flag | Effect |
+|---|---|
+| `--tab-id <n>` | pin one call to a specific tab (tests, debugging) |
+| `--no-bind` | drop the session header and use the legacy active-tab path |
+| `--active` | with `bind-tab`, open the tab in the foreground |
+
+With `CREWLY_SESSION_NAME` unset, behaviour falls back to the legacy active-tab
+path, so scripts written before per-tab dispatch keep working unchanged.
+
+## Examples
+
+```bash
+# Read a page that requires the user's login
+bash execute.sh -a navigate -u "https://app.example.com/reports"
+bash execute.sh -a wait-for-selector -s ".report-table"
+bash execute.sh -a read-text -s ".report-table"
+
+# Fill and submit a form
+bash execute.sh -a fill -s "#email" -v "someone@example.com"
+bash execute.sh -a type -s "#note" -t "Following up on the quote"
+bash execute.sh -a click -s "button[type=submit]"
+
+# Pull structured data out of the DOM
+bash execute.sh -a execute-js -c '[...document.querySelectorAll(".row")].map(r=>r.innerText)'
+
+# Explicit lifecycle when doing a long multi-step job
+bash execute.sh -a bind-tab
+# ... many commands ...
+bash execute.sh -a unbind-tab
+```
+
+## Prerequisites
+
+- Crewly backend running, with the Chrome Extension connected
+  (`bash execute.sh -a status` to confirm)
+- `curl` and `jq` in PATH
+
+## Cautions
+
+- **This is the user's real browser.** Actions have real effects under their
+  real identity: a click can send a message, place an order, or delete
+  something. Confirm before anything irreversible or outward-facing.
+- Never enter passwords, card numbers, or other credentials — ask the user to
+  do it themselves in their own window.
+- Avoid clicking elements that raise native `alert()` / `confirm()` dialogs:
+  they block the extension until dismissed by hand.
+- Always `unbind-tab` when a job is done, or abandoned tabs accumulate in the
+  user's window.

package/config/skills/agent/xiaoyuzhoufm-transcript/SKILL.md

@@ -1,29 +1,37 @@
 ---
 name: xiaoyuzhoufm-transcript
-description: Extract and transcribe 小宇宙 (Xiaoyuzhou FM) podcast episodes. Downloads audio from episode URL and uses Gemini Audio API for transcription with speaker identification and timestamps.
+description: Extract and transcribe 小宇宙 (Xiaoyuzhou FM) podcast episodes. Resolves an episode URL to its audio, then transcribes with Whisper via the transcribe-audio skill — local whisper.cpp by default (free, offline), OpenAI Whisper API as fallback. Segment-level timestamps.
 category: content
 assignableRoles:
   - "*"
-version: "1.0.0"
+version: "2.0.0"
 tags:
   - podcast
   - transcript
   - audio
-  - gemini
+  - whisper
   - xiaoyuzhou
   - chinese
 ---
 
 # 小宇宙 Podcast Transcript
 
-Extract and transcribe podcast episodes from 小宇宙 (Xiaoyuzhou FM) using Gemini Audio API.
+Turn a 小宇宙 episode URL into a timestamped transcript.
 
 ## Pipeline
 
 ```
-Episode URL → Fetch HTML → Extract audio URL → Download m4a → Upload to Gemini → Transcribe → Output
+Episode URL → Fetch HTML → Extract audio URL → Download m4a
+                                                    ↓
+                                    transcribe-audio (Whisper) → Markdown + JSON
 ```
 
+This skill owns the 小宇宙 half — resolving an episode page to its audio and
+labelling the result. Transcription is delegated to the `transcribe-audio`
+skill, so there is one Whisper integration in the repo rather than two, and
+this skill inherits its engine selection, normalization and long-file handling
+for free.
+
 ## Usage
 
 ```bash
@@ -36,8 +44,12 @@ bash execute.sh '{"url":"https://www.xiaoyuzhoufm.com/episode/...","outputFile":
 # Transcribe a local audio file (skip download)
 bash execute.sh '{"audioFile":"/path/to/podcast.m4a"}'
 
-# Specify language hint
-bash execute.sh '{"url":"...","language":"Chinese"}'
+# Language hint (ISO-639-1) — helps on mixed Chinese/English episodes
+bash execute.sh '{"url":"...","language":"zh"}'
+
+# Force an engine
+bash execute.sh '{"url":"...","engine":"local"}'
+bash execute.sh '{"url":"...","engine":"openai"}'
 ```
 
 ## Parameters
@@ -47,7 +59,8 @@ bash execute.sh '{"url":"...","language":"Chinese"}'
 | `url` | Yes* | 小宇宙 episode URL |
 | `audioFile` | Yes* | Path to local audio file (alternative to url) |
 | `outputFile` | No | Save transcript as markdown to this path |
-| `language` | No | Language hint for Gemini (default: auto-detect) |
+| `language` | No | Language hint, ISO-639-1 (`zh`, `en`, `ja`…). Default: auto-detect |
+| `engine` | No | `auto` (default), `local`, or `openai` — passed to `transcribe-audio` |
 
 *Either `url` or `audioFile` is required.
 
@@ -58,28 +71,39 @@ bash execute.sh '{"url":"...","language":"Chinese"}'
   "success": true,
   "title": "Episode Title",
   "url": "https://www.xiaoyuzhoufm.com/episode/...",
-  "audioUrl": "https://media.xyzcdn.net/.../audio.m4a",
-  "model": "gemini-2.5-flash-preview-05-20",
-  "transcriptLength": 12345,
-  "transcript": "**Speaker 1:** [00:00:00] Welcome to the show..."
+  "audioUrl": "https://media.xyzcdn.net/...",
+  "engine": "whisper.cpp",
+  "language": "zh",
+  "duration": 4437,
+  "transcriptLength": 41234,
+  "transcript": "[00:00:00] ...\n[00:00:07] ..."
 }
 ```
 
 ## Transcript Format
 
-- Speaker turns identified with `**Speaker Name:** text`
-- Timestamps at segment/topic changes: `[HH:MM:SS]`
-- Preserves original language (Chinese/English/mixed)
+- One line per segment, prefixed `[HH:MM:SS]`
+- Original language preserved (Chinese / English / mixed)
 
 ## Prerequisites
 
-- `GEMINI_API_KEY` environment variable or configured in Settings > API Keys
-- `curl` and `jq` available in PATH
-- Internet access to xiaoyuzhoufm.com and Gemini API
+- `curl` and `jq` in PATH
+- Whatever `transcribe-audio` needs: `ffmpeg`, plus either a local whisper.cpp
+  install (`whisper-cli` + a `ggml-*` model) or `OPENAI_API_KEY` for the fallback
+- Internet access to xiaoyuzhoufm.com (and to the OpenAI API only if the local
+  engine is unavailable)
+
+**No Gemini key is required.** On the default local path the transcription runs
+entirely offline at zero per-use cost.
 
 ## Notes
 
-- Audio files are typically 30-90 minutes (20-80 MB)
-- Gemini processing takes 30-120 seconds depending on audio length
-- Maximum wait time: 5 minutes for Gemini processing
-- Temp files are auto-cleaned on exit
+- Episodes are typically 30–90 minutes (20–80 MB)
+- Local whisper.cpp roughly tracks real time on Apple Silicon; a 75-minute
+  episode takes on the order of ten minutes
+- Temp files (page HTML, downloaded audio) are auto-cleaned on exit — including
+  on interrupt, so a killed run leaves nothing behind and also keeps nothing
+  to resume from
+- **No speaker identification.** The previous Gemini-based version labelled
+  speaker turns; Whisper does not do diarization, so output is timestamped
+  segments only