openbuilder 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 OpenBuilder Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,309 @@
+# OpenBuilder
+
+**Open-source AI meeting assistant** — a Read AI alternative that joins your Google Meet meetings, captures live transcripts, and generates AI-powered meeting reports.
+
+Get meeting summaries, action items, key decisions, and speaker analytics — all from your terminal.
+
+## Features
+
+- **Audio Capture + Whisper** — Captures browser audio via PulseAudio virtual sink, transcribes with OpenAI Whisper (no captions needed)
+- **Caption Scraping Fallback** — DOM-based caption capture for systems without PulseAudio/ffmpeg
+- **Google Meet Bot** — Headless Chromium bot joins meetings automatically
+- **AI Meeting Reports** — Post-meeting analysis with summaries, chapters, action items, decisions, and questions
+- **Speaker Analytics** — Talk time, word count, and participation percentages per speaker
+- **Multiple AI Providers** — Claude (Anthropic) and OpenAI supported; bring your own API key
+- **Standalone Analysis** — Run `summarize` or `report` on any transcript file, not just live meetings
+- **OpenClaw Skill** — Integrates as a skill for OpenClaw agents
+- **Privacy-First** — Your data stays local; AI processing only happens with your own API keys
+
+## Quick Start
+
+### 1. Install
+
+```bash
+npx openbuilder
+```
+
+This installs the OpenClaw skill and Chromium browser.
+
+### 2. Configure AI (Optional but Recommended)
+
+```bash
+# Set your AI provider API key for meeting reports
+npx openbuilder config set anthropicApiKey sk-ant-your-key-here
+# Or use OpenAI instead
+npx openbuilder config set openaiApiKey sk-your-key-here
+npx openbuilder config set aiProvider openai
+```
+
+Or use environment variables:
+
+```bash
+export ANTHROPIC_API_KEY=sk-ant-your-key-here
+# or
+export OPENAI_API_KEY=sk-your-key-here
+```
+
+### 3. Join a Meeting
+
+```bash
+# As a guest (host must admit)
+npx openbuilder join https://meet.google.com/abc-defg-hij --anon --bot-name "Meeting Bot"
+
+# With Google authentication (no admission needed)
+npx openbuilder auth  # one-time setup
+npx openbuilder join https://meet.google.com/abc-defg-hij --auth
+```
+
+### 4. Get Results
+
+```bash
+# View transcript
+npx openbuilder transcript
+
+# Quick AI summary
+npx openbuilder summarize
+
+# Full meeting report (summary + action items + decisions + analytics)
+npx openbuilder report
+```
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `openbuilder` / `openbuilder install` | Install skill + Chromium |
+| `openbuilder join <url> [options]` | Join a Google Meet meeting |
+| `openbuilder auth` | Save Google session for authenticated joins |
+| `openbuilder transcript [--last N]` | Print the latest transcript |
+| `openbuilder screenshot` | Take an on-demand screenshot |
+| `openbuilder summarize [path]` | AI summary of a transcript |
+| `openbuilder report [path]` | Full AI meeting report |
+| `openbuilder config [subcommand]` | Manage configuration |
+| `openbuilder help` | Show help |
+
+### Join Options
+
+```
+--auth          Join using saved Google account
+--anon          Join as a guest (requires --bot-name)
+--bot-name      Guest display name
+--duration      Auto-leave after duration (30m, 1h, etc.)
+--audio         Force audio capture mode (PulseAudio + Whisper)
+--captions      Force caption scraping mode (DOM-based fallback)
+--headed        Show browser window (debugging)
+--camera        Join with camera on (default: off)
+--mic           Join with microphone on (default: off)
+--no-report     Skip auto-report after meeting ends
+--verbose       Show real-time transcript output
+```
+
+By default, capture mode is `auto`: uses audio capture if PulseAudio, ffmpeg, and `OPENAI_API_KEY` are all available. Otherwise falls back to caption scraping.
+
+## AI Meeting Reports
+
+When a meeting ends (or when you run `openbuilder report`), OpenBuilder generates a structured report:
+
+```markdown
+# Meeting Report: abc-defg-hij — 2026-03-12
+
+## Summary
+The team discussed the Q1 roadmap, focusing on three main initiatives...
+
+## Chapters
+1. [14:30] Sprint Planning — Review of current sprint velocity
+2. [14:42] Q1 Roadmap — Discussion of three main initiatives
+3. [14:55] Resource Allocation — Team capacity and hiring plans
+
+## Action Items
+- [ ] Draft Q1 roadmap document (@Alice)
+- [ ] Schedule design review for new dashboard (@Bob)
+- [ ] Research caching solutions and present options (@Carol)
+
+## Key Decisions
+- Decided to prioritize the dashboard redesign over the API refactor
+- Agreed to move to bi-weekly sprints starting next month
+
+## Key Questions
+- Should we invest in automated testing infrastructure? (unanswered)
+- When is the design system migration complete? (answered)
+
+## Speaker Analytics
+| Speaker | Talk Time | % of Meeting | Words |
+|---------|-----------|--------------|-------|
+| Alice   | 12:30     | 45%          | 1,850 |
+| Bob     | 8:15      | 30%          | 1,200 |
+| Carol   | 6:45      | 25%          | 980   |
+
+## Metadata
+- Duration: 27:30
+- Participants: 3
+```
+
+## Configuration
+
+### Config File
+
+Settings are stored in `~/.openbuilder/config.json`.
+
+```bash
+# View all settings
+npx openbuilder config
+
+# Set values
+npx openbuilder config set aiProvider claude
+npx openbuilder config set anthropicApiKey sk-ant-...
+npx openbuilder config set openaiApiKey sk-...
+npx openbuilder config set botName "My Meeting Bot"
+npx openbuilder config set defaultDuration 60m
+npx openbuilder config set captureMode audio
+npx openbuilder config set whisperModel whisper-1
+
+# Get a value
+npx openbuilder config get aiProvider
+
+# Delete a value
+npx openbuilder config delete defaultDuration
+```
+
+### Environment Variables
+
+Environment variables override config file values:
+
+| Variable | Config Key | Description |
+|----------|-----------|-------------|
+| `OPENBUILDER_AI_PROVIDER` | `aiProvider` | `claude` or `openai` |
+| `ANTHROPIC_API_KEY` | `anthropicApiKey` | Anthropic API key |
+| `OPENAI_API_KEY` | `openaiApiKey` | OpenAI API key (also used for Whisper) |
+| `OPENBUILDER_BOT_NAME` | `botName` | Default bot name |
+| `OPENBUILDER_DEFAULT_DURATION` | `defaultDuration` | Default meeting duration |
+| `OPENBUILDER_CAPTURE_MODE` | `captureMode` | `audio`, `captions`, or `auto` (default) |
+| `OPENBUILDER_WHISPER_MODEL` | `whisperModel` | Whisper model name (default `whisper-1`) |
+
+## Authentication
+
+The bot can join meetings in two modes:
+
+### Guest Mode (default)
+```bash
+npx openbuilder join <url> --anon --bot-name "Meeting Bot"
+```
+The bot joins as a guest. The meeting host must admit the bot.
+
+### Authenticated Mode
+```bash
+# One-time setup: sign into Google in a browser window
+npx openbuilder auth
+
+# Join as your Google account (no host admission needed)
+npx openbuilder join <url> --auth
+```
+
+The auth command opens a headed Chromium browser. Sign into your Google account, then press Enter. The session is saved to `~/.openbuilder/auth.json` and reused for future joins. Re-run if the session expires.
+
+### Automated Auth (Headless Servers)
+
+For headless servers or automated bots, use `--auto` mode with credentials in `.env`:
+
+```bash
+echo "GOOGLE_EMAIL=you@gmail.com" >> .env
+echo "GOOGLE_PASSWORD=yourpassword" >> .env
+npx openbuilder auth --auto
+```
+
+This signs in non-interactively and saves the session to `~/.openbuilder/auth.json`.
+
+## Standalone Transcript Analysis
+
+OpenBuilder's AI analysis works on any transcript file in `[HH:MM:SS] Speaker: text` format — not just from live meetings:
+
+```bash
+# Summarize any transcript
+npx openbuilder summarize ~/meetings/standup-2026-03-12.txt
+
+# Full report on any transcript
+npx openbuilder report ~/meetings/standup-2026-03-12.txt
+```
+
+## File Locations
+
+| Path | Description |
+|------|-------------|
+| `~/.openbuilder/config.json` | Configuration file |
+| `~/.openbuilder/auth.json` | Saved Google session |
+| `~/.openbuilder/auth-meta.json` | Auth metadata (email, timestamp) |
+| `~/.openbuilder/builder.pid` | Running bot PID |
+| `~/.openclaw/workspace/openbuilder/transcripts/` | Live caption transcripts |
+| `~/.openclaw/workspace/openbuilder/reports/` | AI meeting reports |
+
+## How It Works
+
+1. **Join**: Launches headless Chromium with stealth patches (navigator.webdriver, WebGL, plugins), navigates to the Meet URL, enters the bot name, disables camera/mic, and clicks join.
+
+2. **Transcript Capture** (two modes):
+   - **Audio mode** (default when available): Creates a PulseAudio virtual sink, routes all browser audio to it, captures audio with ffmpeg into 30-second WAV chunks, and transcribes each chunk with OpenAI Whisper. No captions needed — works like Read AI / Granola.
+   - **Caption mode** (fallback): Enables Google Meet's built-in live captions, then injects a MutationObserver that watches for caption DOM mutations. Speaker names are extracted from badge elements. Captions are deduplicated and written to disk with timestamps.
+
+3. **AI Analysis**: When the meeting ends, the transcript is sent to Claude or OpenAI with carefully designed prompts. Long transcripts are chunked and merged. The AI extracts summaries, chapters, action items (with assignee detection), key decisions, and key questions.
+
+4. **Speaker Analytics**: Talk time is estimated from transcript timestamps and speaking rate heuristics. Per-speaker word counts and participation percentages are calculated.
+
+## Requirements
+
+- **Node.js** >= 18
+- **playwright-core** (installed automatically)
+- **Optional**: `@anthropic-ai/sdk` or `openai` npm package for AI features
+  ```bash
+  npm install @anthropic-ai/sdk  # For Claude
+  npm install openai              # For OpenAI (also required for audio capture mode)
+  ```
+
+### Audio Capture Mode (recommended)
+
+For audio capture via PulseAudio + Whisper, you also need:
+
+- **PulseAudio** — `apt install pulseaudio` (most Linux desktops have this already)
+- **ffmpeg** — `apt install ffmpeg`
+- **OpenAI API key** — for Whisper transcription (`OPENAI_API_KEY` env var or config)
+
+If these aren't available, OpenBuilder automatically falls back to caption scraping.
+
+## OpenClaw Bot Setup
+
+To set up OpenBuilder as an automated meeting bot (e.g. for OpenClaw agents):
+
+1. **Install**: Clone the repo or `npm install openbuilder`
+2. **Configure credentials** in `.env`:
+   ```bash
+   GOOGLE_EMAIL=you@gmail.com
+   GOOGLE_PASSWORD=yourpassword
+   # For AI reports:
+   ANTHROPIC_API_KEY=sk-ant-...
+   # Or: OPENAI_API_KEY=sk-...
+   ```
+3. **Save Google session**: `npx openbuilder auth --auto`
+4. **Join meetings**: `npx openbuilder join <url> --auth --captions`
+5. **Transcripts** saved to `~/.openclaw/workspace/openbuilder/transcripts/`
+6. **Reports** saved to `~/.openclaw/workspace/openbuilder/reports/`
+
+### Notes
+
+- `.env` is gitignored — safe for credentials
+- Auth session saved to `~/.openbuilder/auth.json` — re-run `auth --auto` if expired
+- Caption mode (`--captions`) is the most reliable on headless servers
+- Audio mode (`--audio`) requires PulseAudio + ffmpeg + OpenAI key + Xvfb (experimental on servers)
+
+## OpenClaw Integration
+
+OpenBuilder ships as an OpenClaw skill. After running `npx openbuilder`, it's available to your OpenClaw agent. The agent can:
+
+- Join meetings on your behalf
+- Capture and summarize transcripts
+- Generate full meeting reports
+- Send screenshots to your chat
+
+See [SKILL.md](./SKILL.md) for the full agent integration guide.
+
+## License
+
+MIT

package/SKILL.md

@@ -0,0 +1,276 @@
+---
+name: open-builder
+description: AI meeting assistant — joins Google Meet, captures live transcripts, generates AI-powered meeting reports with summaries, action items, decisions, and speaker analytics.
+homepage: https://github.com/superliangbot/openbuilder
+metadata: { "openclaw": { "emoji": "📋", "requires": { "bins": ["node"] } } }
+---
+
+# OpenBuilder
+
+## STOP — Ask the user before doing anything
+
+**When the user asks to join a meeting, you MUST do these steps IN ORDER. Do NOT skip ahead.**
+
+**Step A:** Read the file `~/.openbuilder/auth-meta.json` (it may not exist — that's fine).
+
+**Step B:** Ask the user how they want to join. Do NOT launch the bot yet.
+
+If auth-meta.json exists and has an `email` field, ask:
+
+> "How would you like to join the meeting?
+> 1. Join as **user@gmail.com** (authenticated — no host approval needed)
+> 2. Join as **OpenBuilder Bot** (guest — host must admit)
+> 3. Join with a custom name (guest)"
+
+If auth-meta.json does NOT exist, ask:
+
+> "What name should the bot use to join the meeting?"
+
+Default to "OpenBuilder Bot" if the user doesn't have a preference.
+
+**Step C:** WAIT for the user to answer. Only then launch the bot with the right flags.
+
+- User chose authenticated → use `--auth`
+- User chose guest → use `--anon --bot-name "Their Name"` (always ask for a name)
+
+**The bot will refuse to start without `--auth` or `--anon`.
+When using `--anon`, `--bot-name` is also required.**
+
+---
+
+A meeting bot that joins Google Meet meetings via Playwright browser automation,
+captures live captions as a real-time transcript, and generates AI-powered meeting
+reports with summaries, action items, key decisions, and speaker analytics.
+
+## Prerequisites
+
+- `playwright-core` (ships with openclaw)
+- Chromium browser: `npx playwright-core install chromium`
+- Optional: `@anthropic-ai/sdk` or `openai` for AI report generation
+
+## Join a Meeting
+
+**IMPORTANT: Always run join commands with `background:true`** — the bot is a long-running
+process that stays in the meeting. Do not wait for it to complete; background it immediately
+and poll for status updates.
+
+### Launch command
+
+```bash
+exec background:true command:"npx openbuilder join https://meet.google.com/abc-defg-hij --auth|--anon --channel <current-channel> --target <current-chat-id>"
+```
+
+**IMPORTANT:** Always pass `--channel` and `--target` from the current conversation context.
+The bot uses these to send screenshots and status updates directly to the user's chat.
+
+Options (required — bot will error without one):
+
+- `--auth` — join using saved Google account (~/.openbuilder/auth.json)
+- `--anon --bot-name "Name"` — join as a guest with this display name (both required together)
+
+Other options:
+
+- `--headed` — show the browser window (for debugging)
+- `--camera` — join with camera on (default: off)
+- `--mic` — join with microphone on (default: off)
+- `--duration 60m` — auto-leave after duration (supports ms/s/m/h)
+- `--no-report` — skip auto-report generation when meeting ends
+- `--verbose` — show real-time caption output
+
+## Live Caption Transcript
+
+Captions are automatically captured whenever the bot is in a meeting. After joining,
+the bot enables Google Meet's built-in live captions and captures the text via a
+MutationObserver. Captions are deduplicated and flushed to a transcript file every 5 seconds.
+
+**Transcript location:** `~/.openclaw/workspace/openbuilder/transcripts/<meeting-id>.txt`
+
+**Format:**
+```
+[14:30:05] Alice: Hey everyone, let's get started
+[14:30:12] Bob: Sounds good, I have the updates ready
+[14:30:25] Alice: Great, go ahead
+```
+
+## Get Transcript (what are they saying?)
+
+**When the user asks "what are they saying?", "what's happening?", "summarize the meeting",
+or anything about meeting content — run this script. Do NOT use builder-screenshot.ts for this.**
+
+```bash
+exec command:"npx openbuilder transcript"
+```
+
+Use `--last 20` to get only the last 20 lines (for long meetings).
+
+Read the output and summarize it for the user in natural language.
+
+## Take a Screenshot (visual context only)
+
+If the user asks to **see** the meeting (e.g. "send me a screenshot", "what does it look like"):
+
+```bash
+exec command:"npx openbuilder screenshot"
+```
+
+Send the screenshot image to the user via `message`. Do NOT read the screenshot yourself.
+
+## AI Summary (quick summary)
+
+When the user asks for a summary and the meeting is over (or for a standalone transcript):
+
+```bash
+exec command:"npx openbuilder summarize"
+exec command:"npx openbuilder summarize /path/to/transcript.txt"
+```
+
+Returns a 3-5 paragraph summary of the meeting.
+
+## Full Meeting Report (summary + actions + decisions + analytics)
+
+For a comprehensive meeting report with all intelligence:
+
+```bash
+exec command:"npx openbuilder report"
+exec command:"npx openbuilder report /path/to/transcript.txt"
+```
+
+Generates and saves a markdown report with:
+- Meeting summary with chapters
+- Action items with assignee detection
+- Key decisions
+- Key questions (answered/unanswered)
+- Speaker talk-time analytics
+
+Report saved to: `~/.openclaw/workspace/openbuilder/reports/<meeting-id>-report.md`
+
+## Configuration
+
+```bash
+exec command:"npx openbuilder config"
+exec command:"npx openbuilder config set anthropicApiKey sk-ant-..."
+exec command:"npx openbuilder config set aiProvider claude"
+```
+
+Keys: `aiProvider`, `anthropicApiKey`, `openaiApiKey`, `botName`, `defaultDuration`
+
+## How It Works
+
+1. **Join**: Launches headless Chromium, navigates to the Meet URL, enters the bot name, clicks "Ask to join", and waits for host admission.
+
+2. **Caption capture**: After joining, the bot clicks the CC button to enable live captions, then injects a MutationObserver to capture caption text from the DOM. Captions are deduplicated and written to a transcript file.
+
+3. **AI Report** (automatic): When the meeting ends, if an AI API key is configured, the bot automatically processes the transcript through Claude or OpenAI to generate a structured meeting report.
+
+## Authentication (Optional)
+
+By default the bot joins as a guest and needs host admission. To join as an authenticated
+Google user (no admission needed), run the auth script once:
+
+```bash
+npx openbuilder auth
+```
+
+This opens a headed browser — sign into Google, then press Enter. The session is saved to
+`~/.openbuilder/auth.json` and automatically loaded on future joins. Re-run if the session expires.
+
+## Files
+
+- `~/.openbuilder/auth.json` — saved Google session (cookies + localStorage)
+- `~/.openbuilder/auth-meta.json` — email + timestamp
+- `~/.openbuilder/config.json` — bot configuration
+- `~/.openbuilder/chrome-profile/` — persistent Chromium profile
+- `~/.openbuilder/builder.pid` — running bot PID
+- `~/.openclaw/workspace/openbuilder/transcripts/` — live caption transcripts
+- `~/.openclaw/workspace/openbuilder/reports/` — AI meeting reports
+- `~/.openclaw/workspace/openbuilder/on-demand-screenshot.png` — on-demand screenshot
+- `~/.openclaw/workspace/openbuilder/joined-meeting.png` — confirmation screenshot
+- `~/.openclaw/workspace/openbuilder/debug-*.png` — failure screenshots
+
+## Agent Behavior — MANDATORY
+
+After launching the bot with `exec background:true`, you MUST poll the process
+to check for success/failure and send screenshots back to the user.
+
+### Step 1: Poll for output
+
+After starting the background exec, poll the process every 10-15 seconds:
+
+```
+process action:poll
+```
+
+### Step 2: Parse markers and send images using the message tool
+
+The bot prints machine-readable markers. When you see them, you MUST use the
+`message` tool to send the screenshot image to the user.
+
+**On success** — bot prints `[OPENBUILDER_SUCCESS_IMAGE] <path>`:
+
+```
+message action:"send" media:"./openbuilder/joined-meeting.png" content:"Successfully joined the meeting!"
+```
+
+**On screenshot request** — bot prints `[OPENBUILDER_SCREENSHOT] <path>`:
+
+```
+message action:"send" media:"./openbuilder/on-demand-screenshot.png" content:"Here's the current meeting view"
+```
+
+**On failure** — bot prints `[OPENBUILDER_DEBUG_IMAGE] <path>`:
+
+```
+message action:"send" media:"./openbuilder/debug-join-failed.png" content:"Could not join the meeting. Here is what the bot saw"
+```
+
+**On report generated** — bot prints `[OPENBUILDER_REPORT] <path>`:
+
+Read the report file and share a formatted summary with the user.
+
+**CRITICAL: ALWAYS use the `message` tool with `media:"./openbuilder/<filename>.png"` to send screenshots.**
+Use relative paths only (starting with `./`). Never use absolute paths or ~ paths.
+
+### Step 3: When user asks about meeting content
+
+**CRITICAL: When the user asks what's happening, what someone said, or anything about
+meeting content — run `builder-transcript.ts`. NEVER use `builder-screenshot.ts` for this.**
+
+```bash
+exec command:"npx openbuilder transcript"
+```
+
+### Step 4: When meeting ends
+
+When the bot reports the meeting has ended:
+1. Run `npx openbuilder transcript` to get the full transcript
+2. If a report was auto-generated (`[OPENBUILDER_REPORT]` marker), read and share it
+3. If no report was generated, offer to run `npx openbuilder report` for the user
+
+### When to use which command
+
+| User asks...                              | Use this command              |
+|-------------------------------------------|-------------------------------|
+| "what are they saying?"                   | `openbuilder transcript`      |
+| "what's happening in the meeting?"        | `openbuilder transcript`      |
+| "summarize the meeting"                   | `openbuilder summarize`       |
+| "give me a full report"                   | `openbuilder report`          |
+| "what are the action items?"              | `openbuilder report`          |
+| "send me a screenshot"                    | `openbuilder screenshot`      |
+| "what does the meeting look like?"        | `openbuilder screenshot`      |
+| "what did they talk about?"               | `openbuilder transcript`      |
+
+**NEVER read or analyze screenshot images to understand meeting content.**
+
+## Headless VM Tips
+
+- Chrome flags `--use-fake-ui-for-media-stream` and `--use-fake-device-for-media-stream` are set automatically.
+- No X11/Wayland display is required — runs fully headless.
+- Use `--duration` to auto-leave after a set time.
+
+## Troubleshooting
+
+- **Join button not found**: Google Meet UI changes occasionally. The debug screenshot shows what the bot saw — send it to the user.
+- **Not admitted**: The bot joins as a guest and needs host approval. Ask the host to admit the bot. If timed out, the debug screenshot is sent automatically.
+- **No captions captured**: The CC button selector may change with Meet updates. If the transcript is empty, captions may not have been enabled. Try `--headed` to verify.
+- **Headless blocked**: The bot uses stealth patches to bypass headless detection. If Google Meet blocks it, try `--headed` for debugging.
+- **AI report failed**: Ensure an API key is configured via `openbuilder config set anthropicApiKey <key>` or `ANTHROPIC_API_KEY` env var.