markupr 2.4.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +137 -557
- package/dist/cli/index.mjs +2118 -154
- package/dist/main/index.mjs +24 -13
- package/dist/mcp/index.mjs +1612 -69
- package/package.json +27 -8
package/README.md
CHANGED
|
@@ -1,343 +1,110 @@
|
|
|
1
1
|
<p align="center">
|
|
2
|
-
<img src="src/renderer/assets/logo.svg" alt="markupr
|
|
2
|
+
<img src="src/renderer/assets/logo.svg" alt="markupr" width="80" height="80">
|
|
3
3
|
</p>
|
|
4
4
|
|
|
5
5
|
<h1 align="center">markupr</h1>
|
|
6
6
|
|
|
7
7
|
<p align="center">
|
|
8
|
-
<strong>
|
|
8
|
+
<strong>Record your screen. Say what's wrong. Your AI agent fixes it.</strong>
|
|
9
9
|
</p>
|
|
10
10
|
|
|
11
11
|
<p align="center">
|
|
12
|
-
<a href="https://github.com/eddiesanjuan/markupr/actions/workflows/ci.yml"><img src="https://github.com/eddiesanjuan/markupr/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
|
|
13
|
-
<a href="https://github.com/eddiesanjuan/markupr/actions/workflows/release.yml"><img src="https://github.com/eddiesanjuan/markupr/actions/workflows/release.yml/badge.svg" alt="Release"></a>
|
|
14
|
-
<a href="https://github.com/eddiesanjuan/markupr/releases"><img src="https://img.shields.io/github/v/release/eddiesanjuan/markupr?style=flat-square" alt="Latest Release"></a>
|
|
15
|
-
<a href="https://github.com/eddiesanjuan/markupr/releases"><img src="https://img.shields.io/github/downloads/eddiesanjuan/markupr/total?style=flat-square" alt="Downloads"></a>
|
|
16
12
|
<a href="https://www.npmjs.com/package/markupr"><img src="https://img.shields.io/npm/v/markupr?style=flat-square" alt="npm version"></a>
|
|
13
|
+
<a href="https://www.npmjs.com/package/markupr"><img src="https://img.shields.io/npm/dm/markupr?style=flat-square" alt="npm downloads"></a>
|
|
14
|
+
<a href="https://github.com/eddiesanjuan/markupr/actions/workflows/ci.yml"><img src="https://github.com/eddiesanjuan/markupr/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
|
|
17
15
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg?style=flat-square" alt="License"></a>
|
|
18
|
-
<a href="https://
|
|
16
|
+
<a href="https://github.com/eddiesanjuan/markupr/stargazers"><img src="https://img.shields.io/github/stars/eddiesanjuan/markupr?style=flat-square" alt="GitHub stars"></a>
|
|
19
17
|
</p>
|
|
20
18
|
|
|
21
19
|
<p align="center">
|
|
22
|
-
<a href="#quick-start">Quick Start</a>
|
|
23
|
-
<a href="#why-markupr">Why markupr
|
|
24
|
-
<a href="#
|
|
25
|
-
<a href="#
|
|
26
|
-
<a href="#
|
|
27
|
-
<a href="#
|
|
28
|
-
<a href="#mcp-server-for-ai-coding-agents">MCP Server</a> |
|
|
29
|
-
<a href="#usage">Usage</a> |
|
|
30
|
-
<a href="#keyboard-shortcuts">Shortcuts</a> |
|
|
31
|
-
<a href="#export-formats">Export</a> |
|
|
32
|
-
<a href="#development">Development</a> |
|
|
33
|
-
<a href="#contributing">Contributing</a> |
|
|
34
|
-
<a href="CHANGELOG.md">Changelog</a>
|
|
20
|
+
<a href="#quick-start">Quick Start</a> ·
|
|
21
|
+
<a href="#why-markupr">Why markupr</a> ·
|
|
22
|
+
<a href="#mcp-server">MCP Server</a> ·
|
|
23
|
+
<a href="#cli">CLI</a> ·
|
|
24
|
+
<a href="#integrations">Integrations</a> ·
|
|
25
|
+
<a href="#contributing">Contributing</a>
|
|
35
26
|
</p>
|
|
36
27
|
|
|
37
28
|
---
|
|
38
29
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
One hotkey to start. One hotkey to stop. A Markdown file with your words, contextually-placed screenshots, and intelligent structure -- ready to hand to your AI coding agent, paste into a GitHub issue, or drop in a Slack thread.
|
|
42
|
-
|
|
43
|
-
## Quick Start
|
|
44
|
-
|
|
45
|
-
### CLI (for developers and AI agents)
|
|
46
|
-
```bash
|
|
47
|
-
npx markupr analyze ./recording.mov
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
Or install globally:
|
|
51
|
-
```bash
|
|
52
|
-
npm install -g markupr
|
|
53
|
-
# or
|
|
54
|
-
bun install -g markupr
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
### Desktop App
|
|
58
|
-
Download from [markupr.com](https://markupr.com) or the [releases page](https://github.com/eddiesanjuan/markupr/releases) -- available for macOS, Windows, and Linux.
|
|
59
|
-
|
|
60
|
-
**First-time setup:**
|
|
61
|
-
1. Install the application (DMG for macOS, installer for Windows)
|
|
62
|
-
2. Grant required permissions (Microphone, Screen Recording)
|
|
63
|
-
3. Press `Cmd+Shift+F` (macOS) or `Ctrl+Shift+F` (Windows) to start recording
|
|
64
|
-
4. Narrate your feedback while markupr captures screenshots at pause points
|
|
65
|
-
5. Press the hotkey again to stop -- post-processing runs automatically
|
|
66
|
-
6. Paste the file path from your clipboard into your AI coding agent
|
|
67
|
-
|
|
68
|
-
**No API key required!** markupr uses local Whisper transcription by default.
|
|
69
|
-
|
|
70
|
-
If markupr saves you hours, consider [supporting development on Ko-fi](https://ko-fi.com/eddiesanjuan).
|
|
71
|
-
|
|
72
|
-
## See It In Action
|
|
73
|
-
|
|
74
|
-
<!-- TODO: Replace with actual demo recording -->
|
|
75
|
-
> **Demo video coming soon** -- Record your screen, talk through what you see, get structured Markdown with screenshots your AI coding agent can act on. One hotkey in, one hotkey out.
|
|
30
|
+
<!-- hero-screenshot: Replace with an actual screenshot or GIF of markupr in action -->
|
|
76
31
|
|
|
77
|
-
##
|
|
78
|
-
|
|
79
|
-
AI coding agents are transforming development, but they're only as good as the context you give them. Typing out bug reports with manual screenshots is slow and lossy -- you lose the flow of what you saw and the nuance of what you meant.
|
|
80
|
-
|
|
81
|
-
markupr bridges this gap. Record your screen, narrate what you see, and markupr produces a structured Markdown document that AI agents can consume directly. The output isn't a raw transcript with random screenshots -- it's an intelligently structured document where every screenshot is placed at the exact moment you were describing it, every issue is categorized, and the format is optimized for LLM consumption.
|
|
82
|
-
|
|
83
|
-
**The workflow:**
|
|
84
|
-
1. You see a bug. Press `Cmd+Shift+F`.
|
|
85
|
-
2. Talk through what you see: "This button is hidden on mobile, and the spacing is off here..."
|
|
86
|
-
3. Press the hotkey again to stop.
|
|
87
|
-
4. Paste the file path into Claude Code, Cursor, or any AI agent.
|
|
88
|
-
5. The agent reads your structured feedback -- with screenshots -- and fixes the issues.
|
|
89
|
-
|
|
90
|
-
**What makes it different:**
|
|
91
|
-
- **Timestamp-correlated frames** -- screenshots are extracted from the exact video frame matching your narration, not taken at arbitrary intervals
|
|
92
|
-
- **Local-first** -- Whisper runs on your machine, your data stays on your machine
|
|
93
|
-
- **AI-native output** -- Markdown structured for LLM consumption, not human-only reading
|
|
94
|
-
- **Zero-friction capture** -- one global hotkey from any app, no context switching
|
|
95
|
-
|
|
96
|
-
## Features
|
|
97
|
-
|
|
98
|
-
### Voice-Driven Capture
|
|
99
|
-
- **Local Whisper transcription** runs entirely on your machine -- no API key, no internet required
|
|
100
|
-
- **Optional OpenAI cloud transcription** for higher accuracy (BYOK)
|
|
101
|
-
- **Intelligent screenshot timing** captures automatically during voice pauses
|
|
102
|
-
- **Audio waveform visualization** for real-time feedback
|
|
103
|
-
|
|
104
|
-
### Intelligent Post-Processing Pipeline
|
|
105
|
-
- **Timestamp-correlated frame extraction** -- every screenshot corresponds to what you were describing
|
|
106
|
-
- **Key-moment detection** analyzes your transcript to find the most important moments
|
|
107
|
-
- **Video frame extraction** via ffmpeg pulls precise frames from the screen recording
|
|
108
|
-
- **Structured Markdown output** optimized for LLM consumption (llms.txt inspired)
|
|
109
|
-
|
|
110
|
-
### Smart Screenshots
|
|
111
|
-
- **Voice Activity Detection (VAD)** triggers captures at natural pause points
|
|
112
|
-
- **Manual screenshot hotkey** (`Cmd+Shift+S`) for precise control
|
|
113
|
-
- **Multi-monitor support** with display selection
|
|
114
|
-
- **Window-specific capture** for focused feedback
|
|
115
|
-
|
|
116
|
-
### AI-Ready Output
|
|
117
|
-
- **Markdown format** with contextually-placed screenshots and structured feedback items
|
|
118
|
-
- **Multiple export formats**: Markdown, PDF, HTML, JSON
|
|
119
|
-
- **Clipboard bridge** -- file path is copied to clipboard so AI tools can read the full document
|
|
120
|
-
|
|
121
|
-
### Bulletproof Reliability
|
|
122
|
-
- **7-state finite state machine** with watchdog timer -- no state the app can enter and not exit
|
|
123
|
-
- **Crash recovery** with 5-second auto-save -- never lose a feedback session
|
|
124
|
-
- **Graceful degradation** -- if transcription fails, frame extraction continues; if ffmpeg is missing, transcript-only output is generated
|
|
125
|
-
- **Auto-updater** for seamless updates
|
|
126
|
-
|
|
127
|
-
### Professional Experience
|
|
128
|
-
- **Native macOS menu bar** integration (no dock icon)
|
|
129
|
-
- **Windows system tray** support
|
|
130
|
-
- **Global hotkeys** that work from any application
|
|
131
|
-
- **Annotation tools** (arrow, circle, rectangle, freehand, text)
|
|
132
|
-
- **Session history browser** with search and export
|
|
133
|
-
- **Dark/light/system theme** support
|
|
134
|
-
- **Onboarding experience** for first-run setup
|
|
135
|
-
|
|
136
|
-
## How It Works
|
|
137
|
-
|
|
138
|
-
### The Post-Processing Pipeline
|
|
32
|
+
## The Problem
|
|
139
33
|
|
|
140
|
-
When you
|
|
34
|
+
AI coding agents can't see your screen. When you find a bug, you context-switch into writing mode -- describing the layout issue in text, manually screenshotting, cropping, and dragging images into the right spot. You speak at 150 words per minute but type at 60. The context is lost in translation.
|
|
141
35
|
|
|
142
|
-
|
|
143
|
-
2. **Analyze** -- The transcript is analyzed to identify key moments, topic changes, and important observations
|
|
144
|
-
3. **Extract** -- Video frames are extracted at the exact timestamps corresponding to each key moment
|
|
145
|
-
4. **Generate** -- Everything is stitched into a structured Markdown document with screenshots placed exactly where they belong
|
|
36
|
+
## The Solution
|
|
146
37
|
|
|
147
|
-
|
|
38
|
+
markupr records your screen while you narrate what's wrong. When you stop, it runs an intelligent pipeline that correlates your transcript timestamps with the video to extract the right frames at the right moments -- then stitches everything into structured Markdown your AI agent can act on immediately.
|
|
148
39
|
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
- If your clipboard gets overwritten, the file lives on disk permanently
|
|
154
|
-
- AI tools like Claude Code can read the file path and process the full document including screenshots
|
|
155
|
-
- The file is yours -- local, private, no cloud dependency
|
|
156
|
-
|
|
157
|
-
### Session State Machine
|
|
158
|
-
|
|
159
|
-
The recording session is governed by a 7-state FSM with timeouts:
|
|
40
|
+
- **Record** -- press a hotkey, talk through what you see
|
|
41
|
+
- **Process** -- Whisper transcribes, ffmpeg extracts frames at the exact moments you described
|
|
42
|
+
- **Output** -- structured Markdown with screenshots placed exactly where they belong
|
|
160
43
|
|
|
161
44
|
```
|
|
162
|
-
|
|
163
|
-
│
|
|
164
|
-
▼
|
|
165
|
-
processing (10s timeout) ─→ complete (30s auto-idle)
|
|
166
|
-
│
|
|
167
|
-
▼
|
|
168
|
-
error (5s auto-recover)
|
|
45
|
+
Cmd+Shift+F --> talk --> Cmd+Shift+F --> Cmd+V into your agent
|
|
169
46
|
```
|
|
170
47
|
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
## Installation
|
|
174
|
-
|
|
175
|
-
### macOS
|
|
176
|
-
|
|
177
|
-
Download the `.dmg` file from the [releases page](https://github.com/eddiesanjuan/markupr/releases).
|
|
178
|
-
|
|
179
|
-
1. Open the DMG file
|
|
180
|
-
2. Drag markupr to your Applications folder
|
|
181
|
-
3. Launch markupr from Applications
|
|
182
|
-
4. Grant required permissions (Microphone, Screen Recording)
|
|
183
|
-
|
|
184
|
-
### Windows
|
|
185
|
-
|
|
186
|
-
Download the `.exe` installer from the [releases page](https://github.com/eddiesanjuan/markupr/releases).
|
|
187
|
-
|
|
188
|
-
1. Run the installer
|
|
189
|
-
2. Follow the installation wizard
|
|
190
|
-
3. Launch markupr from the Start menu
|
|
191
|
-
|
|
192
|
-
### Linux
|
|
48
|
+
## Quick Start
|
|
193
49
|
|
|
194
|
-
|
|
50
|
+
### CLI (zero install)
|
|
195
51
|
|
|
196
52
|
```bash
|
|
197
|
-
|
|
198
|
-
chmod +x markupr-*.AppImage
|
|
199
|
-
./markupr-*.AppImage
|
|
200
|
-
|
|
201
|
-
# Debian/Ubuntu
|
|
202
|
-
sudo dpkg -i markupr_*.deb
|
|
53
|
+
npx markupr analyze ./recording.mov
|
|
203
54
|
```
|
|
204
55
|
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
### Transcription
|
|
208
|
-
|
|
209
|
-
markupr works out of the box with **local Whisper** transcription -- no API key needed. On first run, you'll be prompted to download a Whisper model (~75MB for tiny, ~500MB for base).
|
|
210
|
-
|
|
211
|
-
For cloud post-session transcription with higher accuracy, add your OpenAI API key:
|
|
212
|
-
|
|
213
|
-
1. Sign up at [platform.openai.com/api-keys](https://platform.openai.com/api-keys)
|
|
214
|
-
2. Create an API key with "Usage" permissions
|
|
215
|
-
3. Open Settings > Advanced > Transcription Service
|
|
216
|
-
4. Enter your OpenAI API key
|
|
217
|
-
|
|
218
|
-
OpenAI usage is billed to your own API account.
|
|
219
|
-
|
|
220
|
-
### AI-Enhanced Analysis (Optional)
|
|
221
|
-
|
|
222
|
-
Add your Anthropic API key for Claude-powered document analysis:
|
|
223
|
-
|
|
224
|
-
1. Get an API key at [console.anthropic.com](https://console.anthropic.com/)
|
|
225
|
-
2. Open Settings > Advanced > AI Analysis
|
|
226
|
-
3. Enter your Anthropic API key
|
|
227
|
-
|
|
228
|
-
Claude analyzes your transcript alongside screenshots to produce an intelligent document -- grouping related feedback, identifying patterns, and writing actionable summaries.
|
|
229
|
-
|
|
230
|
-
### Settings Overview
|
|
231
|
-
|
|
232
|
-
| Category | Setting | Description |
|
|
233
|
-
|----------|---------|-------------|
|
|
234
|
-
| **General** | Output Directory | Where sessions are saved |
|
|
235
|
-
| | Launch at Login | Start markupr on system boot |
|
|
236
|
-
| **Recording** | Countdown | 0, 3, or 5 second countdown before recording |
|
|
237
|
-
| | Transcription Preview | Show live transcription overlay |
|
|
238
|
-
| | Audio Waveform | Visual audio level feedback |
|
|
239
|
-
| **Capture** | Pause Threshold | Voice pause duration before screenshot (500-3000ms) |
|
|
240
|
-
| | Min Time Between | Minimum gap between screenshots |
|
|
241
|
-
| **Appearance** | Theme | Dark, Light, or System |
|
|
242
|
-
| | Accent Color | Customize UI accent color |
|
|
243
|
-
| **Hotkeys** | Toggle Recording | Default: `Cmd/Ctrl+Shift+F` |
|
|
244
|
-
| | Manual Screenshot | Default: `Cmd/Ctrl+Shift+S` |
|
|
245
|
-
|
|
246
|
-
## CLI Usage
|
|
247
|
-
|
|
248
|
-
markupr can run as a standalone CLI tool -- no Electron or desktop app required. This is ideal for:
|
|
249
|
-
- CI/CD pipelines processing screen recordings
|
|
250
|
-
- AI coding agents that need to analyze recordings programmatically
|
|
251
|
-
- Developers who prefer the command line
|
|
252
|
-
|
|
253
|
-
### Installation
|
|
56
|
+
### MCP Server (for AI coding agents)
|
|
254
57
|
|
|
255
58
|
```bash
|
|
256
|
-
|
|
257
|
-
npx markupr analyze ./recording.mov
|
|
258
|
-
|
|
259
|
-
# Install globally via npm
|
|
260
|
-
npm install -g markupr
|
|
261
|
-
|
|
262
|
-
# Install globally via bun
|
|
263
|
-
bun install -g markupr
|
|
59
|
+
npx markupr-mcp
|
|
264
60
|
```
|
|
265
61
|
|
|
266
|
-
###
|
|
62
|
+
### Desktop App
|
|
267
63
|
|
|
268
|
-
|
|
64
|
+
Download from [markupr.com](https://markupr.com) or [GitHub Releases](https://github.com/eddiesanjuan/markupr/releases). Available for macOS and Windows.
|
|
269
65
|
|
|
270
|
-
|
|
66
|
+
No API keys required. Local Whisper transcription works out of the box.
|
|
271
67
|
|
|
272
|
-
|
|
273
|
-
| Flag | Description | Default |
|
|
274
|
-
|------|-------------|---------|
|
|
275
|
-
| `--audio <file>` | Separate audio file (if not embedded in video) | Auto-extracted from video |
|
|
276
|
-
| `--output <dir>` | Output directory | `./markupr-output` |
|
|
277
|
-
| `--whisper-model <path>` | Path to local Whisper model file | Auto-detected in `~/.markupr/whisper-models/` |
|
|
278
|
-
| `--no-frames` | Skip frame extraction | `false` |
|
|
279
|
-
| `--verbose` | Show detailed progress output | `false` |
|
|
68
|
+
## Why markupr?
|
|
280
69
|
|
|
281
|
-
**
|
|
70
|
+
**Local-first.** Whisper runs on your device. Your recordings, transcripts, and screenshots never leave your machine. No cloud dependency, no account required.
|
|
282
71
|
|
|
283
|
-
|
|
284
|
-
# Basic usage - analyze a screen recording
|
|
285
|
-
markupr analyze ./bug-demo.mov
|
|
72
|
+
**AI-native output.** The Markdown output is structured for LLM consumption -- headings, categories, severity levels, and inline screenshots. Not a raw transcript with random images. Every screenshot shows exactly what you were describing when you said it.
|
|
286
73
|
|
|
287
|
-
|
|
288
|
-
markupr analyze ./recording.mov --output ./reports
|
|
74
|
+
**Works everywhere.** Desktop app for daily use. CLI for scripts and CI/CD. MCP server for agent integration. GitHub Action for PR feedback. Same pipeline, four interfaces.
|
|
289
75
|
|
|
290
|
-
|
|
291
|
-
markupr analyze ./screen.mov --audio ./voiceover.wav
|
|
76
|
+
**Open source.** MIT licensed. No telemetry, no tracking, no analytics. Read the source, fork it, ship it.
|
|
292
77
|
|
|
293
|
-
|
|
294
|
-
markupr analyze ./recording.mov --no-frames
|
|
295
|
-
```
|
|
78
|
+
## What the Output Looks Like
|
|
296
79
|
|
|
297
|
-
|
|
80
|
+
```markdown
|
|
81
|
+
# Feedback Session -- Feb 5, 2026
|
|
298
82
|
|
|
299
|
-
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
- Windows: `choco install ffmpeg` or download from ffmpeg.org
|
|
83
|
+
## FB-001: Button sizing issue
|
|
84
|
+
The submit button is way too small on mobile. I'm trying to tap it
|
|
85
|
+
and keep hitting the cancel link underneath. Needs more vertical
|
|
86
|
+
padding, maybe 12px minimum tap target.
|
|
304
87
|
|
|
305
|
-
|
|
88
|
+

|
|
306
89
|
|
|
307
|
-
|
|
90
|
+
## FB-002: Loading state feels janky
|
|
91
|
+
After the spinner disappears, the content pops in with no transition.
|
|
92
|
+
There's a visible layout shift -- the sidebar jumps left by about
|
|
93
|
+
20 pixels.
|
|
308
94
|
|
|
309
|
-
|
|
310
|
-
# Install and process a recording in one command
|
|
311
|
-
npx markupr analyze ./recording.mov --output ./feedback
|
|
312
|
-
|
|
313
|
-
# The output is a structured Markdown file with embedded screenshots
|
|
314
|
-
# Perfect for feeding into Claude, GPT, or any LLM
|
|
315
|
-
cat ./feedback/markupr-report.md
|
|
95
|
+

|
|
316
96
|
```
|
|
317
97
|
|
|
318
|
-
|
|
98
|
+
Each screenshot is extracted from the exact video frame matching your narration timestamp. See full examples in [`examples/`](examples/).
|
|
319
99
|
|
|
320
|
-
## MCP Server
|
|
100
|
+
## MCP Server
|
|
321
101
|
|
|
322
|
-
|
|
102
|
+
Give your AI coding agent eyes and ears. Add markupr as an MCP server and it can capture screenshots, record your screen with voice, and receive structured reports -- all mid-conversation.
|
|
323
103
|
|
|
324
104
|
### Setup
|
|
325
105
|
|
|
326
|
-
Add to your IDE config and your agent gets 6 tools:
|
|
327
|
-
|
|
328
106
|
**Claude Code** (`~/.claude/settings.json`):
|
|
329
|
-
```json
|
|
330
|
-
{
|
|
331
|
-
"mcpServers": {
|
|
332
|
-
"markupr": {
|
|
333
|
-
"command": "npx",
|
|
334
|
-
"args": ["-y", "markupr-mcp"]
|
|
335
|
-
}
|
|
336
|
-
}
|
|
337
|
-
}
|
|
338
|
-
```
|
|
339
107
|
|
|
340
|
-
**Cursor / Windsurf** (MCP settings):
|
|
341
108
|
```json
|
|
342
109
|
{
|
|
343
110
|
"mcpServers": {
|
|
@@ -349,350 +116,163 @@ Add to your IDE config and your agent gets 6 tools:
|
|
|
349
116
|
}
|
|
350
117
|
```
|
|
351
118
|
|
|
352
|
-
|
|
119
|
+
**Cursor / Windsurf** -- same config in your MCP settings.
|
|
353
120
|
|
|
354
|
-
|
|
121
|
+
### Tools
|
|
122
|
+
|
|
123
|
+
| Tool | Description |
|
|
355
124
|
|------|-------------|
|
|
356
125
|
| `capture_screenshot` | Grab the current screen. Your agent sees what you see. |
|
|
357
|
-
| `capture_with_voice` | Record screen +
|
|
126
|
+
| `capture_with_voice` | Record screen + mic for a set duration. Returns a structured report. |
|
|
358
127
|
| `analyze_video` | Process any `.mov` or `.mp4` into Markdown with extracted frames. |
|
|
359
|
-
| `analyze_screenshot` |
|
|
128
|
+
| `analyze_screenshot` | Run a screenshot through the AI analysis pipeline. |
|
|
360
129
|
| `start_recording` | Begin an interactive recording session. |
|
|
361
|
-
| `stop_recording` | End the session
|
|
362
|
-
|
|
363
|
-
### Example: Agent Captures and Fixes a Bug
|
|
130
|
+
| `stop_recording` | End the session and run the full pipeline. |
|
|
364
131
|
|
|
365
|
-
|
|
132
|
+
### Example
|
|
366
133
|
|
|
367
134
|
```
|
|
368
135
|
You: "The sidebar is overlapping the main content on mobile. Can you see it?"
|
|
369
136
|
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
Let me fix the CSS..."
|
|
137
|
+
Agent: [calls capture_screenshot]
|
|
138
|
+
"I can see the issue -- the sidebar has position: fixed but no z-index,
|
|
139
|
+
and it's 280px wide with no responsive breakpoint. Let me fix the CSS..."
|
|
374
140
|
|
|
375
|
-
|
|
141
|
+
[fixes the code]
|
|
376
142
|
```
|
|
377
143
|
|
|
378
144
|
No copy-pasting screenshots. No describing the bug in text. The agent looks at your screen and acts.
|
|
379
145
|
|
|
380
|
-
|
|
146
|
+
Full MCP documentation: [README-MCP.md](README-MCP.md)
|
|
381
147
|
|
|
382
|
-
##
|
|
148
|
+
## CLI
|
|
383
149
|
|
|
384
|
-
###
|
|
385
|
-
|
|
386
|
-
1. **Start Recording**: Press `Cmd+Shift+F` (macOS) or `Ctrl+Shift+F` (Windows)
|
|
387
|
-
2. **Select Source**: Choose which screen or window to capture
|
|
388
|
-
3. **Narrate**: Speak naturally about what you see
|
|
389
|
-
4. **Screenshots**: Captured automatically during pauses (or press `Cmd+Shift+S` manually)
|
|
390
|
-
5. **Stop Recording**: Press the hotkey again
|
|
391
|
-
6. **Post-Processing**: Pipeline automatically transcribes, analyzes, extracts frames, and generates output
|
|
392
|
-
7. **Use Output**: File path is on your clipboard -- paste into your AI tool
|
|
393
|
-
|
|
394
|
-
### Recording Tips
|
|
395
|
-
|
|
396
|
-
- **Speak naturally** -- markupr detects pauses to time screenshots
|
|
397
|
-
- **Pause briefly** when you want a screenshot captured
|
|
398
|
-
- **Use manual capture** (`Cmd+Shift+S`) for precise timing
|
|
399
|
-
- **Review before export** to remove unwanted items
|
|
400
|
-
|
|
401
|
-
### Using with AI Coding Agents
|
|
150
|
+
### Installation
|
|
402
151
|
|
|
403
|
-
|
|
152
|
+
```bash
|
|
153
|
+
# Run without installing
|
|
154
|
+
npx markupr analyze ./recording.mov
|
|
404
155
|
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
156
|
+
# Or install globally
|
|
157
|
+
npm install -g markupr
|
|
158
|
+
```
|
|
408
159
|
|
|
409
|
-
###
|
|
160
|
+
### Commands
|
|
410
161
|
|
|
411
|
-
|
|
162
|
+
**`markupr analyze <video>`** -- Process a screen recording into structured Markdown.
|
|
412
163
|
|
|
413
164
|
```bash
|
|
414
|
-
|
|
165
|
+
markupr analyze ./bug-demo.mov
|
|
166
|
+
markupr analyze ./recording.mov --output ./reports
|
|
167
|
+
markupr analyze ./recording.mov --template github-issue
|
|
168
|
+
markupr analyze ./recording.mov --no-frames # transcript only
|
|
415
169
|
```
|
|
416
170
|
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
### Session Output
|
|
420
|
-
|
|
421
|
-
Sessions are saved to an organized folder:
|
|
171
|
+
**`markupr watch [directory]`** -- Watch for new recordings and auto-process them.
|
|
422
172
|
|
|
173
|
+
```bash
|
|
174
|
+
markupr watch ~/Desktop --output ./reports
|
|
423
175
|
```
|
|
424
|
-
~/markupr/sessions/2026-02-05_14-23-41/
|
|
425
|
-
feedback-report.md # Structured Markdown with inline screenshots
|
|
426
|
-
metadata.json # Session metadata (source, duration, environment)
|
|
427
|
-
screenshots/
|
|
428
|
-
fb-001.png # Extracted frames from key moments
|
|
429
|
-
fb-002.png
|
|
430
|
-
fb-003.png
|
|
431
|
-
session-recording.webm # Full screen recording (optional)
|
|
432
|
-
```
|
|
433
|
-
|
|
434
|
-
## Keyboard Shortcuts
|
|
435
|
-
|
|
436
|
-
### Recording
|
|
437
|
-
|
|
438
|
-
| Action | macOS | Windows/Linux |
|
|
439
|
-
|--------|-------|---------------|
|
|
440
|
-
| Start/Stop Recording | `Cmd+Shift+F` | `Ctrl+Shift+F` |
|
|
441
|
-
| Manual Screenshot | `Cmd+Shift+S` | `Ctrl+Shift+S` |
|
|
442
|
-
| Pause/Resume | `Cmd+Shift+P` | `Ctrl+Shift+P` |
|
|
443
176
|
|
|
444
|
-
|
|
177
|
+
**`markupr push github <report>`** -- Create GitHub issues from a feedback report.
|
|
445
178
|
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
| Keyboard Shortcuts | `Cmd+/` | `Ctrl+/` |
|
|
451
|
-
| Close Dialog | `Escape` | `Escape` |
|
|
452
|
-
|
|
453
|
-
### Editing
|
|
454
|
-
|
|
455
|
-
| Action | macOS | Windows/Linux |
|
|
456
|
-
|--------|-------|---------------|
|
|
457
|
-
| Delete Selected | `Backspace` | `Delete` |
|
|
458
|
-
| Edit Item | `Enter` | `Enter` |
|
|
459
|
-
| Move Up | `Cmd+Up` | `Ctrl+Up` |
|
|
460
|
-
| Move Down | `Cmd+Down` | `Ctrl+Down` |
|
|
461
|
-
| Undo | `Cmd+Z` | `Ctrl+Z` |
|
|
462
|
-
| Redo | `Cmd+Shift+Z` | `Ctrl+Shift+Z` |
|
|
463
|
-
| Select All | `Cmd+A` | `Ctrl+A` |
|
|
464
|
-
|
|
465
|
-
### Annotation Tools
|
|
466
|
-
|
|
467
|
-
| Tool | Shortcut |
|
|
468
|
-
|------|----------|
|
|
469
|
-
| Arrow | `1` |
|
|
470
|
-
| Circle | `2` |
|
|
471
|
-
| Rectangle | `3` |
|
|
472
|
-
| Freehand | `4` |
|
|
473
|
-
| Text | `5` |
|
|
474
|
-
| Clear Annotations | `Cmd/Ctrl+Backspace` |
|
|
179
|
+
```bash
|
|
180
|
+
markupr push github ./report.md --repo myorg/myapp
|
|
181
|
+
markupr push github ./report.md --repo myorg/myapp --dry-run
|
|
182
|
+
```
|
|
475
183
|
|
|
476
|
-
|
|
184
|
+
**`markupr push linear <report>`** -- Create Linear issues from a feedback report.
|
|
477
185
|
|
|
478
|
-
|
|
186
|
+
```bash
|
|
187
|
+
markupr push linear ./report.md --team ENG
|
|
188
|
+
```
|
|
479
189
|
|
|
480
|
-
|
|
190
|
+
### Output Templates
|
|
481
191
|
|
|
482
|
-
-
|
|
483
|
-
- Feedback items with categories and severity
|
|
484
|
-
- Inline screenshot references
|
|
485
|
-
- Summary table with session metadata
|
|
192
|
+
`markdown` (default) | `json` | `github-issue` | `linear` | `jira` | `html`
|
|
486
193
|
|
|
487
|
-
|
|
488
|
-
# Feedback Report: My App
|
|
194
|
+
### Requirements
|
|
489
195
|
|
|
490
|
-
|
|
491
|
-
-
|
|
492
|
-
- **Items**: 5 feedback points
|
|
493
|
-
- **Screenshots**: 3 captured
|
|
196
|
+
- Node.js 18+
|
|
197
|
+
- [ffmpeg](https://ffmpeg.org/) on your PATH (`brew install ffmpeg` / `apt install ffmpeg` / `choco install ffmpeg`)
|
|
494
198
|
|
|
495
|
-
##
|
|
199
|
+
## Integrations
|
|
496
200
|
|
|
497
|
-
###
|
|
498
|
-
**Timestamp**: 00:15 | **Type**: Bug
|
|
201
|
+
### GitHub Action
|
|
499
202
|
|
|
500
|
-
|
|
203
|
+
Run markupr in CI to get visual feedback on pull requests:
|
|
501
204
|
|
|
502
|
-
|
|
205
|
+
```yaml
|
|
206
|
+
- uses: eddiesanjuan/markupr-action@v1
|
|
207
|
+
with:
|
|
208
|
+
video-path: ./recordings/
|
|
209
|
+
github-token: ${{ secrets.GITHUB_TOKEN }}
|
|
503
210
|
```
|
|
504
211
|
|
|
505
|
-
###
|
|
506
|
-
|
|
507
|
-
Professional document with embedded screenshots, print-ready layout, and theme support.
|
|
508
|
-
|
|
509
|
-
### HTML (.html)
|
|
510
|
-
|
|
511
|
-
Self-contained web page with embedded images, dark/light theme toggle, and mobile responsive design.
|
|
212
|
+
### Desktop App Workflow
|
|
512
213
|
|
|
513
|
-
|
|
214
|
+
1. Press `Cmd+Shift+F` (macOS) or `Ctrl+Shift+F` (Windows)
|
|
215
|
+
2. Narrate what you see -- screenshots capture automatically during pauses
|
|
216
|
+
3. Press the hotkey again to stop
|
|
217
|
+
4. Paste the file path from your clipboard into Claude Code, Cursor, or any AI agent
|
|
514
218
|
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
## Architecture
|
|
219
|
+
## How It Works
|
|
518
220
|
|
|
519
221
|
```
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
│ │ ├── App.tsx # Main component
|
|
536
|
-
│ │ ├── components/ # UI components (30+)
|
|
537
|
-
│ │ ├── audio/ # Renderer-side audio bridge
|
|
538
|
-
│ │ ├── capture/ # Renderer-side screen recording
|
|
539
|
-
│ │ └── hooks/ # React hooks (theme, animation)
|
|
540
|
-
│ ├── preload/ # Electron preload (secure IPC bridge)
|
|
541
|
-
│ └── shared/ # Shared types and constants
|
|
542
|
-
├── tests/ # Test suite
|
|
543
|
-
├── docs/ # Documentation
|
|
544
|
-
├── site/ # Landing page
|
|
545
|
-
└── package.json
|
|
222
|
+
+-----------+
|
|
223
|
+
Screen + Voice -> | Whisper | -> Timestamped transcript
|
|
224
|
+
+-----------+
|
|
225
|
+
|
|
|
226
|
+
+-----------+
|
|
227
|
+
| Analyzer | -> Key moments identified
|
|
228
|
+
+-----------+
|
|
229
|
+
|
|
|
230
|
+
+-----------+
|
|
231
|
+
| ffmpeg | -> Frames extracted at exact timestamps
|
|
232
|
+
+-----------+
|
|
233
|
+
|
|
|
234
|
+
+-----------+
|
|
235
|
+
| Generator | -> Structured Markdown with inline screenshots
|
|
236
|
+
+-----------+
|
|
546
237
|
```
|
|
547
238
|
|
|
548
|
-
|
|
239
|
+
The pipeline degrades gracefully. No ffmpeg? Transcript-only output. No Whisper model? Timer-based screenshots. No API keys? Everything runs locally.
|
|
549
240
|
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
- Node.js 18+
|
|
553
|
-
- npm 9+
|
|
554
|
-
- macOS, Windows, or Linux
|
|
241
|
+
For architecture details, see [CLAUDE.md](CLAUDE.md).
|
|
555
242
|
|
|
556
|
-
|
|
243
|
+
## Development
|
|
557
244
|
|
|
558
245
|
```bash
|
|
559
|
-
# Clone the repository
|
|
560
246
|
git clone https://github.com/eddiesanjuan/markupr.git
|
|
561
247
|
cd markupr
|
|
562
|
-
```
|
|
563
|
-
|
|
564
|
-
Use either npm or bun:
|
|
565
|
-
|
|
566
|
-
```bash
|
|
567
|
-
# npm
|
|
568
248
|
npm install
|
|
569
|
-
|
|
570
|
-
# Start development mode
|
|
571
249
|
npm run dev
|
|
572
250
|
```
|
|
573
251
|
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
# Start development mode
|
|
579
|
-
bun run dev
|
|
580
|
-
```
|
|
581
|
-
|
|
582
|
-
### Scripts
|
|
583
|
-
|
|
584
|
-
| Script | Description |
|
|
585
|
-
|--------|-------------|
|
|
586
|
-
| `npm run dev` | Start in development mode with hot reload |
|
|
587
|
-
| `npm run build` | Build for production |
|
|
588
|
-
| `npm run build:desktop` | Build desktop app only |
|
|
589
|
-
| `npm run package` | Package for current platform |
|
|
590
|
-
| `npm run package:mac` | Package for macOS |
|
|
591
|
-
| `npm run package:win` | Package for Windows |
|
|
592
|
-
| `npm run package:linux` | Package for Linux |
|
|
252
|
+
| Command | Description |
|
|
253
|
+
|---------|-------------|
|
|
254
|
+
| `npm run dev` | Development mode with hot reload |
|
|
255
|
+
| `npm run build` | Build everything (desktop + CLI + MCP) |
|
|
593
256
|
| `npm test` | Run all tests |
|
|
594
|
-
| `npm run
|
|
595
|
-
| `npm run
|
|
596
|
-
| `npm run lint` | Lint code |
|
|
597
|
-
| `npm run lint:fix` | Auto-fix lint issues |
|
|
598
|
-
| `npm run typecheck` | TypeScript type checking |
|
|
599
|
-
|
|
600
|
-
You can run the same scripts with bun (`bun run dev`, `bun run test`, etc.).
|
|
601
|
-
|
|
602
|
-
### Running Tests
|
|
603
|
-
|
|
604
|
-
```bash
|
|
605
|
-
# Run all tests
|
|
606
|
-
npm test
|
|
607
|
-
|
|
608
|
-
# Run with coverage
|
|
609
|
-
npm run test:coverage
|
|
610
|
-
|
|
611
|
-
# Run specific test file
|
|
612
|
-
npx vitest tests/unit/postProcessor.test.ts
|
|
613
|
-
```
|
|
257
|
+
| `npm run lint` | Lint |
|
|
258
|
+
| `npm run typecheck` | Type check |
|
|
614
259
|
|
|
615
260
|
## Contributing
|
|
616
261
|
|
|
617
|
-
We welcome contributions! markupr is MIT licensed and community-driven.
|
|
618
|
-
|
|
619
|
-
### Quick Start for Contributors
|
|
620
|
-
|
|
621
262
|
1. Fork the repository
|
|
622
|
-
2. Create a feature branch: `git checkout -b feature/
|
|
623
|
-
3.
|
|
624
|
-
4.
|
|
625
|
-
5. Run lint: `npm run lint`
|
|
626
|
-
6. Commit: `git commit -m 'Add amazing feature'`
|
|
627
|
-
7. Push: `git push origin feature/amazing-feature`
|
|
628
|
-
8. Open a Pull Request
|
|
629
|
-
|
|
630
|
-
### Guidelines
|
|
631
|
-
|
|
632
|
-
- **Tests required** for new functionality
|
|
633
|
-
- **Lint clean** -- run `npm run lint` before submitting
|
|
634
|
-
- **TypeScript strict** -- run `npm run typecheck`
|
|
635
|
-
- **Small PRs preferred** -- focused changes are easier to review
|
|
636
|
-
- **Follow existing patterns** -- check CLAUDE.md for architecture details
|
|
263
|
+
2. Create a feature branch: `git checkout -b feature/your-feature`
|
|
264
|
+
3. Run tests: `npm test && npm run lint && npm run typecheck`
|
|
265
|
+
4. Open a Pull Request
|
|
637
266
|
|
|
638
267
|
See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines.
|
|
639
268
|
|
|
640
|
-
## Troubleshooting
|
|
641
|
-
|
|
642
|
-
### Common Issues
|
|
643
|
-
|
|
644
|
-
**Microphone not detected**
|
|
645
|
-
- Check System Preferences > Security & Privacy > Microphone
|
|
646
|
-
- Grant markupr microphone access
|
|
647
|
-
|
|
648
|
-
**Screen recording permission denied**
|
|
649
|
-
- macOS: System Preferences > Security & Privacy > Screen Recording
|
|
650
|
-
- Restart markupr after granting permission
|
|
651
|
-
|
|
652
|
-
**Whisper model download failing**
|
|
653
|
-
- Check your internet connection
|
|
654
|
-
- Try a smaller model first (tiny: ~75MB)
|
|
655
|
-
- Downloads support resume if interrupted
|
|
656
|
-
|
|
657
|
-
**OpenAI API connection failed**
|
|
658
|
-
- Verify your API key is correct
|
|
659
|
-
- Check your internet connection
|
|
660
|
-
- Ensure your OpenAI project has billing enabled
|
|
661
|
-
|
|
662
|
-
**Hotkeys not working**
|
|
663
|
-
- Check for conflicts with other applications
|
|
664
|
-
- Try customizing hotkeys in Settings > Hotkeys
|
|
665
|
-
|
|
666
|
-
**ffmpeg not found for frame extraction**
|
|
667
|
-
- Install ffmpeg: `brew install ffmpeg` (macOS) or download from [ffmpeg.org](https://ffmpeg.org/)
|
|
668
|
-
- markupr gracefully degrades to transcript-only output without ffmpeg
|
|
669
|
-
|
|
670
|
-
See [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) for more solutions.
|
|
671
|
-
|
|
672
|
-
## Support markupr
|
|
673
|
-
|
|
674
|
-
If markupr is useful in your workflow and you want to keep it improving quickly:
|
|
675
|
-
|
|
676
|
-
- [Support on Ko-fi](https://ko-fi.com/eddiesanjuan)
|
|
677
|
-
- Share the project with a teammate who writes bug reports
|
|
678
|
-
- Open issues with reproducible feedback sessions so fixes land faster
|
|
679
|
-
|
|
680
269
|
## License
|
|
681
270
|
|
|
682
|
-
MIT
|
|
683
|
-
|
|
684
|
-
## Acknowledgments
|
|
685
|
-
|
|
686
|
-
- [Whisper](https://github.com/openai/whisper) for local speech recognition
|
|
687
|
-
- [Anthropic Claude](https://anthropic.com) for AI-enhanced document analysis
|
|
688
|
-
- [OpenAI](https://platform.openai.com) for cloud transcription
|
|
689
|
-
- [Electron](https://electronjs.org) for cross-platform desktop framework
|
|
690
|
-
- [React](https://reactjs.org) for the UI framework
|
|
691
|
-
- [Vite](https://vitejs.dev) for blazing fast builds
|
|
271
|
+
MIT -- see [LICENSE](LICENSE).
|
|
692
272
|
|
|
693
273
|
---
|
|
694
274
|
|
|
695
275
|
<p align="center">
|
|
696
|
-
Built by <a href="https://github.com/eddiesanjuan">Eddie San Juan</a
|
|
697
|
-
<a href="https://
|
|
276
|
+
Built by <a href="https://github.com/eddiesanjuan">Eddie San Juan</a><br>
|
|
277
|
+
<a href="https://markupr.com">markupr.com</a>
|
|
698
278
|
</p>
|