markupr 2.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (299) hide show
  1. package/.claude/commands/review-feedback.md +47 -0
  2. package/.eslintrc.json +35 -0
  3. package/.github/CODEOWNERS +16 -0
  4. package/.github/FUNDING.yml +1 -0
  5. package/.github/ISSUE_TEMPLATE/bug_report.md +56 -0
  6. package/.github/ISSUE_TEMPLATE/feature_request.md +54 -0
  7. package/.github/PULL_REQUEST_TEMPLATE.md +89 -0
  8. package/.github/dependabot.yml +70 -0
  9. package/.github/workflows/ci.yml +184 -0
  10. package/.github/workflows/deploy-landing.yml +134 -0
  11. package/.github/workflows/nightly.yml +288 -0
  12. package/.github/workflows/release.yml +318 -0
  13. package/CHANGELOG.md +127 -0
  14. package/CLAUDE.md +137 -0
  15. package/CODE_OF_CONDUCT.md +9 -0
  16. package/CONTRIBUTING.md +390 -0
  17. package/LICENSE +21 -0
  18. package/PRODUCT_VISION.md +277 -0
  19. package/README.md +517 -0
  20. package/SECURITY.md +51 -0
  21. package/SIGNING_INSTRUCTIONS.md +284 -0
  22. package/assets/DMG_BACKGROUND_INSTRUCTIONS.md +130 -0
  23. package/assets/svg-source/dmg-background.svg +70 -0
  24. package/assets/svg-source/icon.svg +20 -0
  25. package/assets/svg-source/tray-icon-processing.svg +7 -0
  26. package/assets/svg-source/tray-icon-recording.svg +7 -0
  27. package/assets/svg-source/tray-icon.svg +6 -0
  28. package/assets/tray-complete.png +0 -0
  29. package/assets/tray-complete@2x.png +0 -0
  30. package/assets/tray-completeTemplate.png +0 -0
  31. package/assets/tray-completeTemplate@2x.png +0 -0
  32. package/assets/tray-error.png +0 -0
  33. package/assets/tray-error@2x.png +0 -0
  34. package/assets/tray-errorTemplate.png +0 -0
  35. package/assets/tray-errorTemplate@2x.png +0 -0
  36. package/assets/tray-icon-processing.png +0 -0
  37. package/assets/tray-icon-processing@2x.png +0 -0
  38. package/assets/tray-icon-processingTemplate.png +0 -0
  39. package/assets/tray-icon-processingTemplate@2x.png +0 -0
  40. package/assets/tray-icon-recording.png +0 -0
  41. package/assets/tray-icon-recording@2x.png +0 -0
  42. package/assets/tray-icon-recordingTemplate.png +0 -0
  43. package/assets/tray-icon-recordingTemplate@2x.png +0 -0
  44. package/assets/tray-icon.png +0 -0
  45. package/assets/tray-icon@2x.png +0 -0
  46. package/assets/tray-iconTemplate.png +0 -0
  47. package/assets/tray-iconTemplate@2x.png +0 -0
  48. package/assets/tray-idle.png +0 -0
  49. package/assets/tray-idle@2x.png +0 -0
  50. package/assets/tray-idleTemplate.png +0 -0
  51. package/assets/tray-idleTemplate@2x.png +0 -0
  52. package/assets/tray-processing-0.png +0 -0
  53. package/assets/tray-processing-0@2x.png +0 -0
  54. package/assets/tray-processing-0Template.png +0 -0
  55. package/assets/tray-processing-0Template@2x.png +0 -0
  56. package/assets/tray-processing-1.png +0 -0
  57. package/assets/tray-processing-1@2x.png +0 -0
  58. package/assets/tray-processing-1Template.png +0 -0
  59. package/assets/tray-processing-1Template@2x.png +0 -0
  60. package/assets/tray-processing-2.png +0 -0
  61. package/assets/tray-processing-2@2x.png +0 -0
  62. package/assets/tray-processing-2Template.png +0 -0
  63. package/assets/tray-processing-2Template@2x.png +0 -0
  64. package/assets/tray-processing-3.png +0 -0
  65. package/assets/tray-processing-3@2x.png +0 -0
  66. package/assets/tray-processing-3Template.png +0 -0
  67. package/assets/tray-processing-3Template@2x.png +0 -0
  68. package/assets/tray-processing.png +0 -0
  69. package/assets/tray-processing@2x.png +0 -0
  70. package/assets/tray-processingTemplate.png +0 -0
  71. package/assets/tray-processingTemplate@2x.png +0 -0
  72. package/assets/tray-recording.png +0 -0
  73. package/assets/tray-recording@2x.png +0 -0
  74. package/assets/tray-recordingTemplate.png +0 -0
  75. package/assets/tray-recordingTemplate@2x.png +0 -0
  76. package/build/DMG_BACKGROUND_SPEC.md +50 -0
  77. package/build/dmg-background.png +0 -0
  78. package/build/dmg-background@2x.png +0 -0
  79. package/build/entitlements.mac.inherit.plist +27 -0
  80. package/build/entitlements.mac.plist +41 -0
  81. package/build/favicon-16.png +0 -0
  82. package/build/favicon-180.png +0 -0
  83. package/build/favicon-192.png +0 -0
  84. package/build/favicon-32.png +0 -0
  85. package/build/favicon-48.png +0 -0
  86. package/build/favicon-512.png +0 -0
  87. package/build/favicon-64.png +0 -0
  88. package/build/icon-128.png +0 -0
  89. package/build/icon-16.png +0 -0
  90. package/build/icon-24.png +0 -0
  91. package/build/icon-256.png +0 -0
  92. package/build/icon-32.png +0 -0
  93. package/build/icon-48.png +0 -0
  94. package/build/icon-64.png +0 -0
  95. package/build/icon.icns +0 -0
  96. package/build/icon.ico +0 -0
  97. package/build/icon.iconset/icon_128x128.png +0 -0
  98. package/build/icon.iconset/icon_128x128@2x.png +0 -0
  99. package/build/icon.iconset/icon_16x16.png +0 -0
  100. package/build/icon.iconset/icon_16x16@2x.png +0 -0
  101. package/build/icon.iconset/icon_256x256.png +0 -0
  102. package/build/icon.iconset/icon_256x256@2x.png +0 -0
  103. package/build/icon.iconset/icon_32x32.png +0 -0
  104. package/build/icon.iconset/icon_32x32@2x.png +0 -0
  105. package/build/icon.iconset/icon_512x512.png +0 -0
  106. package/build/icon.iconset/icon_512x512@2x.png +0 -0
  107. package/build/icon.png +0 -0
  108. package/build/installer-header.bmp +0 -0
  109. package/build/installer-header.png +0 -0
  110. package/build/installer-sidebar.bmp +0 -0
  111. package/build/installer-sidebar.png +0 -0
  112. package/build/installer.nsh +45 -0
  113. package/build/overlay-processing.png +0 -0
  114. package/build/overlay-recording.png +0 -0
  115. package/build/toolbar-record.png +0 -0
  116. package/build/toolbar-screenshot.png +0 -0
  117. package/build/toolbar-settings.png +0 -0
  118. package/build/toolbar-stop.png +0 -0
  119. package/dist/main/index.mjs +12612 -0
  120. package/dist/preload/index.mjs +907 -0
  121. package/dist/renderer/assets/index-CCmUjl9K.js +19495 -0
  122. package/dist/renderer/assets/index-CUqz_Gs6.css +2270 -0
  123. package/dist/renderer/index.html +27 -0
  124. package/docs/AI_AGENT_QUICKSTART.md +42 -0
  125. package/docs/AI_PIPELINE_DESIGN.md +595 -0
  126. package/docs/API.md +514 -0
  127. package/docs/ARCHITECTURE.md +460 -0
  128. package/docs/CONFIGURATION.md +336 -0
  129. package/docs/DEVELOPMENT.md +508 -0
  130. package/docs/EXPORT_FORMATS.md +451 -0
  131. package/docs/GETTING_STARTED.md +236 -0
  132. package/docs/KEYBOARD_SHORTCUTS.md +334 -0
  133. package/docs/TROUBLESHOOTING.md +418 -0
  134. package/docs/landing/index.html +672 -0
  135. package/docs/landing/script.js +342 -0
  136. package/docs/landing/styles.css +1543 -0
  137. package/electron-builder.yml +140 -0
  138. package/electron.vite.config.ts +63 -0
  139. package/package.json +108 -0
  140. package/railway.json +12 -0
  141. package/scripts/build.mjs +51 -0
  142. package/scripts/generate-icons.mjs +314 -0
  143. package/scripts/generate-installer-images.cjs +253 -0
  144. package/scripts/generate-tray-icons.mjs +258 -0
  145. package/scripts/notarize.cjs +180 -0
  146. package/scripts/one-click-clean-test.sh +147 -0
  147. package/scripts/postinstall.mjs +36 -0
  148. package/scripts/setup-markupr.sh +55 -0
  149. package/setup +17 -0
  150. package/site/index.html +1835 -0
  151. package/site/package.json +11 -0
  152. package/site/railway.json +12 -0
  153. package/site/server.js +31 -0
  154. package/src/main/AutoUpdater.ts +392 -0
  155. package/src/main/CrashRecovery.ts +655 -0
  156. package/src/main/ErrorHandler.ts +703 -0
  157. package/src/main/HotkeyManager.ts +399 -0
  158. package/src/main/MenuManager.ts +529 -0
  159. package/src/main/PermissionManager.ts +420 -0
  160. package/src/main/SessionController.ts +1465 -0
  161. package/src/main/TrayManager.ts +540 -0
  162. package/src/main/ai/AIPipelineManager.ts +199 -0
  163. package/src/main/ai/ClaudeAnalyzer.ts +339 -0
  164. package/src/main/ai/ImageOptimizer.ts +176 -0
  165. package/src/main/ai/StructuredMarkdownBuilder.ts +379 -0
  166. package/src/main/ai/index.ts +16 -0
  167. package/src/main/ai/types.ts +258 -0
  168. package/src/main/analysis/ClarificationGenerator.ts +385 -0
  169. package/src/main/analysis/FeedbackAnalyzer.ts +531 -0
  170. package/src/main/analysis/index.ts +19 -0
  171. package/src/main/audio/AudioCapture.ts +978 -0
  172. package/src/main/audio/audioUtils.ts +100 -0
  173. package/src/main/audio/index.ts +20 -0
  174. package/src/main/capture/index.ts +1 -0
  175. package/src/main/index.ts +1693 -0
  176. package/src/main/ipc/captureHandlers.ts +272 -0
  177. package/src/main/ipc/index.ts +45 -0
  178. package/src/main/ipc/outputHandlers.ts +302 -0
  179. package/src/main/ipc/sessionHandlers.ts +56 -0
  180. package/src/main/ipc/settingsHandlers.ts +471 -0
  181. package/src/main/ipc/types.ts +56 -0
  182. package/src/main/ipc/windowHandlers.ts +277 -0
  183. package/src/main/output/ClipboardService.ts +369 -0
  184. package/src/main/output/ExportService.ts +539 -0
  185. package/src/main/output/FileManager.ts +416 -0
  186. package/src/main/output/MarkdownGenerator.ts +791 -0
  187. package/src/main/output/MarkdownPatcher.ts +299 -0
  188. package/src/main/output/index.ts +186 -0
  189. package/src/main/output/sessionAdapter.ts +207 -0
  190. package/src/main/output/templates/html-template.ts +553 -0
  191. package/src/main/pipeline/FrameExtractor.ts +330 -0
  192. package/src/main/pipeline/PostProcessor.ts +399 -0
  193. package/src/main/pipeline/TranscriptAnalyzer.ts +226 -0
  194. package/src/main/pipeline/index.ts +36 -0
  195. package/src/main/platform/WindowsTaskbar.ts +600 -0
  196. package/src/main/platform/index.ts +16 -0
  197. package/src/main/settings/SettingsManager.ts +730 -0
  198. package/src/main/settings/index.ts +19 -0
  199. package/src/main/transcription/ModelDownloadManager.ts +494 -0
  200. package/src/main/transcription/TierManager.ts +219 -0
  201. package/src/main/transcription/TranscriptionRecoveryService.ts +340 -0
  202. package/src/main/transcription/WhisperService.ts +748 -0
  203. package/src/main/transcription/index.ts +56 -0
  204. package/src/main/transcription/types.ts +135 -0
  205. package/src/main/windows/PopoverManager.ts +284 -0
  206. package/src/main/windows/TaskbarIntegration.ts +452 -0
  207. package/src/main/windows/index.ts +23 -0
  208. package/src/preload/index.ts +1047 -0
  209. package/src/renderer/App.tsx +515 -0
  210. package/src/renderer/AppWrapper.tsx +28 -0
  211. package/src/renderer/assets/logo-dark.svg +7 -0
  212. package/src/renderer/assets/logo.svg +7 -0
  213. package/src/renderer/audio/AudioCaptureRenderer.ts +454 -0
  214. package/src/renderer/capture/ScreenRecordingRenderer.ts +492 -0
  215. package/src/renderer/components/AnnotationOverlay.tsx +836 -0
  216. package/src/renderer/components/AudioWaveform.tsx +811 -0
  217. package/src/renderer/components/ClarificationQuestions.tsx +656 -0
  218. package/src/renderer/components/CountdownTimer.tsx +495 -0
  219. package/src/renderer/components/CrashRecoveryDialog.tsx +632 -0
  220. package/src/renderer/components/DonateButton.tsx +127 -0
  221. package/src/renderer/components/ErrorBoundary.tsx +308 -0
  222. package/src/renderer/components/ExportDialog.tsx +872 -0
  223. package/src/renderer/components/HotkeyHint.tsx +261 -0
  224. package/src/renderer/components/KeyboardShortcuts.tsx +787 -0
  225. package/src/renderer/components/ModelDownloadDialog.tsx +844 -0
  226. package/src/renderer/components/Onboarding.tsx +1830 -0
  227. package/src/renderer/components/ProcessingOverlay.tsx +157 -0
  228. package/src/renderer/components/RecordingOverlay.tsx +423 -0
  229. package/src/renderer/components/SessionHistory.tsx +1746 -0
  230. package/src/renderer/components/SessionReview.tsx +1321 -0
  231. package/src/renderer/components/SettingsPanel.tsx +217 -0
  232. package/src/renderer/components/Skeleton.tsx +347 -0
  233. package/src/renderer/components/StatusIndicator.tsx +86 -0
  234. package/src/renderer/components/ThemeProvider.tsx +429 -0
  235. package/src/renderer/components/Tooltip.tsx +370 -0
  236. package/src/renderer/components/TranscriptionPreview.tsx +183 -0
  237. package/src/renderer/components/TranscriptionTierSelector.tsx +640 -0
  238. package/src/renderer/components/UpdateNotification.tsx +377 -0
  239. package/src/renderer/components/WindowSelector.tsx +947 -0
  240. package/src/renderer/components/index.ts +99 -0
  241. package/src/renderer/components/primitives/ApiKeyInput.tsx +98 -0
  242. package/src/renderer/components/primitives/ColorPicker.tsx +65 -0
  243. package/src/renderer/components/primitives/DangerButton.tsx +45 -0
  244. package/src/renderer/components/primitives/DirectoryPicker.tsx +41 -0
  245. package/src/renderer/components/primitives/Dropdown.tsx +34 -0
  246. package/src/renderer/components/primitives/KeyRecorder.tsx +117 -0
  247. package/src/renderer/components/primitives/SettingsSection.tsx +32 -0
  248. package/src/renderer/components/primitives/Slider.tsx +43 -0
  249. package/src/renderer/components/primitives/Toggle.tsx +36 -0
  250. package/src/renderer/components/primitives/index.ts +10 -0
  251. package/src/renderer/components/settings/AdvancedTab.tsx +174 -0
  252. package/src/renderer/components/settings/AppearanceTab.tsx +77 -0
  253. package/src/renderer/components/settings/GeneralTab.tsx +40 -0
  254. package/src/renderer/components/settings/HotkeysTab.tsx +79 -0
  255. package/src/renderer/components/settings/RecordingTab.tsx +84 -0
  256. package/src/renderer/components/settings/index.ts +9 -0
  257. package/src/renderer/components/settings/settingsStyles.ts +673 -0
  258. package/src/renderer/components/settings/tabConfig.tsx +85 -0
  259. package/src/renderer/components/settings/useSettingsPanel.ts +447 -0
  260. package/src/renderer/contexts/ProcessingContext.tsx +227 -0
  261. package/src/renderer/contexts/RecordingContext.tsx +683 -0
  262. package/src/renderer/contexts/UIContext.tsx +326 -0
  263. package/src/renderer/contexts/index.ts +24 -0
  264. package/src/renderer/donateMessages.ts +69 -0
  265. package/src/renderer/hooks/index.ts +75 -0
  266. package/src/renderer/hooks/useAnimation.tsx +544 -0
  267. package/src/renderer/hooks/useTheme.ts +313 -0
  268. package/src/renderer/index.html +26 -0
  269. package/src/renderer/main.tsx +52 -0
  270. package/src/renderer/styles/animations.css +1093 -0
  271. package/src/renderer/styles/app-shell.css +662 -0
  272. package/src/renderer/styles/globals.css +515 -0
  273. package/src/renderer/styles/theme.ts +578 -0
  274. package/src/renderer/types/electron.d.ts +385 -0
  275. package/src/shared/hotkeys.ts +283 -0
  276. package/src/shared/types.ts +809 -0
  277. package/tests/clipboard.test.ts +228 -0
  278. package/tests/e2e/criticalPaths.test.ts +594 -0
  279. package/tests/feedbackAnalyzer.test.ts +303 -0
  280. package/tests/integration/sessionFlow.test.ts +583 -0
  281. package/tests/markdownGenerator.test.ts +418 -0
  282. package/tests/output.test.ts +96 -0
  283. package/tests/setup.ts +486 -0
  284. package/tests/unit/appIntegration.test.ts +676 -0
  285. package/tests/unit/appViewState.test.ts +281 -0
  286. package/tests/unit/audioIpcChannels.test.ts +17 -0
  287. package/tests/unit/exportService.test.ts +492 -0
  288. package/tests/unit/hotkeys.test.ts +92 -0
  289. package/tests/unit/navigationPreload.test.ts +94 -0
  290. package/tests/unit/onboardingFlow.test.ts +345 -0
  291. package/tests/unit/permissionManager.test.ts +175 -0
  292. package/tests/unit/permissionManagerExpanded.test.ts +296 -0
  293. package/tests/unit/screenRecordingRenderer.test.ts +368 -0
  294. package/tests/unit/sessionController.test.ts +515 -0
  295. package/tests/unit/tierManager.test.ts +61 -0
  296. package/tests/unit/tierManagerExpanded.test.ts +142 -0
  297. package/tests/unit/transcriptAnalyzer.test.ts +64 -0
  298. package/tsconfig.json +25 -0
  299. package/vitest.config.ts +46 -0
package/dist/renderer/index.html ADDED
@@ -0,0 +1,27 @@
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8" />
5
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
6
+ <meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'" />
7
+ <title>markupr</title>
8
+ <style>
9
+ * {
10
+ margin: 0;
11
+ padding: 0;
12
+ box-sizing: border-box;
13
+ }
14
+ html, body, #root {
15
+ width: 100%;
16
+ height: 100%;
17
+ overflow: hidden;
18
+ background: transparent;
19
+ }
20
+ </style>
21
+ <script type="module" crossorigin src="./assets/index-CCmUjl9K.js"></script>
22
+ <link rel="stylesheet" crossorigin href="./assets/index-CUqz_Gs6.css">
23
+ </head>
24
+ <body>
25
+ <div id="root"></div>
26
+ </body>
27
+ </html>
package/docs/AI_AGENT_QUICKSTART.md ADDED
@@ -0,0 +1,42 @@
1
+ # AI Agent Quickstart
2
+
3
+ This repo includes a one-command setup path for coding agents.
4
+
5
+ ## Command
6
+
7
+ From repo root:
8
+
9
+ ```bash
10
+ ./setup markupr
11
+ ```
12
+
13
+ This command:
14
+ - validates Node.js version (18+)
15
+ - installs dependencies if missing
16
+ - runs `npm run typecheck`
17
+
18
+ ## Fast Variants
19
+
20
+ ```bash
21
+ ./setup markupr --skip-install
22
+ ./setup markupr --skip-checks
23
+ ```
24
+
25
+ ## First Run Test Loop
26
+
27
+ After setup:
28
+
29
+ ```bash
30
+ npm run dev
31
+ # or
32
+ bun run dev
33
+ ./scripts/one-click-clean-test.sh --skip-checks
34
+ ```
35
+
36
+ ## BYOK Requirements (Open Source Mode)
37
+
38
+ For full report quality in this repo version:
39
+ - OpenAI API key (transcription)
40
+ - Anthropic API key (analysis)
41
+
42
+ Keys are configured in-app under `Settings > Advanced`.
package/docs/AI_PIPELINE_DESIGN.md ADDED
@@ -0,0 +1,595 @@
1
+ # AI Analysis Pipeline Design Document
2
+
3
+ **Author**: AI Pipeline Architect
4
+ **Date**: February 2026
5
+ **Status**: Design Complete - Ready for Implementation
6
+
7
+ ---
8
+
9
+ ## 1. Architecture Overview
10
+
11
+ ### Data Flow
12
+
13
+ ```
14
+ Session End (Cmd+Shift+F)
15
+ |
16
+ v
17
+ +-------------------+
18
+ | SessionController | Produces: Session object
19
+ | stop() returns | - feedbackItems[] (text + screenshots w/ buffers)
20
+ | | - transcriptBuffer[] (all transcript events)
21
+ | | - screenshotBuffer[] (all screenshots)
22
+ | | - metadata (sourceId, sourceName, recordingPath)
23
+ +-------------------+
24
+ |
25
+ v
26
+ +-------------------+
27
+ | AIPipelineManager | NEW MODULE - Orchestrates the pipeline
28
+ | (main process) |
29
+ +-------------------+
30
+ |
31
+ +----+----+
32
+ | |
33
+ v v
34
+ [FREE] [PREMIUM / BYOK]
35
+ | |
36
+ v v
37
+ +----------+ +------------------+
38
+ | Current | | ClaudeAnalyzer | NEW MODULE
39
+ | Pipeline | | - Builds prompt |
40
+ | (as-is) | | - Sends to API |
41
+ | Adapter | | - Parses result |
42
+ | + MD Gen | +------------------+
43
+ +----------+ |
44
+ | v
45
+ | +------------------+
46
+ | | Structured |
47
+ | | MarkdownBuilder | NEW MODULE
48
+ | | Assembles final |
49
+ | | output from AI |
50
+ | | analysis result |
51
+ | +------------------+
52
+ | |
53
+ v v
54
+ +-------------------+
55
+ | FileManager | Saves to disk (existing)
56
+ | saveSession() | ~/markupr/sessions/
57
+ +-------------------+
58
+ |
59
+ v
60
+ Clipboard: file path
61
+ Notification: "Session complete"
62
+ ```
63
+
64
+ ### Where Processing Happens
65
+
66
+ **Main process.** The AI pipeline runs in the Electron main process, not a separate worker or cloud function. Rationale:
67
+
68
+ 1. **Simplicity** - No IPC overhead between workers. The main process already has full access to session data including screenshot buffers.
69
+ 2. **Access to Node.js APIs** - Direct file system access for reading screenshot buffers, no serialization cost.
70
+ 3. **Non-blocking** - The API call is async/await. The main process UI (tray icon) simply shows "processing" state while waiting. Electron's event loop is not blocked.
71
+ 4. **Consistency** - Same process that handles all other session lifecycle stages.
72
+
73
+ The Claude API call is the only network operation and is inherently async. Processing time is dominated by the API round-trip, not local compute.
74
+
75
+ ---
76
+
77
+ ## 2. Claude API Integration
78
+
79
+ ### Model Selection
80
+
81
+ **Model**: `claude-sonnet-4-5-20250929` (Claude Sonnet 4.5)
82
+
83
+ Rationale:
84
+ - Vision capability (required for screenshot analysis)
85
+ - Fast enough for interactive use (5-15s response time)
86
+ - Significantly cheaper than Opus for a consumer product
87
+ - Excellent at structured output and following formatting instructions
88
+ - Smart enough to identify patterns, group feedback, and write concise summaries
89
+
90
+ Opus is overkill for this use case. Sonnet 4.5 provides the right quality/cost/speed balance for a ~$12/month consumer product.
91
+
92
+ ### System Prompt Design
93
+
94
+ ```
95
+ You are markupr's AI analysis engine. You receive a developer's voice-narrated feedback session: a transcript of everything they said while reviewing software, paired with screenshots captured at natural pause points.
96
+
97
+ Your job is to transform this raw narration into a structured, actionable feedback document.
98
+
99
+ ## Rules
100
+
101
+ 1. **Preserve the user's voice.** Quote their exact words in blockquotes. Never rephrase their observations.
102
+ 2. **Group related feedback.** If the user mentions the same area multiple times, combine those into one item.
103
+ 3. **Match screenshots to feedback.** Each screenshot was captured during or after the text segment it accompanies. Reference screenshots by their index (e.g., [Screenshot 1]).
104
+ 4. **Extract action items.** For each feedback item, write a concrete 1-sentence action item a developer could act on immediately.
105
+ 5. **Assign priority.** Use Critical/High/Medium/Low based on the severity of the issue described.
106
+ 6. **Categorize.** Use exactly one of: Bug, UX Issue, Performance, Suggestion, Question, Positive Note.
107
+ 7. **Write a summary.** 2-3 sentences capturing the most important findings.
108
+ 8. **Be concise.** Developers will paste this into AI coding tools. Every word must earn its place.
109
+
110
+ ## Output Format
111
+
112
+ Respond with ONLY valid JSON matching this schema:
113
+
114
+ {
115
+ "summary": "2-3 sentence overview of key findings",
116
+ "items": [
117
+ {
118
+ "title": "Short descriptive title (5-10 words)",
119
+ "category": "Bug|UX Issue|Performance|Suggestion|Question|Positive Note",
120
+ "priority": "Critical|High|Medium|Low",
121
+ "quote": "User's exact words (the relevant excerpt)",
122
+ "screenshotIndices": [0, 1],
123
+ "actionItem": "Concrete 1-sentence action for a developer",
124
+ "area": "Component or area of the app this relates to (e.g., 'Navigation', 'Login Form', 'Dashboard')"
125
+ }
126
+ ],
127
+ "themes": ["theme1", "theme2"],
128
+ "positiveNotes": ["Things the user explicitly praised"],
129
+ "metadata": {
130
+ "totalItems": 5,
131
+ "criticalCount": 1,
132
+ "highCount": 2
133
+ }
134
+ }
135
+ ```
136
+
137
+ ### User Message Construction
138
+
139
+ The user message is constructed dynamically from session data:
140
+
141
+ ```
142
+ ## Transcript
143
+
144
+ The user narrated the following while reviewing the application "{sourceName}":
145
+
146
+ {segments}
147
+
148
+ ---
149
+
150
+ ## Screenshots
151
+
152
+ {screenshotCount} screenshots were captured at natural pause points during narration.
153
+ They are provided as images below in chronological order.
154
+
155
+ Screenshot 1 (captured at 00:12):
156
+ [image]
157
+
158
+ Screenshot 2 (captured at 00:34):
159
+ [image]
160
+
161
+ ...
162
+ ```
163
+
164
+ ### Vision Usage
165
+
166
+ Screenshots are sent as base64 PNG images using Claude's vision API. Each screenshot is included as a content block with `type: "image"` and `media_type: "image/png"`.
167
+
168
+ **Image optimization before sending:**
169
+ - Resize to max 1568px on the longest edge (Claude's recommended limit)
170
+ - Convert to JPEG at 80% quality if the PNG exceeds 500KB (reduces tokens significantly)
171
+ - Strip EXIF/metadata
172
+
173
+ This is done locally in the main process using Electron's `nativeImage` before the API call.
174
+
175
+ ### Screenshot Selection Strategy
176
+
177
+ **Send all silence-triggered screenshots** (up to 20). Rationale:
178
+ - A typical 5-minute session captures 5-10 screenshots
179
+ - Even a 15-minute session rarely exceeds 20 screenshots
180
+ - Claude's vision pricing makes 10-20 images affordable per session
181
+ - Sending all screenshots lets Claude make the best grouping decisions
182
+
183
+ **If > 20 screenshots:**
184
+ 1. Always include manual/voice-command triggered screenshots (user explicitly wanted these)
185
+ 2. From the remaining pause-triggered ones, select evenly spaced screenshots to stay under 20
186
+ 3. Include a note in the prompt: "Note: {totalCount} screenshots were captured. The {selectedCount} most representative are shown."
187
+
188
+ **Video frames:** Not extracted. The video file (.webm) is referenced in the output for the user to review, but frames are not extracted for AI analysis in v1. Rationale:
189
+ - Adding ffmpeg dependency is a significant build complexity increase
190
+ - Silence-triggered screenshots already capture the relevant moments
191
+ - Video is a "nice to have" reference, not primary AI input
192
+
193
+ ---
194
+
195
+ ## 3. Token Cost Estimation
196
+
197
+ ### Typical Session (5 minutes, ~500 words spoken, 7 screenshots)
198
+
199
+ | Component | Tokens (est.) | Cost (Sonnet 4.5) |
200
+ |-----------|---------------|-------------------|
201
+ | System prompt | ~500 | $0.0015 |
202
+ | Transcript text (~500 words) | ~700 | $0.0021 |
203
+ | Screenshots (7 x ~1000 tokens each for 1080p JPEG) | ~7,000 | $0.021 |
204
+ | Output (~800 tokens structured JSON) | ~800 | $0.008 |
205
+ | **Total** | **~9,000** | **~$0.032** |
206
+
207
+ **Claude Sonnet 4.5 pricing** (as of Feb 2026):
208
+ - Input: $3/1M tokens
209
+ - Output: $15/1M tokens (with extended thinking disabled)
210
+
211
+ ### Cost by Session Length
212
+
213
+ | Session | Transcript | Screenshots | Est. Cost |
214
+ |---------|-----------|-------------|-----------|
215
+ | 2 min, 200 words, 3 screenshots | ~280 | ~3,000 | ~$0.015 |
216
+ | 5 min, 500 words, 7 screenshots | ~700 | ~7,000 | ~$0.032 |
217
+ | 10 min, 1000 words, 12 screenshots | ~1,400 | ~12,000 | ~$0.055 |
218
+ | 15 min, 1500 words, 18 screenshots | ~2,100 | ~18,000 | ~$0.078 |
219
+ | 30 min, 3000 words, 20 screenshots (capped) | ~4,200 | ~20,000 | ~$0.095 |
220
+
221
+ ### Business Model Viability
222
+
223
+ At $12/month per user:
224
+ - Break-even at ~375 sessions/month per user (~12/day, far above expected usage)
225
+ - Expected usage: 2-5 sessions/day = $0.06-$0.16/day = $1.80-$4.80/month
226
+ - **Healthy margin of 60-85%** at typical usage levels
227
+
228
+ ### Handling Large Sessions
229
+
230
+ For sessions exceeding 200K context window (extremely unlikely with voice sessions):
231
+ 1. Truncate transcript to last 3000 words (keep most recent context)
232
+ 2. Cap screenshots at 20
233
+ 3. Add a note: "This session was very long. Analysis covers the most recent portion."
234
+
235
+ This scenario is practically impossible with the 30-minute recording limit and natural speech rate.
236
+
237
+ ---
238
+
239
+ ## 4. Processing Time Estimation
240
+
241
+ | Phase | Time (est.) |
242
+ |-------|-------------|
243
+ | Image optimization (resize/compress 7 images) | 200-500ms |
244
+ | Prompt construction | <50ms |
245
+ | Claude API round-trip (Sonnet 4.5 with 7 images) | 3-8 seconds |
246
+ | JSON parsing + markdown assembly | <50ms |
247
+ | File system write | <100ms |
248
+ | **Total** | **4-9 seconds** |
249
+
250
+ This fits within the product vision's "A few seconds later, a notification tells you the session is complete." The processing state timeout is 10 seconds, which provides adequate headroom.
251
+
252
+ **Optimization opportunities for future:**
253
+ - Stream the response to show progressive output
254
+ - Pre-optimize images while the session is still recording (on each screenshot capture)
255
+ - Use prompt caching for the system prompt (saves ~$0.001 per session)
256
+
257
+ ---
258
+
259
+ ## 5. Free vs Premium Tier Differences
260
+
261
+ ### Free Tier (Current Behavior - No Changes)
262
+
263
+ The existing pipeline remains untouched:
264
+
265
+ 1. `SessionController.stop()` returns `Session`
266
+ 2. `sessionAdapter.adaptSessionForMarkdown()` converts types
267
+ 3. `FeedbackAnalyzer.analyze()` does rule-based categorization
268
+ 4. `MarkdownGenerator.generateFullDocument()` produces output
269
+ 5. `FileManager.saveSession()` writes to disk
270
+
271
+ Output: Well-structured markdown with rule-based categories, keyword-derived titles, and template-driven action items. Functional but mechanical.
272
+
273
+ ### Premium / BYOK Tier (New Pipeline)
274
+
275
+ 1. `SessionController.stop()` returns `Session` (same)
276
+ 2. **NEW**: `AIPipelineManager.analyze(session)` is called
277
+ 3. **NEW**: `ClaudeAnalyzer` sends transcript + screenshots to Claude API
278
+ 4. **NEW**: `StructuredMarkdownBuilder` assembles AI-analyzed output
279
+ 5. `FileManager.saveSession()` writes to disk (same)
280
+
281
+ Output: Intelligent markdown with Claude-analyzed groupings, context-aware titles, nuanced priorities, pattern detection, and natural-language action items. The "wow factor."
282
+
283
+ ### BYOK Flow
284
+
285
+ Users who bring their own Anthropic API key get the same analysis pipeline. The only difference is the API key source:
286
+ - **Premium**: Key is proxied through a Cloudflare Worker (Eddie's key, metered per subscription)
287
+ - **BYOK**: Key is stored locally in the OS keychain via `SettingsManager.getApiKey('anthropic')`
288
+
289
+ The `ClaudeAnalyzer` accepts an API key parameter and doesn't care where it comes from.
290
+
291
+ ### Decision Logic
292
+
293
+ ```typescript
294
+ async function processSession(session: Session): Promise<OutputResult> {
295
+ const tier = await determineTier();
296
+
297
+ if (tier === 'premium' || tier === 'byok') {
298
+ try {
299
+ const analysis = await claudeAnalyzer.analyze(session, {
300
+ apiKey: tier === 'byok'
301
+ ? await settingsManager.getApiKey('anthropic')
302
+ : await getPremiumProxyKey(),
303
+ baseUrl: tier === 'premium'
304
+ ? 'https://api.markupr.com/v1' // Cloudflare Worker proxy
305
+ : 'https://api.anthropic.com',
306
+ });
307
+ return structuredMarkdownBuilder.build(session, analysis);
308
+ } catch (error) {
309
+ // FALLBACK: If AI analysis fails, fall back to free tier output
310
+ console.warn('[AIPipeline] Claude analysis failed, falling back to basic output:', error);
311
+ return generateFreeOutput(session);
312
+ }
313
+ }
314
+
315
+ return generateFreeOutput(session);
316
+ }
317
+
318
+ function determineTier(): 'premium' | 'byok' | 'free' {
319
+ if (premiumSubscription.isActive()) return 'premium';
320
+ if (settingsManager.hasApiKey('anthropic')) return 'byok';
321
+ return 'free';
322
+ }
323
+ ```
324
+
325
+ ---
326
+
327
+ ## 6. Error Handling Strategy
328
+
329
+ ### Failure Modes and Recovery
330
+
331
+ | Failure | Impact | Recovery |
332
+ |---------|--------|----------|
333
+ | Claude API timeout (>10s) | No AI analysis | Fall back to free tier output. User gets basic markdown. |
334
+ | Claude API rate limit (429) | No AI analysis | Fall back to free tier output. Retry on next session. |
335
+ | Claude API auth error (401) | API key invalid | Fall back to free tier. Show settings notification. |
336
+ | Claude API server error (5xx) | Temporary outage | Fall back to free tier output. |
337
+ | Malformed JSON response | Can't parse AI output | Fall back to free tier output. Log for debugging. |
338
+ | Image optimization fails | Can't resize screenshot | Send original image (may be larger). If still fails, skip that screenshot. |
339
+ | Network offline | No API access | Fall back to free tier output immediately (no timeout wait). |
340
+ | Premium proxy down | Proxy unavailable | Fall back to free tier. Show "Premium service temporarily unavailable." |
341
+
342
+ ### Key Principle: Never Lose the Session
343
+
344
+ The AI analysis is an **enhancement layer**. The session data (transcript + screenshots) is always saved regardless of whether AI analysis succeeds. The free tier pipeline is the safety net.
345
+
346
+ ```typescript
347
+ async function processSessionSafely(session: Session): Promise<void> {
348
+ // ALWAYS save the basic output first
349
+ const basicDocument = generateFreeOutput(session);
350
+ const saveResult = await fileManager.saveSession(session, basicDocument);
351
+
352
+ // THEN attempt AI enhancement (if eligible)
353
+ const tier = determineTier();
354
+ if (tier !== 'free') {
355
+ try {
356
+ const aiDocument = await runAIPipeline(session);
357
+ // Overwrite the basic output with the AI-enhanced version
358
+ await fileManager.overwriteReport(saveResult.sessionDir, aiDocument);
359
+ } catch (error) {
360
+ // Basic output is already saved - user still gets value
361
+ console.warn('[AIPipeline] Enhancement failed, basic output preserved');
362
+ }
363
+ }
364
+ }
365
+ ```
366
+
367
+ ### User-Facing Error Messages
368
+
369
+ - **Timeout**: "AI analysis is taking longer than expected. Your feedback was saved with basic formatting."
370
+ - **Auth error**: "Your API key appears to be invalid. Check Settings > API Keys. Your feedback was saved with basic formatting."
371
+ - **Offline**: "No internet connection. Your feedback was saved with basic formatting. AI analysis requires a network connection."
372
+ - **Rate limit**: "AI service is temporarily busy. Your feedback was saved with basic formatting."
373
+
374
+ All messages end with the reassurance that the session was saved. Users should never feel like they lost work.
375
+
376
+ ---
377
+
378
+ ## 7. Output Markdown Format
379
+
380
+ ### AI-Enhanced Output (Premium / BYOK)
381
+
382
+ ```markdown
383
+ # Feedback Report: MyApp - Feb 5, 2026
384
+
385
+ > AI-analyzed by Claude | Duration: 5:23 | 7 screenshots | 5 items identified
386
+
387
+ ## Summary
388
+
389
+ The user identified a critical layout overlap between the action button and header on mobile viewports, a janky loading transition on the dashboard, and noted that the card animation pattern on the detail page should be reused elsewhere. Overall, the UI needs attention on mobile responsiveness and transition polish.
390
+
391
+ ---
392
+
393
+ ## Critical Issues
394
+
395
+ ### 1. Mobile button overlaps header
396
+ > "This button is way too small on mobile. And it's competing with the header -- look, they're practically overlapping."
397
+
398
+ ![Screenshot](./screenshots/fb-001.png)
399
+
400
+ - **Priority:** High
401
+ - **Category:** Bug
402
+ - **Area:** Mobile Layout
403
+ - **Action:** Fix button/header z-index and spacing in the mobile breakpoint. The button needs larger tap target (min 44px) and sufficient margin from the header.
404
+
405
+ ---
406
+
407
+ ## Improvements Needed
408
+
409
+ ### 2. Dashboard loading transition feels janky
410
+ > "The loading spinner here feels sluggish. It shows up but then the content pops in with no transition. Feels janky."
411
+
412
+ ![Screenshot](./screenshots/fb-002.png)
413
+
414
+ - **Priority:** Medium
415
+ - **Category:** Performance
416
+ - **Area:** Dashboard
417
+ - **Action:** Add a fade-in transition (200-300ms ease) when content replaces the loading spinner. Consider a skeleton loader instead of a spinner.
418
+
419
+ ---
420
+
421
+ ## Suggestions
422
+
423
+ ### 3. Reuse card animation pattern on dashboard
424
+ > "Actually, I love how this card animation works. We should use this pattern on the dashboard too."
425
+
426
+ ![Screenshot](./screenshots/fb-003.png)
427
+
428
+ - **Priority:** Low
429
+ - **Category:** Suggestion
430
+ - **Area:** Dashboard, Detail Page
431
+ - **Action:** Extract the card animation from the detail page into a shared component/utility and apply it to dashboard cards.
432
+
433
+ ---
434
+
435
+ ## Positive Notes
436
+
437
+ - The card animation on the detail page was praised as a good interaction pattern worth reusing.
438
+
439
+ ---
440
+
441
+ ## Themes
442
+
443
+ - Mobile responsiveness
444
+ - Transition polish
445
+ - Component reusability
446
+
447
+ ## Session Info
448
+
449
+ - **Session ID:** `a1b2c3d4-...`
450
+ - **Source:** MyApp (screen)
451
+ - **Duration:** 5:23
452
+ - **Screenshots:** 7
453
+ - **Recording:** [session-recording.webm](./session-recording.webm)
454
+ - **Analysis:** Claude Sonnet 4.5 (AI-enhanced)
455
+
456
+ ---
457
+ *Generated by [markupr](https://github.com/eddiesanjuan/markupr) with AI analysis*
458
+ ```
459
+
460
+ ### Key Differences from Free Tier Output
461
+
462
+ | Aspect | Free Tier | Premium/BYOK |
463
+ |--------|-----------|-------------|
464
+ | Summary | Counts-based ("5 items, 2 high priority") | Natural language insight |
465
+ | Grouping | Chronological order | Grouped by priority, then theme |
466
+ | Titles | First sentence of transcript, truncated | AI-generated descriptive title |
467
+ | Action items | Template-based ("Reproduce and patch...") | Context-specific ("Fix button/header z-index...") |
468
+ | Categories | Keyword-matching heuristic | AI-judged with context from screenshots |
469
+ | Positive notes | Not extracted | Explicitly identified section |
470
+ | Themes | Keyword frequency count | AI-identified cross-cutting themes |
471
+ | Cross-references | None | Items linked by shared area/theme |
472
+
473
+ ---
474
+
475
+ ## 8. Implementation Plan
476
+
477
+ ### New Files to Create
478
+
479
+ ```
480
+ src/main/ai/
481
+ AIPipelineManager.ts # Orchestrator - decides tier, runs pipeline
482
+ ClaudeAnalyzer.ts # Claude API integration with vision
483
+ StructuredMarkdownBuilder.ts # Converts AI JSON output to formatted markdown
484
+ ImageOptimizer.ts # Resize/compress screenshots for API
485
+ types.ts # AI pipeline types (AnalysisResult, etc.)
486
+ ```
487
+
488
+ ### Files to Modify
489
+
490
+ ```
491
+ src/main/index.ts # Wire AIPipelineManager into session completion flow
492
+ src/main/output/index.ts # Re-export AI pipeline for convenience
493
+ src/shared/types.ts # Add AI tier types, BYOK settings
494
+ src/preload/index.ts # Expose AI tier status to renderer
495
+ src/renderer/components/ # UI for AI status indicator, BYOK key entry
496
+ ```
497
+
498
+ ### Implementation Order
499
+
500
+ 1. **`src/main/ai/types.ts`** - Define `AIAnalysisResult`, `AITier`, pipeline options
501
+ 2. **`src/main/ai/ImageOptimizer.ts`** - Screenshot resize/compress using `nativeImage`
502
+ 3. **`src/main/ai/ClaudeAnalyzer.ts`** - Core Claude API integration
503
+ - Build the multi-modal message (text + images)
504
+ - Send request, parse JSON response
505
+ - Handle all error cases with typed errors
506
+ 4. **`src/main/ai/StructuredMarkdownBuilder.ts`** - Convert `AIAnalysisResult` to markdown
507
+ - Group items by priority section (Critical, Improvements, Suggestions, Positive)
508
+ - Link screenshots to items using the AI's `screenshotIndices`
509
+ - Generate the final formatted markdown string
510
+ 5. **`src/main/ai/AIPipelineManager.ts`** - Orchestrator
511
+ - Determine tier (premium/byok/free)
512
+ - Run the appropriate pipeline
513
+ - Handle fallback on failure
514
+ - Emit progress events to renderer
515
+ 6. **`src/main/index.ts`** - Integration
516
+ - Replace direct `generateDocumentForFileManager()` call with `AIPipelineManager.process()`
517
+ - Add IPC handlers for AI tier status
518
+ 7. **UI updates** - Settings panel for BYOK Anthropic key, AI status indicator during processing
519
+
520
+ ### Dependencies
521
+
522
+ - `@anthropic-ai/sdk` - Official Anthropic TypeScript SDK (already well-maintained, handles retries, streaming)
523
+ - No other new dependencies required. Image optimization uses Electron's built-in `nativeImage`.
524
+
525
+ ### Testing Strategy
526
+
527
+ - **Unit tests for `ClaudeAnalyzer`**: Mock the Anthropic SDK, verify prompt construction and JSON parsing
528
+ - **Unit tests for `StructuredMarkdownBuilder`**: Given a known `AIAnalysisResult`, verify markdown output matches expected format
529
+ - **Unit tests for `ImageOptimizer`**: Verify resize behavior, format conversion, error handling
530
+ - **Integration test for `AIPipelineManager`**: Mock `ClaudeAnalyzer`, verify tier selection and fallback logic
531
+ - **Manual test**: Record a real session, verify end-to-end AI output quality
532
+
533
+ ---
534
+
535
+ ## 9. API Key Management
536
+
537
+ ### Premium Tier (Cloudflare Worker Proxy)
538
+
539
+ ```
540
+ User App --> https://api.markupr.com/v1/messages
541
+ |
542
+ Cloudflare Worker
543
+ |
544
+ Validates subscription (Stripe customer ID)
545
+ Adds Eddie's API key to request headers
546
+ Forwards to https://api.anthropic.com/v1/messages
547
+ |
548
+ Claude API
549
+ ```
550
+
551
+ The Cloudflare Worker:
552
+ 1. Receives the request with the user's subscription token
553
+ 2. Validates against Stripe that the subscription is active
554
+ 3. Adds the real Anthropic API key (stored as a Worker secret)
555
+ 4. Proxies the request to Claude
556
+ 5. Returns the response to the user
557
+
558
+ This means Eddie's API key never touches the user's machine.
559
+
560
+ ### BYOK (Bring Your Own Key)
561
+
562
+ The user enters their Anthropic API key in Settings. It's stored in the OS keychain via `keytar` (already used for OpenAI keys via `SettingsManager`).
563
+
564
+ ```typescript
565
+ // Store
566
+ await settingsManager.setApiKey('anthropic', userKey);
567
+
568
+ // Retrieve
569
+ const key = await settingsManager.getApiKey('anthropic');
570
+
571
+ // Delete
572
+ await settingsManager.deleteApiKey('anthropic');
573
+ ```
574
+
575
+ The key is never written to disk in plaintext, never logged, and never sent anywhere except directly to `api.anthropic.com`.
576
+
577
+ ---
578
+
579
+ ## 10. Future Considerations (Out of Scope for v1)
580
+
581
+ 1. **Streaming responses** - Show AI analysis progressively in a session review view
582
+ 2. **Video frame extraction** - Use ffmpeg to extract key frames from the .webm recording
583
+ 3. **Multi-session analysis** - "What patterns do you see across my last 10 sessions?"
584
+ 4. **Custom system prompts** - Let power users modify the analysis instructions
585
+ 5. **Prompt caching** - Cache the system prompt to reduce cost by ~40% on input tokens
586
+ 6. **Batch processing** - Re-analyze old sessions with AI after upgrading to premium
587
+ 7. **Claude 4 Opus** - Offer as a "deep analysis" option for longer sessions (higher cost, better reasoning)
588
+
589
+ ---
590
+
591
+ ## Summary
592
+
593
+ The AI pipeline is a clean enhancement layer that sits between `SessionController.stop()` and `FileManager.saveSession()`. It transforms raw session data into an intelligent document using Claude's vision capabilities. The free tier is unaffected. Failures gracefully degrade to the existing output. The architecture is simple: one new module (`src/main/ai/`) with 5 files, integrated via a single function call in `src/main/index.ts`.
594
+
595
+ Cost is approximately $0.03 per typical session, making the $12/month subscription profitable at any reasonable usage level. Processing time is 4-9 seconds, fitting the product vision's "a few seconds" promise.