dashcam 1.0.1-beta.8 → 1.0.2-beta.1

package/.github/workflows/publish.yml

@@ -1,43 +1,49 @@
-name: Publish to npm
+name: Publish Beta to npm
 
 on:
   push:
-    branches:
-      - main
+    branches: [ "main" ]
+  workflow_dispatch:
 
 jobs:
-  publish:
+  publish-beta:
     runs-on: ubuntu-latest
-    permissions:
-      contents: write
-      id-token: write
+
     steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0
-          token: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Setup Node.js
         uses: actions/setup-node@v4
         with:
-          node-version: '20'
-          registry-url: 'https://registry.npmjs.org'
+          node-version: "20"
+          registry-url: "https://registry.npmjs.org/"
 
       - name: Configure Git
         run: |
-          git config user.name "github-actions[bot]"
-          git config user.email "github-actions[bot]@users.noreply.github.com"
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+
+      - name: Install dependencies
+        run: npm install
 
-      - name: Bump version
+      - name: Commit dependency changes
         run: |
-          npm version prerelease --preid=beta -m "chore: bump version to %s [skip ci]"
+          git add package-lock.json
+          git diff --staged --quiet || git commit -m "chore: update package-lock.json"
 
-      - name: Push changes
+      - name: Bump version (prerelease beta)
         run: |
-          git push --follow-tags
+          npm version prerelease --preid=beta -m "chore: bump beta version to %s"
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
-      - name: Publish to npm
-        run: npm publish --access public --tag beta
+      - name: Publish to npm under beta tag
+        run: npm publish --tag beta
         env:
           NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+
+      - name: Push version bump commit + tag
+        run: |
+          git push --follow-tags

package/NPM_PUBLISH_FIX.md

@@ -0,0 +1,104 @@
+# NPM Publish & Version Sync Issues
+
+## Problem
+
+The npm publish workflow was failing with:
+```
+npm error 404 'dashcam@1.0.1-beta.17' is not in this registry.
+```
+
+## Root Cause
+
+**Version drift** between local git tags, package.json, and npm registry:
+
+- **NPM registry**: `1.0.1-beta.13` (latest successfully published)
+- **Local package.json**: `1.0.1-beta.16` (from failed publish attempts)
+- **Git tags**: Only up to `v1.0.1-beta.9`
+
+This happened because the workflow:
+1. Bumped version in package.json
+2. Created git tag
+3. Tried to publish
+4. **Publish failed** (possibly due to test failures or video encoding issues)
+5. Git changes were not reverted, leaving version incremented
+
+## Fixes Applied
+
+### 1. Updated Publish Workflow
+
+**Before**: Version bump → Push → Publish (wrong order!)
+
+**After**: 
+1. Sync with npm registry version
+2. Bump version
+3. **Publish first** ✅
+4. Push only if publish succeeded ✅
+
+```yaml
+- name: Bump version
+  run: |
+    # Get current version from npm
+    CURRENT_NPM_VERSION=$(npm view dashcam dist-tags.beta)
+    
+    # Sync before bumping
+    npm version $CURRENT_NPM_VERSION --no-git-tag-version --allow-same-version
+    
+    # Bump to next
+    npm version prerelease --preid=beta
+
+- name: Publish to npm
+  run: npm publish --access public --tag beta
+
+- name: Push changes  # Only runs if publish succeeded
+  run: git push --follow-tags
+```
+
+### 2. Created Version Sync Script
+
+`scripts/sync-version.sh` - Helps detect and fix version drift:
+
+```bash
+./scripts/sync-version.sh
+```
+
+This will:
+- Compare local version with npm registry
+- Offer to sync package.json
+- Show unpublished git tags that need cleanup
+
+### 3. Synced package.json
+
+Reset from `1.0.1-beta.16` → `1.0.1-beta.13` (current npm version)
+
+## Video Issue Connection
+
+The failed publishes were likely caused by **test failures due to the video encoding bugs** we fixed:
+
+- Frame rate conflicts
+- Buffer size issues
+- Frame dropping
+- Incomplete container metadata
+
+All of these are now fixed in the recorder.js, so future CI runs should succeed.
+
+## Next Steps
+
+1. ✅ Package.json is synced
+2. ✅ Workflow is fixed
+3. ⚠️  Need to clean up unpublished git tags manually (optional):
+
+```bash
+# List tags
+git tag | grep beta
+
+# Delete local tags for unpublished versions (14-16)
+git tag -d v1.0.1-beta.14 v1.0.1-beta.15 v1.0.1-beta.16
+
+# If they exist on remote, delete them
+git push origin :refs/tags/v1.0.1-beta.14
+git push origin :refs/tags/v1.0.1-beta.15
+git push origin :refs/tags/v1.0.1-beta.16
+```
+
+4. Push the fixes to trigger a new publish
+5. Next version will be `1.0.1-beta.14` ✅

package/SINGLE_FRAME_VIDEO_FIX.md

@@ -0,0 +1,129 @@
+# Single-Frame Video & Frame Drop Issues - Root Cause & Fix
+
+## Problem Description
+
+Videos produced by dashcam-cli had two related issues:
+1. **Single-frame appearance**: Videos appearing as single-frame, even though they contained multiple frames
+2. **Frame drops**: Videos showing significantly shorter duration than actual recording time (e.g., 12 seconds shown vs 68 seconds actual)
+
+## Root Causes
+
+### Issue 1: Incomplete WebM Container Metadata
+When ffmpeg's VP9 encoder is terminated before it can properly finalize the stream:
+
+1. **Missing duration metadata** - The WebM container doesn't have duration information
+2. **Premature file ending** - FFprobe reports "File ended prematurely"  
+3. **Playback issues** - Some players show only the first frame because they can't seek without duration metadata
+
+### Issue 2: Frame Dropping During Capture
+Frames were being dropped during the recording process due to:
+
+1. **Conflicting frame rate settings** - Platform config had hardcoded `-r 30` while fps parameter was set to 10
+2. **Insufficient buffer sizes** - Default thread queue size caused frame drops when encoder couldn't keep up
+3. **Premature stream termination** - The `-shortest` flag caused encoding to stop before all buffered frames were processed
+4. **Missing vsync enforcement** - Frames could be skipped instead of encoded
+
+### Example from Real Recording
+
+```bash
+# File: 691cb2b4c2fc02f59ae66e21.mp4
+Frame count: 512 frames
+Actual duration: 17.06 seconds (when decoded)
+Container duration: N/A (missing metadata)
+Warning: "File ended prematurely"
+```
+
+The video has 512 frames spanning 17 seconds, but players that rely on container metadata see it as a single frame.
+
+## Platform Specificity
+
+This issue can occur on all platforms but may be more prevalent on Linux due to:
+- Different screen capture performance characteristics
+- Variations in how X11grab delivers frames vs AVFoundation (macOS) or gdigrab (Windows)
+- System load affecting encoder buffer flush timing
+
+## Fix Applied
+
+### 1. Minimum Recording Duration (recorder.js)
+```javascript
+const MIN_RECORDING_DURATION = 2000; // 2 seconds minimum
+if (recordingDuration < MIN_RECORDING_DURATION) {
+  const waitTime = MIN_RECORDING_DURATION - recordingDuration;
+  await new Promise(resolve => setTimeout(resolve, waitTime));
+}
+```
+
+### 2. Improved FFmpeg Encoding Parameters
+
+**Removed conflicting settings:**
+```javascript
+// REMOVED from platform config:
+'-r', '30'  // This conflicted with fps parameter
+
+// REMOVED from output args:
+'-shortest'  // This caused premature termination
+```
+
+**Added buffer and sync enforcement:**
+```javascript
+// Input buffering (before -i):
+'-thread_queue_size', '512',    // Large input buffer prevents drops
+'-probesize', '50M',            // Better stream detection
+
+// Output sync enforcement:
+'-vsync', '1',                   // Constant frame rate - encode every frame
+'-max_muxing_queue_size', '9999' // Large muxing queue prevents drops
+
+// Existing improvements:
+'-quality', 'good',              // Changed from 'realtime' for better finalization
+'-cpu-used', '4',                // Balanced encoding speed
+'-deadline', 'good',             // Good quality mode
+'-g', fps.toString(),            // Keyframe every second (was 2 seconds)
+'-force_key_frames', `expr:gte(t,n_forced*1)`, // Force keyframes every 1s
+'-fflags', '+genpts',            // Generate presentation timestamps
+'-avoid_negative_ts', 'make_zero' // Prevent timestamp issues
+```
+
+### 3. Extended Graceful Shutdown Timing
+```javascript
+// Graceful quit: 5s -> 8s (VP9 needs time to finalize)
+// SIGTERM timeout: 10s -> 15s
+// Post-exit wait: 3s (for filesystem sync)
+
+currentRecording.stdin.write('q');
+currentRecording.stdin.end(); // Properly close stdin
+```
+
+## Testing
+
+Use the provided test script to verify recordings:
+
+```bash
+# Analyze existing video
+node test-short-recording.js analyze <video-file>
+
+# Run automated tests with various durations
+node test-short-recording.js
+```
+
+The test checks for:
+- ✅ Frame count > 1
+- ✅ No "File ended prematurely" warnings
+- ✅ Container metadata is complete (duration present)
+- ✅ All platforms (macOS/Linux/Windows)
+
+## Prevention
+
+The fix ensures proper container finalization by:
+1. Enforcing minimum recording time for multiple frames
+2. Using encoding parameters that prioritize stream finalization
+3. Allowing sufficient time for VP9 encoder to flush buffers
+4. Properly closing stdin before waiting for process exit
+5. Adding safety timeouts before force-killing the encoder
+
+## Impact
+
+- Short recordings (< 2s) now wait to ensure at least 2 seconds of footage
+- All recordings get properly finalized WebM container metadata
+- Videos play correctly in all players, including web browsers
+- No more "single frame" issues on any platform

package/bin/dashcam.js

@@ -417,30 +417,39 @@ program
 
         console.log('Recording stopped successfully');
         
+        // Wait for upload to complete (background process handles this)
+        logger.debug('Waiting for background upload to complete...');
+        console.log('⏳ Uploading recording...');
+        
+        // Wait up to 2 minutes for upload result to appear
+        const maxWaitForUpload = 120000; // 2 minutes
+        const startWaitForUpload = Date.now();
+        let uploadResult = null;
+        
+        while (!uploadResult && (Date.now() - startWaitForUpload) < maxWaitForUpload) {
+          uploadResult = processManager.readUploadResult();
+          if (!uploadResult) {
+            await new Promise(resolve => setTimeout(resolve, 1000)); // Check every second
+          }
+        }
+        
+        logger.debug('Upload result read attempt', { found: !!uploadResult, shareLink: uploadResult?.shareLink });
+        
+        if (uploadResult && uploadResult.shareLink) {
+          console.log('📹 Watch your recording:', uploadResult.shareLink);
+          // Clean up the result file now that we've read it
+          processManager.cleanup();
+          process.exit(0);
+        }
+        
         // Check if files still exist - if not, background process already uploaded
         const filesExist = fs.existsSync(result.outputPath) && 
                           (!result.gifPath || fs.existsSync(result.gifPath)) && 
                           (!result.snapshotPath || fs.existsSync(result.snapshotPath));
         
         if (!filesExist) {
-          // Files were deleted, meaning background process uploaded
-          // Wait for the upload result to be written
-          logger.debug('Waiting for upload result from background process');
-          await new Promise(resolve => setTimeout(resolve, 2000));
-          
-          // Try to read the upload result from the background process
-          const uploadResult = processManager.readUploadResult();
-          logger.debug('Upload result read attempt', { found: !!uploadResult, shareLink: uploadResult?.shareLink });
-          
-          if (uploadResult && uploadResult.shareLink) {
-            console.log('📹 Watch your recording:', uploadResult.shareLink);
-            // Clean up the result file now that we've read it
-            processManager.cleanup();
-          } else {
-            console.log('✅ Recording uploaded (share link not available)');
-            logger.warn('Upload result not available from background process');
-          }
-          
+          console.log('✅ Recording uploaded by background process');
+          logger.info('Files were cleaned up by background process');
           process.exit(0);
         }
         
@@ -683,13 +692,4 @@ program
     }
   });
 
-// If no command specified and there are options like --md, treat as create command
-program.action(async (options) => {
-  // Default to create command when running just "dashcam"
-  const createCommand = program.commands.find(cmd => cmd.name() === 'create');
-  if (createCommand && createCommand._actionHandler) {
-    await createCommand._actionHandler(options);
-  }
-});
-
 program.parse();

package/lib/auth.js

@@ -18,7 +18,7 @@ const auth = {
       });
       
       // Exchange API key for token
-      const { token } = await got.post('https://api.testdriver.ai/auth/exchange-api-key', {
+      const { token } = await got.post('https://testdriver-api.onrender.com/auth/exchange-api-key', {
         json: { apiKey },
         timeout: 30000 // 30 second timeout
       }).json();
@@ -34,7 +34,7 @@ const auth = {
 
       // Get user info to verify the token works
       logger.debug('Fetching user information to validate token...');
-      const user = await got.get('https://api.testdriver.ai/api/v1/whoami', {
+      const user = await got.get('https://testdriver-api.onrender.com/api/v1/whoami', {
         headers: {
           Authorization: `Bearer ${token}`
         },
@@ -105,7 +105,7 @@ const auth = {
     const token = await this.getToken();
     
     try {
-      const response = await got.get('https://api.testdriver.ai/api/v1/projects', {
+      const response = await got.get('https://testdriver-api.onrender.com/api/v1/projects', {
         headers: {
           Authorization: `Bearer ${token}`
         },
@@ -160,7 +160,7 @@ const auth = {
       requestBody.project = replayData.project;
     }
     
-    const response = await got.post('https://api.testdriver.ai/api/v1/replay/upload', {
+    const response = await got.post('https://testdriver-api.onrender.com/api/v1/replay/upload', {
       headers: {
         Authorization: `Bearer ${token}`
       },
@@ -188,7 +188,7 @@ const auth = {
     const token = await this.getToken();
     
     try {
-      const response = await got.post('https://api.testdriver.ai/api/v1/logs', {
+      const response = await got.post('https://testdriver-api.onrender.com/api/v1/logs', {
         headers: {
           Authorization: `Bearer ${token}`
         },

package/lib/config.js

@@ -17,7 +17,7 @@ export const auth0Config = {
 export const apiEndpoints = {
   development: process.env.API_ENDPOINT || 'http://localhost:3000',
   staging: 'https://replayable-api-staging.herokuapp.com',
-  production: 'https://api.testdriver.ai'
+  production: 'https://testdriver-api.onrender.com'
 };
 
 export const API_ENDPOINT = apiEndpoints[ENV];

package/lib/ffmpeg.js

@@ -18,7 +18,7 @@ export async function createSnapshot(inputVideoPath, outputSnapshotPath, snapsho
     '-i', inputVideoPath,
     '-frames:v', '1',
     '-vf', 'scale=640:-1:force_original_aspect_ratio=decrease:eval=frame',
-    '-compression_level', '6', // Use default compression (was 100, which is extremely slow)
+    '-compression_level', '0', // Fast compression for speed (0 = fastest, 9 = slowest)
     outputSnapshotPath,
     '-y',
     '-hide_banner'
@@ -94,62 +94,32 @@ export async function createGif(inputVideoPath, outputGifPath) {
       stdout
     });
     
-    // Fallback: Just sample the video at a fixed rate (e.g., 1 frame every 3 seconds)
-    const framesPath = path.join(os.tmpdir(), `frames_${id}_%04d.png`);
+    // Fallback: Single-pass GIF creation at fixed rate
+    const filters = `fps=1/3,scale=640:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=128[p];[s1][p]paletteuse=dither=bayer:bayer_scale=5`;
+    
     await execa(ffmpegPath, [
       '-i', inputVideoPath,
-      '-vf', `fps=1/3`, // Sample 1 frame every 3 seconds
-      framesPath
-    ]);
-
-    // Create GIF from frames
-    await execa(ffmpegPath, [
-      '-framerate', `${gifFps}`,
-      '-i', framesPath,
+      '-vf', filters,
       '-loop', '0',
       outputGifPath,
       '-y',
       '-hide_banner'
     ]);
-
-    // Clean up temporary frame files
-    const framesToDelete = fs.readdirSync(os.tmpdir())
-      .filter(file => file.startsWith(`frames_${id}_`) && file.endsWith('.png'))
-      .map(file => path.join(os.tmpdir(), file));
-    
-    for (const frame of framesToDelete) {
-      fs.unlinkSync(frame);
-    }
     
     return;
   }
   
   const extractedFramesInterval = videoDuration / gifFrames;
 
-  // Extract frames
-  const framesPath = path.join(os.tmpdir(), `frames_${id}_%04d.png`);
+  // Single-pass GIF creation with palette for better quality and performance
+  const filters = `fps=1/${extractedFramesInterval},scale=640:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=128[p];[s1][p]paletteuse=dither=bayer:bayer_scale=5`;
+  
   await execa(ffmpegPath, [
     '-i', inputVideoPath,
-    '-vf', `fps=1/${extractedFramesInterval}`,
-    framesPath
-  ]);
-
-  // Create GIF from frames
-  await execa(ffmpegPath, [
-    '-framerate', `${gifFps}`,
-    '-i', framesPath,
+    '-vf', filters,
     '-loop', '0',
     outputGifPath,
     '-y',
     '-hide_banner'
   ]);
-
-  // Clean up temporary frame files
-  const framesToDelete = fs.readdirSync(os.tmpdir())
-    .filter(file => file.startsWith(`frames_${id}_`) && file.endsWith('.png'))
-    .map(file => path.join(os.tmpdir(), file));
-  
-  for (const frame of framesToDelete) {
-    fs.unlinkSync(frame);
-  }
 }

package/lib/processManager.js

@@ -160,8 +160,9 @@ class ProcessManager {
       logger.info('Stopping active recording process', { pid });
       process.kill(pid, 'SIGINT');
       
-      // Wait for the process to actually finish
-      const maxWaitTime = 30000; // 30 seconds max
+      // Wait for the process to actually finish and upload
+      // Increase timeout to allow for upload to complete
+      const maxWaitTime = 120000; // 2 minutes max to allow for upload
       const startWait = Date.now();
       
       while (this.isProcessRunning(pid) && (Date.now() - startWait) < maxWaitTime) {
@@ -174,41 +175,27 @@ class ProcessManager {
         await new Promise(resolve => setTimeout(resolve, 1000));
       }
       
-      // Call the actual recorder's stopRecording function to get complete results
+      // The background process handles stopRecording() internally via SIGINT
+      // We just need to return the basic result from the status file
       if (status) {
-        try {
-          const { stopRecording } = await import('./recorder.js');
-          const result = await stopRecording();
-          
-          logger.info('Recording stopped successfully via recorder', { 
-            outputPath: result.outputPath,
-            duration: result.duration,
-            hasLogs: result.logs?.length > 0,
-            hasApps: result.apps?.length > 0
-          });
-          
-          // Cleanup process files but preserve upload result for stop command
-          this.cleanup({ preserveResult: true });
-          
-          return result;
-        } catch (recorderError) {
-          logger.warn('Failed to stop via recorder, falling back to basic result', { error: recorderError.message });
-          
-          // Fallback to basic result if recorder fails
-          const basePath = status.outputPath.substring(0, status.outputPath.lastIndexOf('.'));
-          const result = {
-            outputPath: status.outputPath,
-            gifPath: `${basePath}.gif`,
-            snapshotPath: `${basePath}.png`,
-            duration: Date.now() - status.startTime,
-            clientStartDate: status.startTime,
-            apps: [],
-            logs: []
-          };
-          
-          this.cleanup({ preserveResult: true });
-          return result;
-        }
+        logger.info('Background recording stopped, returning status', { 
+          outputPath: status.outputPath,
+          duration: Date.now() - status.startTime
+        });
+        
+        const basePath = status.outputPath.substring(0, status.outputPath.lastIndexOf('.'));
+        const result = {
+          outputPath: status.outputPath,
+          gifPath: `${basePath}.gif`,
+          snapshotPath: `${basePath}.png`,
+          duration: Date.now() - status.startTime,
+          clientStartDate: status.startTime,
+          apps: [],
+          logs: []
+        };
+        
+        this.cleanup({ preserveResult: true });
+        return result;
       } else {
         throw new Error('No status information available for active recording');
       }