@x12i/youtube-video-uploader-cli 1.2.0 → 1.4.0

package/README.md

@@ -1,18 +1,22 @@
 # @x12i/youtube-video-uploader-cli 📺
 
-> Automated batch video uploader for YouTube Data API v3 with OAuth2 authentication, playlist assignment, quota estimation, and smart upload-state resume.
+> Automated batch video uploader for YouTube Data API v3 with OAuth2 authentication, album/playlist metadata, safe-quota batching, playlist video swap, and smart upload-state resume.
 
 ---
 
 ## 🚀 Features
 
-- **🔐 Interactive OAuth2 Authentication**: Authenticates with YouTube using Google OAuth2. Features a local loopback server to automatically capture the callback redirect, with a terminal prompt fallback.
-- **💾 Token Persistence**: Caches tokens locally in `token.json` so you only need to authorize once. Automatically handles token refresh.
-- **📄 Metadata-Driven Batch Uploads**: Configure titles, descriptions, tags, category, privacy, and playlist via a simple `metadata.json` file.
-- **⚡ Smart Skip & Resume**: Saves upload status to `.upload-history.json` and skips previously uploaded files on rerun to protect your API quota.
-- **📑 Playlist Integration**: Automatically adds uploaded videos to a specified YouTube Playlist (`playlistId`).
-- **📊 Quota Estimation**: Calculates estimated YouTube Data API units (1,600 units/video, 50 units/playlist addition) and warns if approaching the standard 10,000 units/day limit.
-- **🔍 Dry Run Mode**: Validate your `metadata.json`, inspect files, and check estimated quota before uploading.
+- **🔐 Interactive OAuth2 Authentication**: Local loopback server to capture the callback redirect, with a terminal prompt fallback. Scopes: `youtube.upload` + `youtube.force-ssl`.
+- **💾 Token Persistence**: Caches tokens in `token.json` and refreshes automatically.
+- **📄 Metadata-Driven Batch Uploads**: Flat `videos[]` schema **or** rich album/`songs` + `playlist` + `uploadPolicy` + `{{template}}` variables.
+- **📑 Playlist Auto-Create**: Creates a playlist from `playlist.snippet` when no `playlistId` is set, then inserts videos at their metadata positions.
+- **⚡ Smart Skip & Resume**: Saves status to `.upload-history.json` and skips already-uploaded files on rerun.
+- **🛡️ Safe Quota Batching**: `--safe-quota`, `--limit`, and `--max-quota` defer excess uploads for the next day (~1,600 units/video + 50/playlist item).
+- **📊 Quota Estimation**: Warns when estimated units approach the standard 10,000/day limit.
+- **🔁 Playlist Video Swap**: `swap` subcommand replaces a playlist entry in place; old video is unlisted, never deleted. Resumable via `.swap-history.json`.
+- **🏷️ Metadata & Title Updates**: `update-metadata` / `update-titles` subcommand to batch update video titles and playlist titles on YouTube and in `metadata.json` (e.g. adding postfixes, prefixes, or syncing).
+- **🎬 Automated Stock Media Sourcing**: Search Pexels and Pixabay for high-res stock videos/photos, rank and auto-select best matches, cache responses, and compose final `.mp4` files from audio tracks via ffmpeg.
+- **🔍 Dry Run Mode**: Validate metadata and estimate quota before uploading or updating.
 
 ---
 
@@ -39,6 +43,9 @@ Before using the uploader, you need OAuth2 credentials from Google Cloud:
 ```bash
 # In the folder containing your .mp4 files and metadata.json:
 npx @x12i/youtube-video-uploader-cli
+
+# Cap under daily quota (~5–6 videos), defer the rest:
+npx @x12i/youtube-video-uploader-cli --safe-quota
 ```
 
 ### Install globally
@@ -52,12 +59,12 @@ yt-uploader [folder]
 yvu-cli [folder]
 ```
 
+Credentials can also come from environment variables (`YOUTUBE_CLIENT_ID` / `YOUTUBE_CLIENT_SECRET`, or `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET`) or a nearby `.env` file.
+
 ---
 
 ## 📂 Folder Structure & `metadata.json`
 
-Place your videos and a `metadata.json` file in your target folder:
-
 ```text
 my-videos/
 ├── metadata.json
@@ -66,7 +73,7 @@ my-videos/
 └── track3.mp4
 ```
 
-### `metadata.json` Schema
+### Flat `videos[]` schema
 
 ```json
 {
@@ -90,6 +97,72 @@ my-videos/
 }
 ```
 
+### Rich album / `songs` schema
+
+Supports album metadata, upload policy, playlist auto-create, ordered keys, and `{{playlistUrl}}` / `{{channelUrl}}` templates:
+
+```json
+{
+  "channel": {
+    "name": "My Artist",
+    "channelUrl": "https://www.youtube.com/@myartist"
+  },
+  "album": {
+    "title": "GRAY TO LIGHT",
+    "artist": "My Artist"
+  },
+  "templateVariables": {
+    "channelUrl": "https://www.youtube.com/@myartist",
+    "playlistUrl": null
+  },
+  "uploadPolicy": {
+    "initialVideoPrivacyStatus": "private",
+    "categoryId": "10",
+    "notifySubscribersOnInitialUpload": false
+  },
+  "playlist": {
+    "key": "gray_to_light",
+    "orderedSongKeys": ["01-first.mp4", "02-second.mp4"],
+    "youtube": {
+      "snippet": {
+        "title": "GRAY TO LIGHT — Full Album",
+        "description": "Full album playlist.\n\nChannel: {{channelUrl}}"
+      },
+      "status": { "privacyStatus": "private" }
+    }
+  },
+  "songs": {
+    "01-first.mp4": {
+      "trackNumber": 1,
+      "playlistPosition": 0,
+      "youtube": {
+        "snippet": {
+          "title": "First Song — My Artist | GRAY TO LIGHT",
+          "description": "Track 1.\n\nPlaylist: {{playlistUrl}}",
+          "tags": ["album", "music"]
+        },
+        "status": { "privacyStatus": "private" }
+      }
+    },
+    "02-second.mp4": {
+      "trackNumber": 2,
+      "playlistPosition": 1,
+      "youtube": {
+        "snippet": {
+          "title": "Second Song — My Artist | GRAY TO LIGHT",
+          "description": "Track 2.",
+          "tags": ["album", "music"]
+        }
+      }
+    }
+  }
+}
+```
+
+If `playlistId` is missing but `playlist.youtube.snippet` is present, the uploader creates the playlist on first run and stores its ID in `.upload-history.json`. Template variables are re-interpolated after creation so `{{playlistUrl}}` resolves correctly.
+
+If no `videos` / `songs` entries exist, all `.mp4` files in the folder are auto-discovered.
+
 #### Category IDs:
 - `"10"` = Music (default)
 - `"22"` = People & Blogs
@@ -100,6 +173,8 @@ my-videos/
 
 ## ⚙️ CLI Options Reference
 
+### Batch upload (default command)
+
 ```text
 Usage: youtube-video-uploader [directory] [options]
 
@@ -111,7 +186,10 @@ Options:
   -d, --dir <path>              Explicit target directory path
   -m, --metadata <file>         Custom path to metadata.json file
   -f, --force                   Force re-upload of already uploaded videos (default: false)
-  -n, --dry-run                 Preview video uploads and estimate quota without uploading (default: false)
+  -n, --dry-run                 Preview video uploads and estimate quota without uploading
+  -l, --limit <number>          Maximum number of videos to upload in this run
+  --safe-quota                  Cap batch under ~9,950 daily units (~5–6 videos); defer the rest
+  --max-quota <units>           Hard API quota unit cap for this run
   --client-id <id>              Google OAuth Client ID
   --client-secret <secret>      Google OAuth Client Secret
   --client-secrets-file <file>  Path to client_secret.json downloaded from Google Cloud
@@ -120,6 +198,58 @@ Options:
   -h, --help                    Display help
 ```
 
+### `swap` subcommand
+
+```text
+Usage: youtube-video-uploader swap <videoFile> [options]
+
+Arguments:
+  videoFile                     Path to the new MP4 video file
+
+Options:
+  --playlist <id>               Playlist ID containing the old video (required)
+  --old-video <videoId>         Video ID of the old video to replace (required)
+  -k, --key <songKey>           Stable song key for swap state (default: video file basename)
+  --title <title>               Override the new video title (default: carried over from old)
+  --state-file <file>           Swap state filename (default: .swap-history.json)
+  --token-file <file>           OAuth token path (default: token.json in cwd)
+  --client-secrets-file <file>  Path to client_secret.json
+  --client-id <id>              Google OAuth Client ID
+  --client-secret <secret>      Google OAuth Client Secret
+  --fresh                       Discard prior swap state for this key and start over
+  -n, --dry-run                 Print the swap plan without any API calls
+  -q, --quiet                   Minimal output mode
+  -h, --help                    Display help
+```
+
+### `update-metadata` / `update-titles` subcommand
+
+```text
+Usage: youtube-video-uploader update-metadata [directory] [options]
+
+Arguments:
+  directory                     Target folder containing metadata.json and upload history (default: ".")
+
+Options:
+  -d, --dir <path>              Explicit target directory path
+  -m, --metadata <file>         Custom path to metadata.json file
+  --title-postfix <postfix>     Postfix to append to each video title (e.g. " (long version)")
+  --playlist-postfix <postfix>  Postfix to append to playlist title (e.g. " (long versions)")
+  --title-prefix <prefix>       Prefix to prepend to each video title
+  --playlist-title <title>      Explicit new title for the playlist
+  --video <videoId>             Target a specific YouTube video ID
+  --title <title>               Explicit new title for the targeted video
+  --playlist <playlistId>       Target a specific YouTube playlist ID
+  --no-update-file              Do not update metadata.json on disk
+  -n, --dry-run                 Preview changes without making any API calls
+  --token-file <file>           OAuth token path (default: token.json in target dir)
+  --client-secrets-file <file>  Path to client_secret.json
+  --client-id <id>              Google OAuth Client ID
+  --client-secret <secret>      Google OAuth Client Secret
+  -q, --quiet                   Minimal output mode
+  -h, --help                    Display help
+```
+
 ---
 
 ## 💡 Examples
@@ -129,7 +259,18 @@ Options:
 npx @x12i/youtube-video-uploader-cli ./my-videos --dry-run
 ```
 
-### 2. Upload with credentials from environment variables
+### 2. Safe-quota batch (upload today’s slice, defer the rest)
+```bash
+npx @x12i/youtube-video-uploader-cli ./my-videos --safe-quota
+# Re-run the same command tomorrow for the next batch
+```
+
+### 3. Hard limit of N videos
+```bash
+npx @x12i/youtube-video-uploader-cli ./my-videos --limit 5
+```
+
+### 4. Upload with credentials from environment variables
 ```bash
 export YOUTUBE_CLIENT_ID="your-client-id.apps.googleusercontent.com"
 export YOUTUBE_CLIENT_SECRET="your-client-secret"
@@ -137,7 +278,7 @@ export YOUTUBE_CLIENT_SECRET="your-client-secret"
 npx @x12i/youtube-video-uploader-cli ./my-videos
 ```
 
-### 3. Upload with downloaded `client_secret.json`
+### 5. Upload with downloaded `client_secret.json`
 ```bash
 npx @x12i/youtube-video-uploader-cli ./my-videos --client-secrets-file ./client_secret.json
 ```
@@ -151,6 +292,8 @@ Re-uploads a song (e.g. a new version with lyrics subtitles) and swaps it into a
 ```bash
 yt-uploader swap ./song-v2.mp4 --playlist PLxxxx --old-video VIDEOID
 yt-uploader swap ./song-v2.mp4 --playlist PLxxxx --old-video VIDEOID --dry-run
+yt-uploader swap ./song-v2.mp4 --playlist PLxxxx --old-video VIDEOID --key "01-first.mp4" --title "First Song (Lyrics)"
+yt-uploader swap ./song-v2.mp4 --playlist PLxxxx --old-video VIDEOID --fresh
 ```
 
 The swap runs as a resumable five-step state machine (state persisted to `.swap-history.json` after every step):
@@ -161,31 +304,212 @@ The swap runs as a resumable five-step state machine (state persisted to `.swap-
 4. **REMOVE** — remove the old playlist item
 5. **FINALIZE** — old video → unlisted, new video → public
 
-If any step fails, re-run the same command — completed steps are skipped automatically (no duplicate uploads or playlist entries). A missing old video aborts with a "needs manual review" flag instead of guessing.
+If any step fails, re-run the same command — completed steps are skipped automatically (no duplicate uploads or playlist entries). A missing old video aborts with a "needs manual review" flag instead of guessing. Use `--fresh` to discard prior state for that key.
 
 **Quota cost:** ~150 shared-pool units + 1 upload-bucket unit per swap (well within the 10,000/day default pool).
 
 **Notes:**
-- OAuth scopes are now `youtube.upload` + `youtube.force-ssl` (narrower than before) — existing `token.json` files need a one-time re-consent.
+- OAuth scopes are `youtube.upload` + `youtube.force-ssl` — existing `token.json` files need a one-time re-consent if issued under older scopes.
 - The target playlist must use **Manual** ordering for position-preserving inserts (set in YouTube Studio).
 - Videos from an unverified Google Cloud project are forced private until the project passes YouTube's API audit.
 
-## 🛠️ Programmatic Node.js API
+---
 
-```javascript
-import { batchUpload, uploadVideo, loadMetadata } from '@x12i/youtube-video-uploader-cli';
-
-const result = await batchUpload('./my-videos', {
-  clientId: process.env.YOUTUBE_CLIENT_ID,
-  clientSecret: process.env.YOUTUBE_CLIENT_SECRET,
-  onEvent: (event) => {
-    if (event.type === 'upload_success') {
-      console.log(`Uploaded ${event.item.filename} -> ${event.record.url}`);
+## 🎬 Automated Stock Media Sourcing (Pexels + Pixabay)
+
+If you have audio tracks (`.wav`, `.mp3`) and no visuals, the uploader can automatically query Pexels and Pixabay for relevant stock video/photo backgrounds, select the best candidate, download it, and compose `.mp4` videos via FFmpeg before uploading.
+
+### Environment Variables & Credentials
+
+| Variable | Description | Aliases |
+|---|---|---|
+| `PEXELS_KEY` | Pexels API Key | `PEXEL_KEY` |
+| `PIXABAY_KEY` | Pixabay API Key | `PICABAY_KEY` |
+| `MEDIA_PROVIDER_ORDER` | Fallback search order (default: `pexels,pixabay`) | |
+
+API keys can also be specified via CLI flags (`--pexels-key <key>`, `--pixabay-key <key>`) or in a `.env` file.
+
+### CLI Commands
+
+#### 1. Preview search results (`search-media`)
+Preview ranked candidates without downloading anything:
+```bash
+yt-uploader search-media "calm night drive neon city" --type video --count 5
+yt-uploader search-media "sunset over ocean" --provider pixabay --orientation landscape --json
+```
+
+#### 2. Resolve media & write to `metadata.json` (`resolve-media`)
+Search, download winning media to `.media-cache/assets/`, and optionally compose the final `.mp4`:
+```bash
+# Resolve and download assets, update metadata.json
+yt-uploader resolve-media ./my-songs
+
+# Resolve, download, and build the final .mp4 with ffmpeg immediately
+yt-uploader resolve-media ./my-songs --compose
+
+# Dry-run preview
+yt-uploader resolve-media ./my-songs --dry-run
+```
+
+#### 3. Upload with automatic media resolution (`--auto-media`)
+Run media resolution and ffmpeg composition automatically for missing video files before running the batch upload:
+```bash
+yt-uploader ./my-songs --auto-media
+```
+
+#### 4. Standalone Video Composer (`compose-video`)
+Mux any background video or photo with an audio file:
+```bash
+yt-uploader compose-video --background ./bg.mp4 --audio ./song.wav --output ./final.mp4
+yt-uploader compose-video --background ./photo.jpg --audio ./song.wav --output ./final.mp4 --mode ken-burns
+```
+
+### `metadata.json` Configuration
+
+Add optional top-level `mediaPolicy` or per-song `media` overrides:
+
+```json
+{
+  "mediaPolicy": {
+    "provider": "auto",
+    "providerOrder": ["pexels", "pixabay"],
+    "mergeStrategy": "fallback",
+    "type": "video",
+    "orientation": "landscape",
+    "minWidth": 1280,
+    "minHeight": 720,
+    "safeSearch": true,
+    "avoidReuse": true,
+    "composeMode": "video-loop",
+    "cacheDir": ".media-cache"
+  },
+  "songs": {
+    "01_track.wav": {
+      "youtube": {
+        "snippet": {
+          "title": "Night Lights"
+        }
+      },
+      "media": {
+        "query": "neon city driving rain",
+        "type": "video",
+        "orientation": "landscape"
+      }
     }
+  }
+}
+```
+
+After resolution, `resolvedMedia` is saved directly to `metadata.json` and `ATTRIBUTIONS.md` is generated automatically with full licensing credits.
+
+---
+
+## 🎬 Batch Video Actions Pipeline
+
+The actions pipeline allows non-destructive video modifications (shortening, background replacement, and playlist journey overlays) processed into a **new sibling folder** with cloned `metadata.json`:
+
+### Available Actions
+
+1. **`shorten`**: Cut video at `cutPoint`, smoothly fade out audio and video into the cut, then append a silent padding buffer.
+2. **`replace-background`**: Keep existing audio and swap the visual background with a freshly sourced stock video or photo.
+3. **`journey-overlay`**: Burn in a modern journey overlay graphic (either a sleek top-right step badge or playlist panel) and optional cycling center concepts.
+
+### `metadata.json` Actions Configuration
+
+```json
+{
+  "defaultActions": [
+    { "type": "journey-overlay" }
+  ],
+  "journey": {
+    "enabled": true,
+    "style": "badge",
+    "position": "right",
+    "sidebarLabel": "stage",
+    "sidebarDisplayDuration": 4,
+    "theme": "dark",
+    "maxCenterChars": 60
   },
+  "songs": {
+    "01-first.mp4": {
+      "journeyStage": "Decision",
+      "actions": [
+        { "type": "shorten", "cutPoint": "02:15", "fadeDuration": 3, "paddingDuration": 2 },
+        { 
+          "type": "journey-overlay", 
+          "conceptsDisplayMode": "full",
+          "conceptDisplaySeconds": 7,
+          "conceptFadeSeconds": 1,
+          "centerConcepts": [
+            "You don't feel ready.",
+            "Move anyway."
+          ]
+        }
+      ]
+    },
+    "02-second.mp4": {
+      "journeyStage": "Sensation",
+      "actions": [
+        { "type": "replace-background", "query": "empty highway sunrise" }
+      ]
+    }
+  }
+}
+```
+
+### CLI Commands
+
+#### Apply Actions (`apply-actions`)
+```bash
+# Process all videos and output to sibling folder (e.g. ./my-videos (short))
+yt-uploader apply-actions ./my-videos
+
+# Preview planned ffmpeg operations without modifying disk
+yt-uploader apply-actions ./my-videos --dry-run
+
+# Run only a specific video entry
+yt-uploader apply-actions ./my-videos --only 01-first.mp4
+
+# Overwrite existing outputs
+yt-uploader apply-actions ./my-videos --force
+```
+
+#### Preview Journey Overlay (`preview-journey`)
+Generate a PNG preview of the journey graphic to iterate on sidebar and center text appearance without re-compositing videos:
+```bash
+yt-uploader preview-journey 01-first.mp4 --dir ./my-videos
+yt-uploader preview-journey 01-first.mp4 --center-text "Custom prompt question?" -o ./preview.png
+```
+
+---
+
+## 🛠️ Programmatic Node.js API
+
+```javascript
+import {
+  searchMedia,
+  resolveMediaForMetadata,
+  composeVideo,
+  batchUpload,
+} from '@x12i/youtube-video-uploader-cli';
+
+// 1. Search stock media
+const results = await searchMedia({
+  query: 'calm night drive neon city',
+  type: 'video',
+  provider: 'auto',
 });
 
-console.log(`Uploaded: ${result.uploaded.length}, Skipped: ${result.skipped.length}`);
+// 2. Resolve media for metadata.json and compose videos
+const { resolved, skipped, errors } = await resolveMediaForMetadata('./my-videos', {
+  compose: true,
+  provider: 'auto',
+});
+
+// 3. Batch upload
+const uploadResult = await batchUpload('./my-videos', {
+  safeQuota: true,
+});
 ```
 
 ---