appydave-tools 0.14.1 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +14 -0
- data/CLAUDE.md +537 -28
- data/README.md +262 -86
- data/bin/subtitle_manager.rb +18 -12
- data/bin/subtitle_processor.rb +158 -0
- data/docs/archive/codebase-audit-2025-01.md +424 -0
- data/docs/archive/documentation-framework-proposal.md +808 -0
- data/docs/archive/purpose-and-philosophy.md +110 -0
- data/docs/archive/test-coverage-quick-wins.md +342 -0
- data/docs/archive/tool-discovery.md +199 -0
- data/docs/archive/tool-documentation-analysis.md +592 -0
- data/docs/tools/bank-reconciliation.md +269 -0
- data/docs/tools/cli-actions.md +444 -0
- data/docs/tools/configuration.md +329 -0
- data/docs/{usage → tools}/gpt-context.md +118 -7
- data/docs/tools/index.md +324 -0
- data/docs/tools/move-images.md +295 -0
- data/docs/tools/name-manager.md +322 -0
- data/docs/tools/prompt-tools.md +209 -0
- data/docs/tools/subtitle-processor.md +242 -0
- data/docs/tools/youtube-automation.md +258 -0
- data/docs/tools/youtube-manager.md +248 -0
- data/exe/ad_config +6 -0
- data/exe/gpt_context +6 -0
- data/exe/prompt_tools +6 -0
- data/exe/subtitle_manager +6 -0
- data/exe/youtube_automation +6 -0
- data/exe/youtube_manager +6 -0
- data/lib/appydave/tools/{subtitle_manager → subtitle_processor}/clean.rb +1 -1
- data/lib/appydave/tools/{subtitle_manager → subtitle_processor}/join.rb +5 -2
- data/lib/appydave/tools/version.rb +1 -1
- data/lib/appydave/tools.rb +2 -4
- data/package.json +1 -1
- metadata +43 -14
- data/lib/mj-paste-test/main.rb +0 -35
- data/lib/mj-paste-test/prompts.txt +0 -18
- data/lib/mj-paste-test/readme-leonardo.md +0 -0
- /data/lib/appydave/tools/{subtitle_manager → subtitle_processor}/_doc-clean.md +0 -0
- /data/lib/appydave/tools/{subtitle_manager → subtitle_processor}/_doc-join.md +0 -0
- /data/lib/appydave/tools/{subtitle_manager → subtitle_processor}/_doc-todo.md +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: dfc47ae2c891b47ff973f7f5605c18f0d05f98f44045c7b40590d03d63fdf276
|
|
4
|
+
data.tar.gz: cd97470ff0937e41972b1777eabaaaf2da3321e2f0a8d1cd6845125b8a15d705
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c2ce45fd490af1172db2e274e733fdb329d9ef143078ada86f90aadc1cc4860bdde91c4ca19cf781dffd128de909118271ce9087657edfd6a7ad98c38ba68b94
|
|
7
|
+
data.tar.gz: 931ca82db8f193bb2b6f581d1691893ac8912949d37384d1a7f2baea35cf7fc90343c2234a259f73f69489a9405bd07c984699c0dc7716416103c65b9e8406bf
|
data/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,17 @@
|
|
|
1
|
+
# [0.15.0](https://github.com/appydave/appydave-tools/compare/v0.14.1...v0.15.0) (2025-11-08)
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
### Features
|
|
5
|
+
|
|
6
|
+
* update claude and git leaks ([0b8212f](https://github.com/appydave/appydave-tools/commit/0b8212fa65a920c608876c4f4f37c166dc552039))
|
|
7
|
+
|
|
8
|
+
## [0.14.1](https://github.com/appydave/appydave-tools/compare/v0.14.0...v0.14.1) (2025-08-06)
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
### Bug Fixes
|
|
12
|
+
|
|
13
|
+
* updagte claude and readmegs ([308af2f](https://github.com/appydave/appydave-tools/commit/308af2f3085f6bfd054c9d12154da7055d47bb3b))
|
|
14
|
+
|
|
1
15
|
# [0.14.0](https://github.com/appydave/appydave-tools/compare/v0.13.0...v0.14.0) (2025-04-27)
|
|
2
16
|
|
|
3
17
|
|
data/CLAUDE.md
CHANGED
|
@@ -2,9 +2,48 @@
|
|
|
2
2
|
|
|
3
3
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
4
|
|
|
5
|
+
## Project Purpose
|
|
6
|
+
|
|
7
|
+
**AppyDave Tools** is a consolidated productivity toolkit built for AppyDave's YouTube content creation workflow. All utilities live in one repository for easier maintenance than managing separate codebases.
|
|
8
|
+
|
|
9
|
+
**📖 See [docs/purpose-and-philosophy.md](./docs/purpose-and-philosophy.md) for complete philosophy and design principles.**
|
|
10
|
+
|
|
11
|
+
**Key Points:**
|
|
12
|
+
- **Consolidated toolkit** - One codebase for easier maintenance
|
|
13
|
+
- **YouTube workflow** - Built specifically for content creator productivity
|
|
14
|
+
- **Single-purpose tools** - Each tool solves one problem independently
|
|
15
|
+
- **Shareable individually** - Tools can be featured in standalone videos
|
|
16
|
+
- **Language flexible** - Currently Ruby, could be rewritten if needed
|
|
17
|
+
|
|
5
18
|
## Common Commands
|
|
6
19
|
|
|
7
20
|
### Development Setup
|
|
21
|
+
|
|
22
|
+
**⚠️ IMPORTANT: Bundler Setup for Claude Code**
|
|
23
|
+
|
|
24
|
+
This project requires Bundler 2.6.2. If you encounter `Could not find 'bundler' (2.6.2)` errors during Claude Code execution:
|
|
25
|
+
|
|
26
|
+
**Automatic fix (recommended):**
|
|
27
|
+
```bash
|
|
28
|
+
eval "$(rbenv init -)" && gem install bundler:2.6.2
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
**Make permanent** - Add to your `.zshrc` or `.bashrc`:
|
|
32
|
+
```bash
|
|
33
|
+
# Add rbenv to PATH and initialize
|
|
34
|
+
eval "$(rbenv init - zsh)"
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
**For Claude Code:** The `eval "$(rbenv init -)"` command is automatically prepended to bash commands when needed.
|
|
38
|
+
|
|
39
|
+
**Manual verification:**
|
|
40
|
+
```bash
|
|
41
|
+
which ruby # Should show: /Users/[user]/.rbenv/shims/ruby
|
|
42
|
+
ruby --version # Should show: ruby 3.4.2
|
|
43
|
+
bundler --version # Should show: Bundler version 2.6.2
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
**Standard setup:**
|
|
8
47
|
```bash
|
|
9
48
|
bin/setup # Install dependencies and setup development environment
|
|
10
49
|
bin/console # Interactive Ruby console for experimentation
|
|
@@ -12,46 +51,217 @@ bin/console # Interactive Ruby console for experimentation
|
|
|
12
51
|
|
|
13
52
|
### CLI Tools Usage
|
|
14
53
|
|
|
15
|
-
|
|
54
|
+
**Installation Note:** When installed as a gem (`gem install appydave-tools`), these tools are available as system commands. During development, run them from `bin/` directory as `bin/script_name.rb`.
|
|
55
|
+
|
|
56
|
+
#### Quick Reference Index
|
|
57
|
+
|
|
58
|
+
| Command | Gem Command | Description | Status |
|
|
59
|
+
|---------|-------------|-------------|--------|
|
|
60
|
+
| **GPT Context** | `gpt_context` | Collect project files for AI context | ⭐ PRIMARY |
|
|
61
|
+
| **YouTube Manager** | `youtube_manager` | CRUD operations on YouTube video metadata | ✅ ACTIVE |
|
|
62
|
+
| **Subtitle Processor** | `subtitle_processor` | Transform SRT files (clean/merge) | ✅ ACTIVE |
|
|
63
|
+
| **Configuration** | `ad_config` | Manage JSON configs (channels, paths, sequences) | ✅ ACTIVE |
|
|
64
|
+
| **Move Images** | N/A (dev only) | Organize video asset images | ✅ ACTIVE |
|
|
65
|
+
| **Prompt Tools** | `prompt_tools` | OpenAI Completion API wrapper | ⚠️ DEPRECATED API |
|
|
66
|
+
| **YouTube Automation** | `youtube_automation` | Prompt sequence runner | ⚠️ INTERNAL USE |
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
#### 1. GPT Context Gatherer (`bin/gpt_context.rb`) ⭐ PRIMARY TOOL
|
|
16
71
|
Collect and organize project files for AI context:
|
|
17
72
|
|
|
18
73
|
```bash
|
|
19
74
|
# Basic usage - gather files with debug output and file output
|
|
20
|
-
gpt_context -i '**/*.rb' -e 'spec/**/*' -d -o context.txt
|
|
75
|
+
bin/gpt_context.rb -i '**/*.rb' -e 'spec/**/*' -d -o context.txt
|
|
21
76
|
|
|
22
77
|
# Multiple formats and patterns
|
|
23
|
-
gpt_context -i 'lib/**/*.rb' -i 'bin/**/*.rb' -f tree,content -d
|
|
78
|
+
bin/gpt_context.rb -i 'lib/**/*.rb' -i 'bin/**/*.rb' -f tree,content -d
|
|
24
79
|
|
|
25
80
|
# Advanced web project filtering
|
|
26
|
-
gpt_context -i 'apps/**/*.ts' -i 'apps/**/*.tsx' -e '**/node_modules/**/*' -e '**/_generated/**/*' -d -f tree -o typescript.txt
|
|
81
|
+
bin/gpt_context.rb -i 'apps/**/*.ts' -i 'apps/**/*.tsx' -e '**/node_modules/**/*' -e '**/_generated/**/*' -d -f tree -o typescript.txt
|
|
27
82
|
|
|
28
83
|
# Tree view only for project structure
|
|
29
|
-
gpt_context -i '**/*' -e '**/node_modules/**/*' -e '.git/**/*' -f tree -d
|
|
84
|
+
bin/gpt_context.rb -i '**/*' -e '**/node_modules/**/*' -e '.git/**/*' -f tree -d
|
|
30
85
|
|
|
31
|
-
# Line-limited content gathering
|
|
32
|
-
gpt_context -i '**/*.rb' -l 20 -f content -d
|
|
86
|
+
# Line-limited content gathering
|
|
87
|
+
bin/gpt_context.rb -i '**/*.rb' -l 20 -f content -d
|
|
33
88
|
|
|
34
89
|
# Multiple output targets
|
|
35
|
-
gpt_context -i 'docs/**/*' -f tree,content -o clipboard -o docs-context.txt
|
|
90
|
+
bin/gpt_context.rb -i 'docs/**/*' -f tree,content -o clipboard -o docs-context.txt
|
|
36
91
|
```
|
|
37
92
|
|
|
38
|
-
|
|
93
|
+
See detailed usage guide: [docs/usage/gpt-context.md](./docs/usage/gpt-context.md)
|
|
94
|
+
|
|
95
|
+
#### 2. YouTube Manager (`bin/youtube_manager.rb`)
|
|
96
|
+
Manage YouTube video metadata via YouTube API:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
# Get video details
|
|
100
|
+
bin/youtube_manager.rb get --video-id VIDEO_ID
|
|
101
|
+
|
|
102
|
+
# Update video metadata
|
|
103
|
+
bin/youtube_manager.rb update --video-id ID --title "New Title"
|
|
104
|
+
bin/youtube_manager.rb update --video-id ID --description "New Description"
|
|
105
|
+
bin/youtube_manager.rb update --video-id ID --tags "tag1,tag2,tag3"
|
|
106
|
+
bin/youtube_manager.rb update --video-id ID --category-id 28
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
**Features:**
|
|
110
|
+
- YouTube API authorization and authentication
|
|
111
|
+
- Retrieve video details, captions, and metadata
|
|
112
|
+
- Update video title, description, tags, and category
|
|
113
|
+
- Generate detailed reports
|
|
114
|
+
|
|
115
|
+
#### 3. Subtitle Processor (`bin/subtitle_processor.rb`)
|
|
116
|
+
Process and manage SRT subtitle files:
|
|
117
|
+
|
|
39
118
|
```bash
|
|
40
|
-
#
|
|
41
|
-
bin/
|
|
42
|
-
|
|
119
|
+
# Clean and normalize SRT files
|
|
120
|
+
bin/subtitle_processor.rb clean -f input.srt -o cleaned.srt
|
|
121
|
+
|
|
122
|
+
# Join multiple SRT files
|
|
123
|
+
bin/subtitle_processor.rb join -d ./subtitles -f "*.srt" -o merged.srt
|
|
124
|
+
bin/subtitle_processor.rb join -f "part1.srt,part2.srt" -s asc -b 100 -o final.srt
|
|
125
|
+
|
|
126
|
+
# Join with options
|
|
127
|
+
bin/subtitle_processor.rb join -d ./subs -f "*.srt" -s inferred -b 200 -o output.srt -L detail
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
**Operations:**
|
|
131
|
+
- **clean**: Removes HTML tags (`<u>`), merges duplicate entries, normalizes spacing
|
|
132
|
+
- **join**: Parses multiple SRT files, adjusts timestamps with buffer, merges timeline
|
|
133
|
+
|
|
134
|
+
**Options:**
|
|
135
|
+
- `-d, --directory` - Directory containing SRT files
|
|
136
|
+
- `-f, --files` - File pattern or comma-separated list
|
|
137
|
+
- `-s, --sort` - Sort order: asc, desc, or inferred (default)
|
|
138
|
+
- `-b, --buffer` - Buffer between files in milliseconds (default: 100)
|
|
139
|
+
- `-L, --log-level` - Log level: none, info, detail
|
|
140
|
+
|
|
141
|
+
**Use cases:** Cleaning YouTube auto-captions, merging FliVideo multi-part recording subtitles
|
|
142
|
+
|
|
143
|
+
**Note:** Renamed from `subtitle_manager` to `subtitle_processor` (accurate - processes files, doesn't manage state)
|
|
43
144
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
bin/subtitle_manager.rb join [options]
|
|
145
|
+
#### 4. Prompt Tools (`bin/prompt_tools.rb`) ⚠️ DEPRECATED API
|
|
146
|
+
OpenAI Completion API wrapper with template support:
|
|
47
147
|
|
|
48
|
-
|
|
49
|
-
|
|
148
|
+
```bash
|
|
149
|
+
# Run prompt from text
|
|
150
|
+
bin/prompt_tools.rb completion -p "Your prompt" -o output.txt
|
|
50
151
|
|
|
51
|
-
#
|
|
52
|
-
bin/
|
|
152
|
+
# Run prompt from file with placeholders
|
|
153
|
+
bin/prompt_tools.rb completion -f template.md -k key1=value1,key2=value2 -c
|
|
53
154
|
```
|
|
54
155
|
|
|
156
|
+
**What it does:**
|
|
157
|
+
- Sends prompts to OpenAI **Completion API** (older GPT-3 models like `davinci-codex`)
|
|
158
|
+
- Supports template files with `{placeholder}` substitution
|
|
159
|
+
- Outputs to file, clipboard, or stdout
|
|
160
|
+
|
|
161
|
+
**Status:** ⚠️ **Not in active use** - Uses **deprecated OpenAI Completion API**
|
|
162
|
+
|
|
163
|
+
**Modern alternative:** Use ChatGPT/Claude directly or migrate to OpenAI Chat API
|
|
164
|
+
|
|
165
|
+
**Potential use cases:** Template-based content generation (if migrated to Chat API)
|
|
166
|
+
|
|
167
|
+
#### 5. YouTube Automation (`bin/youtube_automation.rb`) ⚠️ INTERNAL USE
|
|
168
|
+
Prompt sequence runner for content workflows:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
# Run automation sequence
|
|
172
|
+
bin/youtube_automation.rb -s 01-1
|
|
173
|
+
bin/youtube_automation.rb -s 01-1 -d # with debug output
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
**What it does:**
|
|
177
|
+
- Loads sequence config from `~/.config/appydave/youtube_automation.json`
|
|
178
|
+
- Reads prompt templates from Dropbox path (`_common/raw_prompts/`)
|
|
179
|
+
- Executes OpenAI Completion API calls
|
|
180
|
+
- Saves responses to output files
|
|
181
|
+
|
|
182
|
+
**Requirements:**
|
|
183
|
+
- Sequence definitions in JSON config
|
|
184
|
+
- Prompt template files in configured Dropbox location
|
|
185
|
+
- `OPENAI_ACCESS_TOKEN` environment variable
|
|
186
|
+
|
|
187
|
+
**Status:** ⚠️ **Internal tool** - Hardcoded paths, deprecated API, not documented for external use
|
|
188
|
+
|
|
189
|
+
**Relationship to Move Images:** These are separate tools - Move Images organizes downloaded images into video asset folders
|
|
190
|
+
|
|
191
|
+
**Options:**
|
|
192
|
+
- `-s, --sequence` - Sequence number (required, e.g., 01-1)
|
|
193
|
+
- `-d, --debug` - Enable debug mode
|
|
194
|
+
|
|
195
|
+
#### 6. Configuration Tool (`bin/configuration.rb`)
|
|
196
|
+
Manage appydave-tools configuration files:
|
|
197
|
+
|
|
198
|
+
```bash
|
|
199
|
+
# List all configurations
|
|
200
|
+
bin/configuration.rb -l
|
|
201
|
+
|
|
202
|
+
# Print specific configuration keys
|
|
203
|
+
bin/configuration.rb -p settings,channels
|
|
204
|
+
|
|
205
|
+
# Create missing configuration files
|
|
206
|
+
bin/configuration.rb -c
|
|
207
|
+
|
|
208
|
+
# Edit configurations in VS Code
|
|
209
|
+
bin/configuration.rb -e
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
**Configuration Types:**
|
|
213
|
+
- **settings** - General settings and paths (project folders: content, video, published, abandoned)
|
|
214
|
+
- **channels** - YouTube channel definitions (code, name, youtube_handle)
|
|
215
|
+
- **youtube_automation** - Automation workflow configurations (prompt sequences)
|
|
216
|
+
|
|
217
|
+
**Team Collaboration Features:**
|
|
218
|
+
- **Shareable configs**: JSON files can be version-controlled (no secrets included)
|
|
219
|
+
- **Per-developer paths**: Each team member customizes paths in their `~/.config/appydave/`
|
|
220
|
+
- **Consistent structure**: Same channel codes/names across team
|
|
221
|
+
- **Secrets separation**: API keys stored in `.env` files (gitignored), not in configs
|
|
222
|
+
|
|
223
|
+
#### 7. Move Images (`bin/move_images.rb`)
|
|
224
|
+
Organize and rename downloaded images into video project asset folders.
|
|
225
|
+
|
|
226
|
+
**Purpose:** Batch move and rename images from a download folder into organized video project asset directories with proper naming conventions.
|
|
227
|
+
|
|
228
|
+
```bash
|
|
229
|
+
# Move images to video project assets folder
|
|
230
|
+
bin/move_images.rb -f <project-folder> <section> <prefix>
|
|
231
|
+
|
|
232
|
+
# Example: Move images for intro section of project b40
|
|
233
|
+
bin/move_images.rb -f b40 intro b40
|
|
234
|
+
# Result: Creates files like b40-intro-1.jpg, b40-intro-2.jpg in /path/to/b40/assets/intro/
|
|
235
|
+
|
|
236
|
+
# Example: Move thumbnail images
|
|
237
|
+
bin/move_images.rb -f b40 thumb b40
|
|
238
|
+
# Result: Creates files like b40-thumb-1.jpg, b40-thumb-2.jpg in /path/to/b40/assets/thumb/
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
**Configuration:**
|
|
242
|
+
- **Source:** `~/Sync/smart-downloads/download-images/` (downloads folder)
|
|
243
|
+
- **Destination:** `/Volumes/Expansion/Sync/tube-channels/video-projects/<folder>/assets/<section>/`
|
|
244
|
+
- **Supported sections:** intro, outro, content, teaser, thumb (or any custom section name)
|
|
245
|
+
- **File format:** Processes `.jpg` files only
|
|
246
|
+
|
|
247
|
+
**Arguments:**
|
|
248
|
+
- `-f, --folder` - Project folder name (e.g., b40, b41, etc.)
|
|
249
|
+
- `<section>` - Asset section name (intro, outro, content, teaser, thumb)
|
|
250
|
+
- `<prefix>` - Filename prefix (typically matches project folder)
|
|
251
|
+
|
|
252
|
+
**Workflow:**
|
|
253
|
+
1. Download images to smart-downloads folder
|
|
254
|
+
2. Run move_images with project folder, section, and prefix
|
|
255
|
+
3. Images are renamed sequentially and moved to organized asset directory
|
|
256
|
+
4. Automatically creates section subdirectory if it doesn't exist
|
|
257
|
+
|
|
258
|
+
**Note:** This is a development/workflow tool specific to video project organization. Not installed as a gem command.
|
|
259
|
+
|
|
260
|
+
#### 8. Bank Reconciliation (`bin/bank_reconciliation.rb`) 🗄️ DEPRECATED
|
|
261
|
+
Bank transaction reconciliation tool - **DEPRECATED, DO NOT USE**
|
|
262
|
+
|
|
263
|
+
**WARNING:** This tool contains deprecated code and should not be used for new work. Code has been moved to `lib/appydave/tools/deprecated/bank_reconciliation/`
|
|
264
|
+
|
|
55
265
|
### Testing & Quality
|
|
56
266
|
```bash
|
|
57
267
|
rake spec # Run all RSpec tests
|
|
@@ -76,12 +286,20 @@ gem build # Build gemspec into .gem file
|
|
|
76
286
|
|
|
77
287
|
## Architecture Overview
|
|
78
288
|
|
|
79
|
-
|
|
289
|
+
`appydave-tools` is AppyDave's consolidated productivity toolkit for YouTube content creation. Single-purpose utilities in one repository for easier maintenance than separate codebases.
|
|
290
|
+
|
|
291
|
+
**Philosophy:** See [docs/purpose-and-philosophy.md](./docs/purpose-and-philosophy.md) for project philosophy and design principles.
|
|
292
|
+
|
|
293
|
+
**Tool Categories:**
|
|
294
|
+
- AI & Context Management (GPT Context, Prompt Tools)
|
|
295
|
+
- Content & Media (Subtitles, YouTube Manager, YouTube Automation, Move Images)
|
|
296
|
+
- Configuration (Multi-channel config management)
|
|
80
297
|
|
|
81
298
|
### Core Structure
|
|
82
299
|
- **CLI Tools**: Multiple executable scripts in `bin/` for different functionalities
|
|
83
300
|
- **Modular Design**: Organized into focused modules under `lib/appydave/tools/`
|
|
84
|
-
- **
|
|
301
|
+
- **Independent Operation**: Each tool solves a specific problem standalone
|
|
302
|
+
- **Configuration System**: Flexible config management for multi-project workflows
|
|
85
303
|
- **Type System**: Custom type classes for data validation and transformation
|
|
86
304
|
|
|
87
305
|
### Key Components
|
|
@@ -108,7 +326,7 @@ This is a Ruby gem called `appydave-tools` that provides YouTube automation and
|
|
|
108
326
|
- Caption/subtitle management
|
|
109
327
|
- Detailed reporting capabilities
|
|
110
328
|
|
|
111
|
-
#### Subtitle Management (`lib/appydave/tools/
|
|
329
|
+
#### Subtitle Management (`lib/appydave/tools/subtitle_processor/`)
|
|
112
330
|
- SRT file cleaning and normalization
|
|
113
331
|
- Multi-part subtitle joining
|
|
114
332
|
- Timeline synchronization
|
|
@@ -144,9 +362,300 @@ This is a Ruby gem called `appydave-tools` that provides YouTube automation and
|
|
|
144
362
|
- `clipboard`: System clipboard operations
|
|
145
363
|
- `dotenv`: Environment variable management
|
|
146
364
|
|
|
147
|
-
## Configuration
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
365
|
+
## Configuration Management
|
|
366
|
+
|
|
367
|
+
### Configuration Overview
|
|
368
|
+
The gem uses JSON configuration files for managing YouTube channels, OpenAI settings, and automation workflows.
|
|
369
|
+
|
|
370
|
+
**Configuration Tool:** `bin/configuration.rb`
|
|
371
|
+
|
|
372
|
+
### Setting Up Configuration
|
|
373
|
+
|
|
374
|
+
1. **List available configurations:**
|
|
375
|
+
```bash
|
|
376
|
+
bin/configuration.rb -l
|
|
377
|
+
```
|
|
378
|
+
|
|
379
|
+
2. **Create missing configuration files:**
|
|
380
|
+
```bash
|
|
381
|
+
bin/configuration.rb -c
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
3. **View configuration values:**
|
|
385
|
+
```bash
|
|
386
|
+
bin/configuration.rb -p settings,channels,youtube_automation
|
|
387
|
+
```
|
|
388
|
+
|
|
389
|
+
4. **Edit configurations in VS Code:**
|
|
390
|
+
```bash
|
|
391
|
+
bin/configuration.rb -e
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
### Configuration Types
|
|
395
|
+
|
|
396
|
+
#### 1. Settings Config
|
|
397
|
+
General application settings and paths.
|
|
398
|
+
|
|
399
|
+
**Typical structure:**
|
|
400
|
+
```json
|
|
401
|
+
{
|
|
402
|
+
"project_paths": {
|
|
403
|
+
"content": "/path/to/content",
|
|
404
|
+
"video": "/path/to/video",
|
|
405
|
+
"published": "/path/to/published",
|
|
406
|
+
"abandoned": "/path/to/abandoned"
|
|
407
|
+
}
|
|
408
|
+
}
|
|
409
|
+
```
|
|
410
|
+
|
|
411
|
+
#### 2. Channels Config
|
|
412
|
+
YouTube channel definitions for multi-channel management.
|
|
413
|
+
|
|
414
|
+
**Typical structure:**
|
|
415
|
+
```json
|
|
416
|
+
{
|
|
417
|
+
"channels": [
|
|
418
|
+
{
|
|
419
|
+
"code": "main",
|
|
420
|
+
"name": "Main Channel",
|
|
421
|
+
"youtube_handle": "@channelhandle"
|
|
422
|
+
}
|
|
423
|
+
]
|
|
424
|
+
}
|
|
425
|
+
```
|
|
426
|
+
|
|
427
|
+
#### 3. YouTube Automation Config
|
|
428
|
+
Automation workflow configurations and sequences.
|
|
429
|
+
|
|
430
|
+
#### 4. OpenAI Config (via dotenv)
|
|
431
|
+
OpenAI API integration settings managed through environment variables:
|
|
432
|
+
|
|
433
|
+
```bash
|
|
434
|
+
# .env file
|
|
435
|
+
OPENAI_API_KEY=your_api_key_here
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
### Configuration Locations
|
|
439
|
+
Configuration files are typically stored in:
|
|
440
|
+
- `~/.config/appydave-tools/` (user-level)
|
|
441
|
+
- Or project-specific locations as defined in settings
|
|
442
|
+
|
|
443
|
+
### Environment Variables
|
|
444
|
+
The gem uses `dotenv` for environment variable management. Create a `.env` file in your project root:
|
|
445
|
+
|
|
446
|
+
```bash
|
|
447
|
+
# Required for YouTube API
|
|
448
|
+
GOOGLE_CLIENT_ID=your_client_id
|
|
449
|
+
GOOGLE_CLIENT_SECRET=your_client_secret
|
|
450
|
+
|
|
451
|
+
# Required for OpenAI features
|
|
452
|
+
OPENAI_API_KEY=your_openai_api_key
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
**IMPORTANT:** Never commit `.env` files to version control. Ensure `.env` is in your `.gitignore`.
|
|
456
|
+
|
|
457
|
+
## Git Hooks & Security
|
|
458
|
+
|
|
459
|
+
### Pre-commit Hooks
|
|
460
|
+
This repository uses custom git hooks located in `.githooks/` directory.
|
|
461
|
+
|
|
462
|
+
**Setup:**
|
|
463
|
+
```bash
|
|
464
|
+
git config core.hookspath .githooks
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
**Current pre-commit checks:**
|
|
468
|
+
- ✅ Detects debug code: `binding.pry`, `byebug`, `debugger`
|
|
469
|
+
- ✅ Warns about: `console.log`, `console.dir`
|
|
470
|
+
- ✅ Excludes specific files: Gemfile, .json, .yml files
|
|
471
|
+
- ❌ **Does NOT check for secrets/API keys** (see Security Notes below)
|
|
472
|
+
|
|
473
|
+
**Force commit (bypass hooks):**
|
|
474
|
+
```bash
|
|
475
|
+
git commit --no-verify -m "message"
|
|
476
|
+
```
|
|
477
|
+
|
|
478
|
+
### Security Notes
|
|
479
|
+
|
|
480
|
+
**⚠️ IMPORTANT SECURITY CONSIDERATIONS:**
|
|
481
|
+
|
|
482
|
+
1. **No Secret Scanning:** The current pre-commit hook does NOT scan for:
|
|
483
|
+
- API keys (OpenAI, Google, AWS, etc.)
|
|
484
|
+
- OAuth tokens or credentials
|
|
485
|
+
- Private keys or certificates
|
|
486
|
+
- Credit card numbers or financial data
|
|
487
|
+
- Email addresses or phone numbers
|
|
488
|
+
|
|
489
|
+
2. **Manual Vigilance Required:** Always review your commits for sensitive data before pushing.
|
|
490
|
+
|
|
491
|
+
3. **Recommended Secret Scanning Tools:**
|
|
492
|
+
|
|
493
|
+
#### Option 1: Gitleaks (Recommended for Ruby/Go projects)
|
|
494
|
+
|
|
495
|
+
**Installation (macOS):**
|
|
496
|
+
```bash
|
|
497
|
+
brew install gitleaks
|
|
498
|
+
```
|
|
499
|
+
|
|
500
|
+
**Setup as pre-commit hook:**
|
|
501
|
+
```bash
|
|
502
|
+
# Create .gitleaks.toml configuration
|
|
503
|
+
cat > .gitleaks.toml << 'EOF'
|
|
504
|
+
title = "Gitleaks Configuration"
|
|
505
|
+
|
|
506
|
+
[extend]
|
|
507
|
+
useDefault = true
|
|
508
|
+
|
|
509
|
+
[[rules]]
|
|
510
|
+
description = "AWS Access Key"
|
|
511
|
+
id = "aws-access-key"
|
|
512
|
+
regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}'''
|
|
513
|
+
|
|
514
|
+
[[rules]]
|
|
515
|
+
description = "OpenAI API Key"
|
|
516
|
+
id = "openai-api-key"
|
|
517
|
+
regex = '''sk-[a-zA-Z0-9]{48}'''
|
|
518
|
+
|
|
519
|
+
[[rules]]
|
|
520
|
+
description = "Google OAuth"
|
|
521
|
+
id = "google-oauth"
|
|
522
|
+
regex = '''[0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com'''
|
|
523
|
+
|
|
524
|
+
[[rules]]
|
|
525
|
+
description = "Credit Card Numbers"
|
|
526
|
+
id = "credit-card"
|
|
527
|
+
regex = '''\b(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|6(?:011|5[0-9]{2})[0-9]{12}|(?:2131|1800|35\d{3})\d{11})\b'''
|
|
528
|
+
|
|
529
|
+
[allowlist]
|
|
530
|
+
paths = [
|
|
531
|
+
'''\.md$''',
|
|
532
|
+
'''^\.releaserc\.json$''',
|
|
533
|
+
'''CHANGELOG\.md'''
|
|
534
|
+
]
|
|
535
|
+
EOF
|
|
536
|
+
|
|
537
|
+
# Add to pre-commit hook (append to existing .githooks/pre-commit)
|
|
538
|
+
# Add after line 89 (before final 'end'):
|
|
539
|
+
echo "
|
|
540
|
+
# Gitleaks secret scanning
|
|
541
|
+
echo 'Running gitleaks scan...'
|
|
542
|
+
gitleaks protect --staged --verbose
|
|
543
|
+
if [ \$? -ne 0 ]; then
|
|
544
|
+
echo 'ERROR: Gitleaks found secrets in staged files'
|
|
545
|
+
echo 'Use --no-verify to bypass (NOT RECOMMENDED)'
|
|
546
|
+
exit 1
|
|
547
|
+
fi
|
|
548
|
+
" >> .githooks/pre-commit
|
|
549
|
+
```
|
|
550
|
+
|
|
551
|
+
**Manual scan:**
|
|
552
|
+
```bash
|
|
553
|
+
# Scan staged files
|
|
554
|
+
gitleaks protect --staged
|
|
555
|
+
|
|
556
|
+
# Scan entire repository
|
|
557
|
+
gitleaks detect
|
|
558
|
+
|
|
559
|
+
# Scan git history
|
|
560
|
+
gitleaks detect --verbose --log-opts="--all"
|
|
561
|
+
```
|
|
562
|
+
|
|
563
|
+
#### Option 2: git-secrets (AWS-focused)
|
|
564
|
+
|
|
565
|
+
**Installation (macOS):**
|
|
566
|
+
```bash
|
|
567
|
+
brew install git-secrets
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
**Setup:**
|
|
571
|
+
```bash
|
|
572
|
+
# Initialize git-secrets in repository
|
|
573
|
+
git secrets --install
|
|
574
|
+
git secrets --register-aws
|
|
575
|
+
|
|
576
|
+
# Add custom patterns
|
|
577
|
+
git secrets --add 'sk-[a-zA-Z0-9]{48}' # OpenAI
|
|
578
|
+
git secrets --add '[0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com' # Google OAuth
|
|
579
|
+
```
|
|
580
|
+
|
|
581
|
+
#### Option 3: truffleHog (Deep history scanning)
|
|
582
|
+
|
|
583
|
+
**Installation:**
|
|
584
|
+
```bash
|
|
585
|
+
brew install trufflehog
|
|
586
|
+
```
|
|
587
|
+
|
|
588
|
+
**Usage:**
|
|
589
|
+
```bash
|
|
590
|
+
# Scan entire git history
|
|
591
|
+
trufflehog git file://. --json
|
|
592
|
+
|
|
593
|
+
# Scan from remote
|
|
594
|
+
trufflehog git https://github.com/user/repo
|
|
595
|
+
```
|
|
596
|
+
|
|
597
|
+
4. **`.gitignore` Critical Files:**
|
|
598
|
+
```gitignore
|
|
599
|
+
.env
|
|
600
|
+
.env.*
|
|
601
|
+
*.key
|
|
602
|
+
*.pem
|
|
603
|
+
credentials.json
|
|
604
|
+
client_secret*.json
|
|
605
|
+
**/bank_reconciliation/**/*.csv
|
|
606
|
+
```
|
|
607
|
+
|
|
608
|
+
### Gem Version Management
|
|
609
|
+
|
|
610
|
+
**Current Published Version:** `0.14.1` (on RubyGems.org)
|
|
611
|
+
|
|
612
|
+
**IMPORTANT: Automated Versioning System**
|
|
613
|
+
|
|
614
|
+
This project uses **semantic-release** for automated versioning and publishing. **DO NOT manually edit version files.**
|
|
615
|
+
|
|
616
|
+
**How it works:**
|
|
617
|
+
1. Commits are analyzed using conventional commit messages (feat, fix, chore, docs, etc.)
|
|
618
|
+
2. GitHub Actions CI/CD automatically:
|
|
619
|
+
- Determines next version based on commit types
|
|
620
|
+
- Updates `lib/appydave/tools/version.rb`
|
|
621
|
+
- Updates `CHANGELOG.md`
|
|
622
|
+
- Builds and publishes gem to RubyGems.org
|
|
623
|
+
- Creates git tag
|
|
624
|
+
- Pushes changes back to repository
|
|
625
|
+
|
|
626
|
+
**Commit Message Format:**
|
|
627
|
+
```bash
|
|
628
|
+
feat: add new feature # Triggers minor version bump (0.14.0 → 0.15.0)
|
|
629
|
+
fix: fix bug # Triggers patch version bump (0.14.0 → 0.14.1)
|
|
630
|
+
chore: update dependencies # No version bump
|
|
631
|
+
docs: update readme # No version bump
|
|
632
|
+
|
|
633
|
+
# Breaking change (major version bump)
|
|
634
|
+
feat!: remove deprecated API
|
|
635
|
+
BREAKING CHANGE: removes old API
|
|
636
|
+
|
|
637
|
+
# Or with body
|
|
638
|
+
feat: new feature
|
|
639
|
+
|
|
640
|
+
BREAKING CHANGE: This breaks existing API
|
|
641
|
+
```
|
|
642
|
+
|
|
643
|
+
**Configuration:**
|
|
644
|
+
- **CI/CD:** `.github/workflows/main.yml`
|
|
645
|
+
- **Semantic Release:** `.releaserc.json`
|
|
646
|
+
- **Plugins:**
|
|
647
|
+
- `@semantic-release/commit-analyzer` - Analyzes commits
|
|
648
|
+
- `@semantic-release/release-notes-generator` - Generates changelog
|
|
649
|
+
- `@klueless-js/semantic-release-rubygem` - Publishes to RubyGems
|
|
650
|
+
- `@semantic-release/git` - Commits version bumps
|
|
651
|
+
- `@semantic-release/github` - Creates GitHub releases
|
|
652
|
+
|
|
653
|
+
**Manual Publishing (Emergency Only):**
|
|
654
|
+
If you absolutely must publish manually:
|
|
655
|
+
1. Update version in `lib/appydave/tools/version.rb`
|
|
656
|
+
2. Run tests: `rake spec`
|
|
657
|
+
3. Build gem: `rake build`
|
|
658
|
+
4. Publish: `rake publish`
|
|
659
|
+
5. Tag release: `git tag v0.14.1 && git push --tags`
|
|
660
|
+
|
|
661
|
+
**Note:** Manual publishing will break the automated workflow. Use conventional commits and let CI/CD handle releases.
|