appydave-tools 0.21.2 → 0.22.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +7 -0
  3. data/bin/dam +137 -0
  4. data/docs/README.md +187 -90
  5. data/docs/architecture/dam/dam-cli-enhancements.md +642 -0
  6. data/docs/architecture/dam/dam-cli-implementation-guide.md +1041 -0
  7. data/docs/architecture/dam/dam-data-model.md +466 -0
  8. data/docs/architecture/dam/dam-visualization-requirements.md +641 -0
  9. data/docs/architecture/dam/implementation-roadmap.md +328 -0
  10. data/docs/architecture/dam/jan-collaboration-guide.md +309 -0
  11. data/lib/appydave/tools/dam/s3_operations.rb +57 -5
  12. data/lib/appydave/tools/dam/s3_scanner.rb +139 -0
  13. data/lib/appydave/tools/version.rb +1 -1
  14. data/lib/appydave/tools.rb +1 -0
  15. data/package.json +1 -1
  16. metadata +37 -32
  17. data/docs/development/CODEX-recommendations.md +0 -258
  18. data/docs/development/README.md +0 -100
  19. /data/docs/{development/pattern-comparison.md → architecture/cli/cli-pattern-comparison.md} +0 -0
  20. /data/docs/{development/cli-architecture-patterns.md → architecture/cli/cli-patterns.md} +0 -0
  21. /data/docs/{project-brand-systems-analysis.md → architecture/configuration/configuration-systems.md} +0 -0
  22. /data/docs/{dam → architecture/dam}/dam-vision.md +0 -0
  23. /data/docs/{dam/prd-client-sharing.md → architecture/dam/design-decisions/002-client-sharing.md} +0 -0
  24. /data/docs/{dam/prd-git-integration.md → architecture/dam/design-decisions/003-git-integration.md} +0 -0
  25. /data/docs/{prd-unified-brands-configuration.md → architecture/design-decisions/001-unified-brands-config.md} +0 -0
  26. /data/docs/{dam/session-summary-2025-11-09.md → architecture/design-decisions/session-2025-11-09.md} +0 -0
  27. /data/docs/{configuration/README.md → guides/configuration-setup.md} +0 -0
  28. /data/docs/{dam → guides/platforms}/windows/README.md +0 -0
  29. /data/docs/{dam → guides/platforms}/windows/dam-testing-plan-windows-powershell.md +0 -0
  30. /data/docs/{dam → guides/platforms}/windows/installation.md +0 -0
  31. /data/docs/{tools → guides/tools}/bank-reconciliation.md +0 -0
  32. /data/docs/{tools → guides/tools}/cli-actions.md +0 -0
  33. /data/docs/{tools → guides/tools}/configuration.md +0 -0
  34. /data/docs/{dam → guides/tools/dam}/dam-testing-plan.md +0 -0
  35. /data/docs/{dam/usage.md → guides/tools/dam/dam-usage.md} +0 -0
  36. /data/docs/{tools → guides/tools}/gpt-context.md +0 -0
  37. /data/docs/{tools → guides/tools}/index.md +0 -0
  38. /data/docs/{tools → guides/tools}/move-images.md +0 -0
  39. /data/docs/{tools → guides/tools}/name-manager.md +0 -0
  40. /data/docs/{tools → guides/tools}/prompt-tools.md +0 -0
  41. /data/docs/{tools → guides/tools}/subtitle-processor.md +0 -0
  42. /data/docs/{tools → guides/tools}/youtube-automation.md +0 -0
  43. /data/docs/{tools → guides/tools}/youtube-manager.md +0 -0
  44. /data/docs/{configuration → templates}/.env.example +0 -0
  45. /data/docs/{configuration → templates}/channels.example.json +0 -0
  46. /data/docs/{configuration → templates}/settings.example.json +0 -0
@@ -389,7 +389,21 @@ module Appydave
389
389
 
390
390
  # Calculate MD5 hash of a file
391
391
  def file_md5(file_path)
392
- Digest::MD5.file(file_path).hexdigest
392
+ # Use chunked reading for large files to avoid "Invalid argument @ io_fread" errors
393
+ puts " 🔍 Calculating MD5 for #{File.basename(file_path)}..." if ENV['DEBUG']
394
+ md5 = Digest::MD5.new
395
+ File.open(file_path, 'rb') do |file|
396
+ while (chunk = file.read(8192))
397
+ md5.update(chunk)
398
+ end
399
+ end
400
+ result = md5.hexdigest
401
+ puts " ✓ MD5: #{result[0..7]}..." if ENV['DEBUG']
402
+ result
403
+ rescue StandardError => e
404
+ puts " ⚠️ Warning: Failed to calculate MD5 for #{File.basename(file_path)}: #{e.message}"
405
+ puts ' → Will upload without MD5 comparison'
406
+ nil
393
407
  end
394
408
 
395
409
  # Get MD5 of file in S3 (from ETag)
@@ -413,21 +427,59 @@ module Appydave
413
427
  # Detect MIME type for proper browser handling
414
428
  content_type = detect_content_type(local_file)
415
429
 
416
- File.open(local_file, 'rb') do |file|
417
- s3_client.put_object(
430
+ # For large files, use TransferManager for managed uploads (supports multipart)
431
+ file_size = File.size(local_file)
432
+ start_time = Time.now
433
+
434
+ if file_size > 100 * 1024 * 1024 # > 100MB
435
+ puts " 📤 Uploading large file (#{file_size_human(file_size)})..."
436
+
437
+ # Use TransferManager for multipart upload (modern AWS SDK approach)
438
+ transfer_manager = Aws::S3::TransferManager.new(client: s3_client)
439
+ transfer_manager.upload_file(
440
+ local_file,
418
441
  bucket: brand_info.aws.s3_bucket,
419
442
  key: s3_path,
420
- body: file,
421
443
  content_type: content_type
422
444
  )
445
+ else
446
+ # For smaller files, use direct put_object
447
+ File.open(local_file, 'rb') do |file|
448
+ s3_client.put_object(
449
+ bucket: brand_info.aws.s3_bucket,
450
+ key: s3_path,
451
+ body: file,
452
+ content_type: content_type
453
+ )
454
+ end
423
455
  end
424
456
 
425
- puts " ✓ Uploaded: #{File.basename(local_file)} (#{file_size_human(File.size(local_file))})"
457
+ elapsed = Time.now - start_time
458
+ elapsed_str = format_duration(elapsed)
459
+ puts " ✓ Uploaded: #{File.basename(local_file)} (#{file_size_human(file_size)}) in #{elapsed_str}"
426
460
  true
427
461
  rescue Aws::S3::Errors::ServiceError => e
428
462
  puts " ✗ Failed: #{File.basename(local_file)}"
429
463
  puts " Error: #{e.message}"
430
464
  false
465
+ rescue StandardError => e
466
+ puts " ✗ Failed: #{File.basename(local_file)}"
467
+ puts " Error: #{e.class} - #{e.message}"
468
+ false
469
+ end
470
+
471
+ def format_duration(seconds)
472
+ if seconds < 60
473
+ "#{seconds.round(1)}s"
474
+ elsif seconds < 3600
475
+ minutes = (seconds / 60).floor
476
+ secs = (seconds % 60).round
477
+ "#{minutes}m #{secs}s"
478
+ else
479
+ hours = (seconds / 3600).floor
480
+ minutes = ((seconds % 3600) / 60).floor
481
+ "#{hours}h #{minutes}m"
482
+ end
431
483
  end
432
484
 
433
485
  def detect_content_type(filename)
@@ -0,0 +1,139 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'aws-sdk-s3'
4
+
5
+ module Appydave
6
+ module Tools
7
+ module Dam
8
+ # Scan S3 bucket for project files
9
+ class S3Scanner
10
+ attr_reader :brand_info, :brand, :s3_client
11
+
12
+ def initialize(brand, brand_info: nil, s3_client: nil)
13
+ @brand_info = brand_info || load_brand_info(brand)
14
+ @brand = @brand_info.key
15
+ @s3_client = s3_client || create_s3_client(@brand_info)
16
+ end
17
+
18
+ # Scan S3 for a specific project
19
+ # @param project_id [String] Project ID (e.g., "b65-guy-monroe-marketing-plan")
20
+ # @return [Hash] S3 file data with :file_count, :total_bytes, :last_modified
21
+ def scan_project(project_id)
22
+ bucket = @brand_info.aws.s3_bucket
23
+ prefix = File.join(@brand_info.aws.s3_prefix, project_id, '')
24
+
25
+ puts " 🔍 Scanning #{project_id}..."
26
+
27
+ files = list_s3_objects(bucket, prefix)
28
+
29
+ if files.empty?
30
+ return {
31
+ exists: false,
32
+ file_count: 0,
33
+ total_bytes: 0,
34
+ last_modified: nil
35
+ }
36
+ end
37
+
38
+ total_bytes = files.sum(&:size)
39
+ last_modified = files.map(&:last_modified).max
40
+
41
+ {
42
+ exists: true,
43
+ file_count: files.size,
44
+ total_bytes: total_bytes,
45
+ last_modified: last_modified.utc.iso8601
46
+ }
47
+ rescue Aws::S3::Errors::ServiceError => e
48
+ puts " ⚠️ S3 scan failed for #{project_id}: #{e.message}"
49
+ { exists: false, file_count: 0, total_bytes: 0, last_modified: nil, error: e.message }
50
+ end
51
+
52
+ # Scan all projects in brand's S3 bucket
53
+ # @return [Hash] Map of project_id => scan result
54
+ def scan_all_projects
55
+ bucket = @brand_info.aws.s3_bucket
56
+ prefix = @brand_info.aws.s3_prefix
57
+
58
+ puts "🔍 Scanning all projects in S3: s3://#{bucket}/#{prefix}"
59
+ puts ''
60
+
61
+ # List all "directories" (prefixes) under brand prefix
62
+ project_prefixes = list_s3_prefixes(bucket, prefix)
63
+
64
+ if project_prefixes.empty?
65
+ puts ' 📭 No projects found in S3'
66
+ return {}
67
+ end
68
+
69
+ puts " Found #{project_prefixes.size} projects in S3"
70
+ puts ''
71
+
72
+ results = {}
73
+ project_prefixes.each do |project_id|
74
+ results[project_id] = scan_project(project_id)
75
+ end
76
+
77
+ results
78
+ end
79
+
80
+ private
81
+
82
+ def load_brand_info(brand)
83
+ Appydave::Tools::Configuration::Config.configure
84
+ Appydave::Tools::Configuration::Config.brands.get_brand(brand)
85
+ end
86
+
87
+ def create_s3_client(brand_info)
88
+ profile_name = brand_info.aws.profile
89
+ raise "AWS profile not configured for brand '#{@brand}'" if profile_name.nil? || profile_name.empty?
90
+
91
+ credentials = Aws::SharedCredentials.new(profile_name: profile_name)
92
+
93
+ Aws::S3::Client.new(
94
+ credentials: credentials,
95
+ region: brand_info.aws.region,
96
+ http_wire_trace: false,
97
+ ssl_verify_peer: false
98
+ )
99
+ end
100
+
101
+ # List all objects under a prefix
102
+ def list_s3_objects(bucket, prefix)
103
+ objects = []
104
+ continuation_token = nil
105
+
106
+ loop do
107
+ resp = s3_client.list_objects_v2(
108
+ bucket: bucket,
109
+ prefix: prefix,
110
+ continuation_token: continuation_token
111
+ )
112
+
113
+ objects.concat(resp.contents)
114
+ break unless resp.is_truncated
115
+
116
+ continuation_token = resp.next_continuation_token
117
+ end
118
+
119
+ objects
120
+ end
121
+
122
+ # List project-level prefixes (directories) under brand prefix
123
+ def list_s3_prefixes(bucket, prefix)
124
+ resp = s3_client.list_objects_v2(
125
+ bucket: bucket,
126
+ prefix: prefix,
127
+ delimiter: '/'
128
+ )
129
+
130
+ # common_prefixes returns array of prefixes like "staging/v-appydave/b65-guy-monroe/"
131
+ resp.common_prefixes.map do |cp|
132
+ # Extract project ID from prefix
133
+ File.basename(cp.prefix.chomp('/'))
134
+ end
135
+ end
136
+ end
137
+ end
138
+ end
139
+ end
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Appydave
4
4
  module Tools
5
- VERSION = '0.21.2'
5
+ VERSION = '0.22.0'
6
6
  end
7
7
  end
@@ -56,6 +56,7 @@ require 'appydave/tools/dam/config'
56
56
  require 'appydave/tools/dam/project_resolver'
57
57
  require 'appydave/tools/dam/config_loader'
58
58
  require 'appydave/tools/dam/s3_operations'
59
+ require 'appydave/tools/dam/s3_scanner'
59
60
  require 'appydave/tools/dam/share_operations'
60
61
  require 'appydave/tools/dam/project_listing'
61
62
  require 'appydave/tools/dam/manifest_generator'
data/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "appydave-tools",
3
- "version": "0.21.2",
3
+ "version": "0.22.0",
4
4
  "description": "AppyDave YouTube Automation Tools",
5
5
  "scripts": {
6
6
  "release": "semantic-release"
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: appydave-tools
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.21.2
4
+ version: 0.22.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - David Cruwys
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2025-11-17 00:00:00.000000000 Z
11
+ date: 2025-11-19 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activemodel
@@ -228,42 +228,46 @@ files:
228
228
  - bin/youtube_automation.rb
229
229
  - bin/youtube_manager.rb
230
230
  - docs/README.md
231
+ - docs/architecture/cli/cli-pattern-comparison.md
232
+ - docs/architecture/cli/cli-patterns.md
233
+ - docs/architecture/configuration/configuration-systems.md
234
+ - docs/architecture/dam/dam-cli-enhancements.md
235
+ - docs/architecture/dam/dam-cli-implementation-guide.md
236
+ - docs/architecture/dam/dam-data-model.md
237
+ - docs/architecture/dam/dam-vision.md
238
+ - docs/architecture/dam/dam-visualization-requirements.md
239
+ - docs/architecture/dam/design-decisions/002-client-sharing.md
240
+ - docs/architecture/dam/design-decisions/003-git-integration.md
241
+ - docs/architecture/dam/implementation-roadmap.md
242
+ - docs/architecture/dam/jan-collaboration-guide.md
243
+ - docs/architecture/design-decisions/001-unified-brands-config.md
244
+ - docs/architecture/design-decisions/session-2025-11-09.md
231
245
  - docs/archive/codebase-audit-2025-01.md
232
246
  - docs/archive/documentation-framework-proposal.md
233
247
  - docs/archive/purpose-and-philosophy.md
234
248
  - docs/archive/test-coverage-quick-wins.md
235
249
  - docs/archive/tool-discovery.md
236
250
  - docs/archive/tool-documentation-analysis.md
237
- - docs/configuration/.env.example
238
- - docs/configuration/README.md
239
- - docs/configuration/channels.example.json
240
- - docs/configuration/settings.example.json
241
- - docs/dam/dam-testing-plan.md
242
- - docs/dam/dam-vision.md
243
- - docs/dam/prd-client-sharing.md
244
- - docs/dam/prd-git-integration.md
245
- - docs/dam/session-summary-2025-11-09.md
246
- - docs/dam/usage.md
247
- - docs/dam/windows/README.md
248
- - docs/dam/windows/dam-testing-plan-windows-powershell.md
249
- - docs/dam/windows/installation.md
250
- - docs/development/CODEX-recommendations.md
251
- - docs/development/README.md
252
- - docs/development/cli-architecture-patterns.md
253
- - docs/development/pattern-comparison.md
254
- - docs/prd-unified-brands-configuration.md
255
- - docs/project-brand-systems-analysis.md
256
- - docs/tools/bank-reconciliation.md
257
- - docs/tools/cli-actions.md
258
- - docs/tools/configuration.md
259
- - docs/tools/gpt-context.md
260
- - docs/tools/index.md
261
- - docs/tools/move-images.md
262
- - docs/tools/name-manager.md
263
- - docs/tools/prompt-tools.md
264
- - docs/tools/subtitle-processor.md
265
- - docs/tools/youtube-automation.md
266
- - docs/tools/youtube-manager.md
251
+ - docs/guides/configuration-setup.md
252
+ - docs/guides/platforms/windows/README.md
253
+ - docs/guides/platforms/windows/dam-testing-plan-windows-powershell.md
254
+ - docs/guides/platforms/windows/installation.md
255
+ - docs/guides/tools/bank-reconciliation.md
256
+ - docs/guides/tools/cli-actions.md
257
+ - docs/guides/tools/configuration.md
258
+ - docs/guides/tools/dam/dam-testing-plan.md
259
+ - docs/guides/tools/dam/dam-usage.md
260
+ - docs/guides/tools/gpt-context.md
261
+ - docs/guides/tools/index.md
262
+ - docs/guides/tools/move-images.md
263
+ - docs/guides/tools/name-manager.md
264
+ - docs/guides/tools/prompt-tools.md
265
+ - docs/guides/tools/subtitle-processor.md
266
+ - docs/guides/tools/youtube-automation.md
267
+ - docs/guides/tools/youtube-manager.md
268
+ - docs/templates/.env.example
269
+ - docs/templates/channels.example.json
270
+ - docs/templates/settings.example.json
267
271
  - exe/ad_config
268
272
  - exe/dam
269
273
  - exe/gpt_context
@@ -297,6 +301,7 @@ files:
297
301
  - lib/appydave/tools/dam/repo_status.rb
298
302
  - lib/appydave/tools/dam/repo_sync.rb
299
303
  - lib/appydave/tools/dam/s3_operations.rb
304
+ - lib/appydave/tools/dam/s3_scanner.rb
300
305
  - lib/appydave/tools/dam/share_operations.rb
301
306
  - lib/appydave/tools/dam/status.rb
302
307
  - lib/appydave/tools/dam/sync_from_ssd.rb
@@ -1,258 +0,0 @@
1
- # CODEX Recommendations - Review & Status
2
-
3
- > Last updated: 2025-11-10 11:01:36 UTC
4
- > Original recommendations provided by Codex (GPT-5) on 2025-11-09
5
-
6
- This document captures Codex's architectural recommendations with implementation status and verdicts after engineering review.
7
-
8
- ## Executive Summary
9
-
10
- **Overall Assessment:** Mixed recommendations - some valuable, some outdated, some architecturally inappropriate.
11
-
12
- **Implemented:** ✅ P3 (Filesystem fixtures)
13
- **Rejected:** ❌ P0 (Configuration DI), P1 (RuboCop - already clean), P4 (method_missing removal)
14
- **Deferred:** ⚠️ P2 (IO separation - CLI tool doesn't need it)
15
- **Future Work:** 🔍 VAT Manifest bugs (valid technical debt)
16
-
17
- ---
18
-
19
- ## Priority Recommendations
20
-
21
- ### ✅ P3: Standardize Filesystem Fixtures (IMPLEMENTED)
22
-
23
- **Recommendation:** Extract `Dir.mktmpdir + FileUtils.mkdir_p` boilerplate into shared RSpec context.
24
-
25
- **Status:** ✅ **Implemented** (2025-11-10)
26
-
27
- **What was done:**
28
- - Created `spec/support/vat_filesystem_helpers.rb` with shared contexts
29
- - `include_context 'vat filesystem'` - provides temp_folder, projects_root, auto-cleanup
30
- - `include_context 'vat filesystem with brands', brands: %w[appydave voz]` - adds brand path helpers
31
- - Refactored 3 VAT specs: config_spec, project_resolver_spec, config_loader_spec
32
- - All tests passing (149 examples, 0 failures)
33
-
34
- **Benefits delivered:**
35
- - Reduced duplication across VAT specs
36
- - Centralized cleanup logic (safer tests)
37
- - Easier to maintain and extend
38
-
39
- ---
40
-
41
- ### ❌ P1: RuboCop Cleanup (ALREADY COMPLETE)
42
-
43
- **Recommendation:** Run `rubocop --auto-correct` to fix 93 offenses, track 15 manual fixes.
44
-
45
- **Status:** ❌ **Obsolete** - RuboCop already clean
46
-
47
- **Current state:**
48
- ```bash
49
- bundle exec rubocop
50
- # => 103 files inspected, no offenses detected
51
- ```
52
-
53
- **Verdict:** The recommendations document was based on outdated codebase state. No action needed.
54
-
55
- ---
56
-
57
- ### ❌ P0: Configuration Dependency Injection (REJECTED)
58
-
59
- **Recommendation:** Replace `allow_any_instance_of(SettingsConfig)` with dependency injection pattern:
60
- ```ruby
61
- # Proposed:
62
- Config.projects_root(settings: custom_settings)
63
- ```
64
-
65
- **Status:** ❌ **Rejected** - Architecturally inappropriate
66
-
67
- **Why rejected:**
68
- 1. **Singleton pattern is correct for configuration** - Global config state is intentional
69
- 2. **Breaking API change** - Would require threading `settings:` through entire call chain:
70
- - `bin/vat` → `ProjectResolver` → `Config.brand_path` → `Config.projects_root`
71
- - Every method needs new parameter (massive churn)
72
- 3. **Tests work correctly** - `allow_any_instance_of` is intentionally allowed in `.rubocop.yml`
73
- 4. **No real benefit** - Adds complexity without solving actual problems
74
-
75
- **Codex's concern:** "Tests must stub *every* SettingsConfig instance"
76
-
77
- **Reality:** This is fine. Configuration is a singleton. Testing strategy is appropriate.
78
-
79
- **Lesson for Codex:** Dependency injection is not always superior to singleton patterns. Context matters. CLI tools with global configuration state don't benefit from DI complexity.
80
-
81
- ---
82
-
83
- ### ❌ P4: Remove method_missing from Configuration::Config (REJECTED)
84
-
85
- **Recommendation:** Replace `method_missing` with explicit reader methods or `Forwardable`.
86
-
87
- **Status:** ❌ **Rejected** - This is a design pattern, not a code smell
88
-
89
- **Why rejected:**
90
- 1. **Registry pattern** - `method_missing` enables dynamic configuration registration:
91
- ```ruby
92
- Config.register(:settings, SettingsConfig)
93
- Config.register(:channels, ChannelsConfig)
94
- Config.settings # Dynamic dispatch via method_missing
95
- ```
96
- 2. **Proper implementation** - Has `respond_to_missing?` (Ruby best practice ✅)
97
- 3. **Good error handling** - Clear messages listing available configs
98
- 4. **Plugin architecture** - Can add new configs without modifying `Config` class
99
-
100
- **Codex's concern:** "Hides failures until runtime and complicates auto-complete"
101
-
102
- **Reality:** This is a common Ruby pattern (Rails uses it extensively). The implementation is correct.
103
-
104
- **Lesson for Codex:** `method_missing` is not inherently bad. When properly implemented with `respond_to_missing?` and clear errors, it enables powerful metaprogramming patterns. Don't dogmatically avoid it.
105
-
106
- ---
107
-
108
- ### ⚠️ P2: Decouple Terminal IO from VAT Services (DEFERRED)
109
-
110
- **Recommendation:** Extract interactive prompts from `ProjectResolver.resolve` business logic.
111
-
112
- **Codex's concern:** Interactive `puts`/`$stdin.gets` blocks automation agents.
113
-
114
- **Status:** ⚠️ **Low priority** - Not needed for current use case
115
-
116
- **Why deferred:**
117
- 1. **CLI-only tool** - VAT is a command-line interface, not a library
118
- 2. **Intentional UX** - Interactive prompts provide good user experience for ambiguous cases
119
- 3. **No automation use cases** - Agents use exact project names, don't trigger prompts
120
- 4. **Current code location:** `lib/appydave/tools/vat/project_resolver.rb:41-49`
121
-
122
- **When to revisit:** If VAT needs programmatic API for automation tools, add non-interactive mode:
123
- ```ruby
124
- def resolve(brand, project_hint, interactive: true)
125
- # Return all matches if !interactive (for automation)
126
- end
127
- ```
128
-
129
- **Lesson for Codex:** Not all code needs maximum abstraction. CLI tools can have terminal IO in business logic if that's their primary use case.
130
-
131
- ---
132
-
133
- ## Architecture-Wide Observations
134
-
135
- ### ✅ Valid Technical Debt: VAT Manifest Generator
136
-
137
- **Issues identified (lines 116-125 in original doc):**
138
-
139
- 1. **Archived projects silently dropped** - `collect_project_ids` rejects archived folder entirely
140
- 2. **SSD paths lose grouping context** - Stores only `project_id`, not `range/project_id`
141
- 3. **Heavy file detection shallow** - Only checks top-level, misses nested videos
142
- 4. **Quadratic disk scanning** - Walks every file twice per project
143
- 5. **Code duplication** - Standalone `bin/generate_manifest.rb` diverged from lib class
144
-
145
- **Status:** 🔍 **Acknowledged as real bugs** - Worth investigating
146
-
147
- **Note:** These are legitimate technical debt items, not style preferences. Recommend creating GitHub issues for tracking.
148
-
149
- ### 🔍 DAM Manifest & Sync Addendum (2025-11-10)
150
-
151
- Phase 1 added S3/git metadata, but several inconsistencies remain between the manifest generator and the SSD sync tooling:
152
-
153
- - **Range directories are inconsistent:** `ManifestGenerator#determine_range` groups projects in 50-count buckets per letter (`lib/appydave/tools/dam/manifest_generator.rb:271-285`), but `SyncFromSsd#determine_range` still assumes only `b`-series projects and uses 10-count buckets (`lib/appydave/tools/dam/sync_from_ssd.rb:186-196`). Any non-`b` brand (AITLDR, VOZ) or the new 50-count scheme will pick the wrong destination folder.
154
- - **SSD paths are lossy:** Manifests store `storage[:ssd][:path] = project_id` even when the data actually lives under `ssd/<range>/<project>` (`lib/appydave/tools/dam/manifest_generator.rb:178-185`). `SyncFromSsd#sync_project` then only checks `ssd/<project>` (`lib/appydave/tools/dam/sync_from_ssd.rb:162-170`), so range-based backups always report “SSD path not found” despite `ssd_exists: true` in the manifest. Persist the relative range folder so both tools agree on the same layout.
155
- - **Heavy file detection still stops at the top level:** `heavy_files?` only scans `dir/*.{mp4,...}` (`lib/appydave/tools/dam/manifest_generator.rb:314-318`), so nested footage (e.g., `final/video.mp4`) reports `has_heavy_files: false`, skewing SSD/cleanup metrics. Mirror the recursive approach used in `light_files?`.
156
- - **Exclude patterns are fragile:** `SyncFromSsd#excluded_file?` strips `**/` from patterns and only compares path segments (`lib/appydave/tools/dam/sync_from_ssd.rb:198-223`), which means globs like `**/.DS_Store` or `**/*.lock` do not behave like actual glob patterns. Replace the manual parsing with `File.fnmatch?(pattern, relative_path, File::FNM_PATHNAME | File::FNM_DOTMATCH)` so `.git`, `.turbo`, etc., are consistently ignored.
157
-
158
- Action: Align the manifest and sync code on the same range conventions and relative paths before we rely on Phase 2 git workflows; otherwise `dam sync-ssd` will never restore the projects that manifests say exist.
159
-
160
- ### ⚠️ DAM Git Workflow (Phase 2)
161
-
162
- - **Project resolver instantiated incorrectly:** Both `Status#resolve_project_path` and `RepoPush#validate_project` call `ProjectResolver.new.resolve` (`lib/appydave/tools/dam/status.rb:36-40`, `lib/appydave/tools/dam/repo_push.rb:45-63`), but `ProjectResolver` only exposes class methods inside `class << self`. These code paths raise `NoMethodError` the moment you ask for project status or run `dam repo-push … <project>`. Switch to `ProjectResolver.resolve(...)` (or add an instance API) before shipping.
163
- - **Auto-detected brands pollute configuration:** When you run `dam status` from inside `v-appydave`, the auto-detect logic passes the literal `v-appydave` string (`bin/dam:269-290`) into `Status`, which in turn calls `Config.git_remote`. That method persists the inferred remote under whatever key it was given (`lib/appydave/tools/dam/config.rb:43-71`), so a new `v-appydave` entry gets written to `brands.json`, duplicating the real `appydave` record. Normalize auto-detected names back to the canonical brand key before calling configuration APIs.
164
- - **Naming drift in the CLI:** `bin/dam` still defines `class VatCLI` (line 11), so stack traces and help output reference the old VAT class name. Rename the class (and any references) to avoid confusion when both `vat` and `dam` binaries coexist.
165
-
166
- ---
167
-
168
- ### ⚠️ CLI Standardization (Worth Auditing)
169
-
170
- **Observation:** Not all bin scripts use `BaseAction` pattern consistently.
171
-
172
- **Example:** `bin/gpt_context.rb` hand-rolls `OptionParser` instead of using `lib/appydave/tools/cli_actions/base_action.rb`.
173
-
174
- **Status:** ⚠️ **Worth reviewing** for consistency
175
-
176
- **Action:** Audit which CLI scripts follow standard patterns vs. custom implementations.
177
-
178
- ---
179
-
180
- ## Lessons Learned (for future Codex reviews)
181
-
182
- ### What Codex got right:
183
- 1. ✅ **Filesystem fixtures** - Practical refactoring with clear benefits
184
- 2. ✅ **Manifest bugs** - Identified real logic issues worth fixing
185
- 3. ✅ **CLI consistency** - Valid observation about pattern divergence
186
-
187
- ### Where Codex was dogmatic:
188
- 1. ❌ **Dependency injection everywhere** - Not all singletons need DI
189
- 2. ❌ **Avoid method_missing** - Valid Ruby pattern when done correctly
190
- 3. ❌ **Separate all IO** - CLI tools can mix IO with logic appropriately
191
-
192
- ### What Codex missed:
193
- 1. **Current state validation** - Recommended RuboCop fixes already applied
194
- 2. **Cost/benefit analysis** - P0 config adapter would break entire API for minimal gain
195
- 3. **Context awareness** - CLI tools have different constraints than libraries
196
-
197
- ---
198
-
199
- ## Conclusion
200
-
201
- **Codex recommendations score: 4/10**
202
-
203
- **Good advice:**
204
- - Filesystem fixture extraction (implemented ✅)
205
- - Manifest generator bugs (valid technical debt 🔍)
206
- - CLI standardization audit (worth reviewing ⚠️)
207
-
208
- **Bad advice:**
209
- - Configuration dependency injection (wrong pattern for this use case ❌)
210
- - Remove method_missing (misunderstands design pattern ❌)
211
- - Outdated RuboCop recommendations (already fixed ❌)
212
-
213
- **Key takeaway:** Mix pragmatic refactoring suggestions with dogmatic "purity" recommendations. Cherry-pick the valuable insights, reject the inappropriate ones.
214
-
215
- ---
216
-
217
- ## Implementation Notes
218
-
219
- ### P3 Filesystem Fixtures - Details
220
-
221
- **Files created:**
222
- - `spec/support/vat_filesystem_helpers.rb`
223
-
224
- **Shared contexts:**
225
- ```ruby
226
- # Basic fixture
227
- include_context 'vat filesystem'
228
- # => Provides: temp_folder, projects_root, auto-cleanup, config mocking
229
-
230
- # With brand directories
231
- include_context 'vat filesystem with brands', brands: %w[appydave voz]
232
- # => Also provides: appydave_path, voz_path (auto-created)
233
- ```
234
-
235
- **Files refactored:**
236
- - `spec/appydave/tools/vat/config_spec.rb` (removed 11 lines boilerplate)
237
- - `spec/appydave/tools/vat/project_resolver_spec.rb` (removed 18 lines boilerplate)
238
- - `spec/appydave/tools/vat/config_loader_spec.rb` (removed 9 lines boilerplate)
239
-
240
- **Test results:**
241
- - 149 VAT spec examples, 0 failures
242
- - Coverage: 76.38% (2131/2790 lines)
243
-
244
- ---
245
-
246
- **Document maintained by:** AppyDave engineering team
247
- **Next review:** After addressing VAT manifest bugs
248
-
249
- ## Communication Patterns & Practices
250
-
251
- Because this document is now a shared artifact between CODEx and Claude, align on the following collaboration rules so recommendations stay constructive and actionable:
252
-
253
- 1. **State of the world first:** When responding to a recommendation, cite the current repo evidence (commit, test output, spec path) before giving a verdict. This keeps future readers from guessing which version you inspected.
254
- 2. **Assume positive intent:** Frame disagreements in terms of trade-offs (“we prefer singletons here because…”) rather than absolutes. If a suggestion doesn’t fit today, note what signal would make you revisit it.
255
- 3. **Acknowledge deltas:** When new findings arrive (e.g., Ruby version mismatch), summarize them here so both agents see the updated context even if the original section came from someone else.
256
- 4. **Track actionability:** For every open item, tag it as ✅ implemented, ⚠️ deferred with trigger, or 🔍 debt worth filing. Avoid leaving “bad pattern” remarks without a next step.
257
- 5. **Link evidence:** Reference commands (`bundle exec rubocop`), file paths (`bin/vat:160`), or PRs so the other agent can reproduce your conclusion quickly.
258
- 6. **Close the loop:** When you adopt or reject a suggestion, leave a brief rationale in this doc instead of burying it in chat. That keeps the shared history centralized.