appydave-tools 0.77.1 → 0.77.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -0
- data/docs/planning/BACKLOG.md +9 -6
- data/docs/planning/batch-a-features/IMPLEMENTATION_PLAN.md +5 -5
- data/docs/planning/s3-operations-split/AGENTS.md +686 -0
- data/docs/planning/s3-operations-split/IMPLEMENTATION_PLAN.md +42 -0
- data/lib/appydave/tools/dam/s3_base.rb +310 -0
- data/lib/appydave/tools/dam/s3_operations.rb +15 -314
- data/lib/appydave/tools/version.rb +1 -1
- data/lib/appydave/tools.rb +1 -0
- data/package.json +1 -1
- metadata +4 -1
|
@@ -0,0 +1,686 @@
|
|
|
1
|
+
# AGENTS.md — AppyDave Tools / s3-operations-split campaign
|
|
2
|
+
|
|
3
|
+
> Self-contained operational knowledge. You receive only this file + your work unit prompt.
|
|
4
|
+
> Inherited from: batch-a-features (2026-03-20)
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Project Overview
|
|
9
|
+
|
|
10
|
+
**Stack:** Ruby 3.4.2, Bundler 2.6.2, RSpec, RuboCop, semantic-release CI/CD.
|
|
11
|
+
**Baseline:** 870 examples, 0 failures, 86.47% line coverage, v0.77.1
|
|
12
|
+
**Commits:** `kfeat "message"` for new features (minor bump), `kfix "message"` for fixes (patch bump). Never `git commit` directly.
|
|
13
|
+
**Wave size:** 1 — all work units are SEQUENTIAL. Do not attempt parallel execution. Each WU modifies s3_operations.rb AND creates a new file; concurrent edits would conflict.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## ⚠️ kfix/kfeat Staging Behaviour
|
|
18
|
+
|
|
19
|
+
`kfix` and `kfeat` run `git add .` internally — they stage EVERYTHING in the working tree.
|
|
20
|
+
|
|
21
|
+
**Before calling kfix/kfeat:**
|
|
22
|
+
```bash
|
|
23
|
+
git status # confirm ONLY your intended files are modified
|
|
24
|
+
git diff # review the actual changes
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
If unintended files appear:
|
|
28
|
+
```bash
|
|
29
|
+
git checkout -- path/to/unintended/file # discard specific file
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Then call kfix/kfeat once the tree is clean.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Build & Run Commands
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
eval "$(rbenv init -)"
|
|
40
|
+
|
|
41
|
+
RUBYOPT="-W0" bundle exec rspec # Full suite (870 examples baseline)
|
|
42
|
+
bundle exec rspec spec/path/to/file_spec.rb # Single file
|
|
43
|
+
bundle exec rubocop --format clang # Lint (must be 0 offenses)
|
|
44
|
+
|
|
45
|
+
kfeat "add feature description" # new feature — minor version bump
|
|
46
|
+
kfix "fix description" # fix/improvement — patch version bump
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Architecture Overview — What We're Splitting
|
|
52
|
+
|
|
53
|
+
`lib/appydave/tools/dam/s3_operations.rb` is 1,021 lines with:
|
|
54
|
+
- Mixed I/O (`puts` everywhere) and computation (MD5, path building)
|
|
55
|
+
- All upload, download, status, archive logic in one class
|
|
56
|
+
- Used by: `bin/dam` (8 call sites), `project_listing.rb` (calculate_sync_status + sync_timestamps), `status.rb` (indirectly via project_listing)
|
|
57
|
+
|
|
58
|
+
### Target Architecture
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
S3Base (shared infrastructure + shared helpers)
|
|
62
|
+
├── S3Uploader < S3Base (upload only)
|
|
63
|
+
├── S3Downloader < S3Base (download only)
|
|
64
|
+
├── S3StatusChecker < S3Base (status, calculate_sync_status, sync_timestamps)
|
|
65
|
+
├── S3Archiver < S3Base (archive, cleanup, cleanup_local)
|
|
66
|
+
└── S3Operations < S3Base (thin facade — delegates to above 4)
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
**S3Operations inherits from S3Base** so that existing specs calling `.send(:build_s3_key, ...)` etc. continue to work. Public methods delegate to focused sub-classes.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Current S3Operations Method Map
|
|
74
|
+
|
|
75
|
+
Use this as your reference when deciding where each method belongs.
|
|
76
|
+
|
|
77
|
+
### Constructor + Client (→ S3Base)
|
|
78
|
+
- L32 `initialize(brand, project_id, brand_info: nil, brand_path: nil, s3_client: nil)`
|
|
79
|
+
- L43 `s3_client` (public, lazy-loaded)
|
|
80
|
+
- L49 `load_brand_info(brand)` (private)
|
|
81
|
+
- L55 `project_directory_path` (private)
|
|
82
|
+
- L65 `determine_aws_profile(brand_info)` (private)
|
|
83
|
+
- L85 `create_s3_client(brand_info)` (private)
|
|
84
|
+
- L102 `configure_ssl_options` (private)
|
|
85
|
+
|
|
86
|
+
### Shared Helpers (→ S3Base)
|
|
87
|
+
- L590 `build_s3_key(relative_path)` (public used by specs via send)
|
|
88
|
+
- L595 `extract_relative_path(s3_key)` (public used by specs via send)
|
|
89
|
+
- L600 `file_md5(file_path)` (private)
|
|
90
|
+
- L619 `s3_file_md5(s3_path)` (private)
|
|
91
|
+
- L631 `multipart_etag?(etag)` (private)
|
|
92
|
+
- L640 `compare_files(local_file:, s3_etag:, s3_size:)` (private)
|
|
93
|
+
- L660 `s3_file_size(s3_path)` (private)
|
|
94
|
+
- L721 `format_duration(seconds)` (private)
|
|
95
|
+
- L735 `format_time_ago(seconds)` (private)
|
|
96
|
+
- L850 `list_s3_files` (private — used by upload, download, status, calculate_sync_status, sync_timestamps)
|
|
97
|
+
- L873 `get_s3_file_info(s3_key)` (private — used by upload)
|
|
98
|
+
- L902 `file_size_human(bytes)` (private)
|
|
99
|
+
- L973 `excluded_path?(relative_path)` (private — used by upload + copy_with_exclusions)
|
|
100
|
+
|
|
101
|
+
### Upload Operations (→ S3Uploader)
|
|
102
|
+
- L111 `upload(dry_run: false)` (public)
|
|
103
|
+
- L671 `upload_file(local_file, s3_path, dry_run: false)` (private)
|
|
104
|
+
- L757 `detect_content_type(filename)` (private)
|
|
105
|
+
|
|
106
|
+
### Download Operations (→ S3Downloader)
|
|
107
|
+
- L188 `download(dry_run: false)` (public)
|
|
108
|
+
- L790 `download_file(s3_key, local_file, dry_run: false)` (private)
|
|
109
|
+
|
|
110
|
+
### Status Operations (→ S3StatusChecker)
|
|
111
|
+
- L260 `status` (public)
|
|
112
|
+
- L495 `calculate_sync_status` (public — called by project_listing.rb)
|
|
113
|
+
- L562 `sync_timestamps` (public — called by project_listing.rb)
|
|
114
|
+
- L890 `list_local_files(staging_dir)` (private)
|
|
115
|
+
|
|
116
|
+
### Archive Operations (→ S3Archiver)
|
|
117
|
+
- L354 `cleanup(force: false, dry_run: false)` (public)
|
|
118
|
+
- L393 `cleanup_local(force: false, dry_run: false)` (public)
|
|
119
|
+
- L448 `archive(force: false, dry_run: false)` (public)
|
|
120
|
+
- L818 `delete_s3_file(s3_key, dry_run: false)` (private)
|
|
121
|
+
- L836 `delete_local_file(file_path, dry_run: false)` (private)
|
|
122
|
+
- L915 `copy_to_ssd(source_dir, dest_dir, dry_run: false)` (private)
|
|
123
|
+
- L949 `copy_with_exclusions(source_dir, dest_dir)` (private)
|
|
124
|
+
- L990 `delete_local_project(project_dir, dry_run: false)` (private)
|
|
125
|
+
- L1015 `calculate_directory_size(dir_path)` (private)
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## Directory Structure
|
|
130
|
+
|
|
131
|
+
```
|
|
132
|
+
lib/appydave/tools/
|
|
133
|
+
dam/
|
|
134
|
+
s3_operations.rb EXISTING — shrinks to thin facade (~70 lines after all WUs)
|
|
135
|
+
s3_base.rb NEW (WU1) — shared infrastructure + shared helpers
|
|
136
|
+
s3_uploader.rb NEW (WU2) — upload only
|
|
137
|
+
s3_downloader.rb NEW (WU3) — download only
|
|
138
|
+
s3_status_checker.rb NEW (WU4) — status, calculate_sync_status, sync_timestamps
|
|
139
|
+
s3_archiver.rb NEW (WU5) — archive, cleanup, cleanup_local
|
|
140
|
+
tools.rb MODIFY (WU5) — add requires for new files
|
|
141
|
+
spec/appydave/tools/dam/
|
|
142
|
+
s3_operations_spec.rb EXISTING — do NOT touch until all 5 WUs complete
|
|
143
|
+
s3_base_spec.rb (optional, WU1) — only if time allows
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
## WU1: Extract S3Base
|
|
149
|
+
|
|
150
|
+
**File to create:** `lib/appydave/tools/dam/s3_base.rb`
|
|
151
|
+
**File to modify:** `lib/appydave/tools/dam/s3_operations.rb`
|
|
152
|
+
|
|
153
|
+
### What to build
|
|
154
|
+
|
|
155
|
+
Create S3Base containing everything from the "Constructor + Client" and "Shared Helpers" sections above.
|
|
156
|
+
|
|
157
|
+
```ruby
|
|
158
|
+
# frozen_string_literal: true
|
|
159
|
+
|
|
160
|
+
require 'fileutils'
|
|
161
|
+
require 'json'
|
|
162
|
+
require 'digest'
|
|
163
|
+
require 'aws-sdk-s3'
|
|
164
|
+
|
|
165
|
+
module Appydave
|
|
166
|
+
module Tools
|
|
167
|
+
module Dam
|
|
168
|
+
# Shared infrastructure and helpers for S3 operations.
|
|
169
|
+
# All S3 operation classes inherit from this base.
|
|
170
|
+
class S3Base
|
|
171
|
+
attr_reader :brand_info, :brand, :project_id, :brand_path
|
|
172
|
+
|
|
173
|
+
EXCLUDE_PATTERNS = %w[
|
|
174
|
+
**/node_modules/**
|
|
175
|
+
**/.git/**
|
|
176
|
+
**/.next/**
|
|
177
|
+
**/dist/**
|
|
178
|
+
**/build/**
|
|
179
|
+
**/out/**
|
|
180
|
+
**/.cache/**
|
|
181
|
+
**/coverage/**
|
|
182
|
+
**/.turbo/**
|
|
183
|
+
**/.vercel/**
|
|
184
|
+
**/tmp/**
|
|
185
|
+
**/.DS_Store
|
|
186
|
+
**/*:Zone.Identifier
|
|
187
|
+
].freeze
|
|
188
|
+
|
|
189
|
+
def initialize(brand, project_id, brand_info: nil, brand_path: nil, s3_client: nil)
|
|
190
|
+
# ... (copy exactly from s3_operations.rb)
|
|
191
|
+
end
|
|
192
|
+
|
|
193
|
+
def s3_client
|
|
194
|
+
# ... (copy exactly)
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
private
|
|
198
|
+
|
|
199
|
+
# All private infrastructure methods ...
|
|
200
|
+
# All shared helper methods ...
|
|
201
|
+
end
|
|
202
|
+
end
|
|
203
|
+
end
|
|
204
|
+
end
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
### Modify S3Operations after creating S3Base
|
|
208
|
+
|
|
209
|
+
Change the class declaration to inherit from S3Base, and REMOVE all methods that are now in S3Base:
|
|
210
|
+
|
|
211
|
+
```ruby
|
|
212
|
+
# frozen_string_literal: true
|
|
213
|
+
|
|
214
|
+
module Appydave
|
|
215
|
+
module Tools
|
|
216
|
+
module Dam
|
|
217
|
+
# Facade for S3 operations — delegates to focused sub-classes.
|
|
218
|
+
# Inherits shared helpers from S3Base for backward-compatible spec access.
|
|
219
|
+
class S3Operations < S3Base
|
|
220
|
+
# upload, download, status, cleanup, cleanup_local, archive
|
|
221
|
+
# (these remain here for now — moved out in WU2-WU5)
|
|
222
|
+
end
|
|
223
|
+
end
|
|
224
|
+
end
|
|
225
|
+
end
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
**Important**: S3Operations must keep `require 'fileutils'` etc. removed since S3Base now has them. But s3_base.rb has the requires, and since s3_operations.rb will require s3_base via the autoload chain, that's fine.
|
|
229
|
+
|
|
230
|
+
Actually: DO NOT add `require` calls inside individual lib files — the load order is managed by `lib/appydave/tools.rb`. Just make the class inherit.
|
|
231
|
+
|
|
232
|
+
### Modify lib/appydave/tools.rb (WU1)
|
|
233
|
+
|
|
234
|
+
Add `require 'appydave/tools/dam/s3_base'` on a NEW line immediately BEFORE line 70 (`require 'appydave/tools/dam/s3_operations'`). Without this, S3Operations < S3Base will fail at load time.
|
|
235
|
+
|
|
236
|
+
```ruby
|
|
237
|
+
# Line 69: require 'appydave/tools/dam/config_loader'
|
|
238
|
+
require 'appydave/tools/dam/s3_base' # ADD THIS
|
|
239
|
+
require 'appydave/tools/dam/s3_operations' # EXISTING line 70
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
### Done when WU1 is complete
|
|
243
|
+
- `lib/appydave/tools/dam/s3_base.rb` created with all infrastructure + helpers
|
|
244
|
+
- `lib/appydave/tools/dam/s3_operations.rb` still has upload/download/status/cleanup/archive methods intact, but class is now `S3Operations < S3Base` and constructor + shared helpers are removed
|
|
245
|
+
- `lib/appydave/tools/tools.rb` has new s3_base require before s3_operations
|
|
246
|
+
- `RUBYOPT="-W0" bundle exec rspec` → 870 examples, 0 failures
|
|
247
|
+
- `bundle exec rubocop --format clang` → 0 offenses
|
|
248
|
+
- Commit: `kfix "extract S3Base with shared infrastructure and helpers from S3Operations"`
|
|
249
|
+
|
|
250
|
+
---
|
|
251
|
+
|
|
252
|
+
## WU2: Extract S3Uploader
|
|
253
|
+
|
|
254
|
+
**Prerequisite:** WU1 complete.
|
|
255
|
+
**File to create:** `lib/appydave/tools/dam/s3_uploader.rb`
|
|
256
|
+
**File to modify:** `lib/appydave/tools/dam/s3_operations.rb`
|
|
257
|
+
|
|
258
|
+
### What to build
|
|
259
|
+
|
|
260
|
+
```ruby
|
|
261
|
+
# frozen_string_literal: true
|
|
262
|
+
|
|
263
|
+
module Appydave
|
|
264
|
+
module Tools
|
|
265
|
+
module Dam
|
|
266
|
+
# Handles S3 upload operations.
|
|
267
|
+
class S3Uploader < S3Base
|
|
268
|
+
def upload(dry_run: false)
|
|
269
|
+
# MOVE exactly from s3_operations.rb L111-L185
|
|
270
|
+
end
|
|
271
|
+
|
|
272
|
+
private
|
|
273
|
+
|
|
274
|
+
def upload_file(local_file, s3_path, dry_run: false)
|
|
275
|
+
# MOVE exactly from s3_operations.rb L671-L719
|
|
276
|
+
end
|
|
277
|
+
|
|
278
|
+
def detect_content_type(filename)
|
|
279
|
+
# MOVE exactly from s3_operations.rb L757-L787
|
|
280
|
+
end
|
|
281
|
+
end
|
|
282
|
+
end
|
|
283
|
+
end
|
|
284
|
+
end
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
### Modify S3Operations
|
|
288
|
+
|
|
289
|
+
Replace the `upload` method body with delegation. Remove `upload_file` and `detect_content_type` from S3Operations:
|
|
290
|
+
|
|
291
|
+
```ruby
|
|
292
|
+
def upload(dry_run: false)
|
|
293
|
+
S3Uploader.new(brand, project_id, brand_info: brand_info, brand_path: brand_path, s3_client: @s3_client_override).upload(dry_run: dry_run)
|
|
294
|
+
end
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
### Modify lib/appydave/tools.rb (WU2)
|
|
298
|
+
|
|
299
|
+
Add s3_uploader require immediately before s3_operations:
|
|
300
|
+
```ruby
|
|
301
|
+
require 'appydave/tools/dam/s3_base'
|
|
302
|
+
require 'appydave/tools/dam/s3_uploader' # ADD THIS
|
|
303
|
+
require 'appydave/tools/dam/s3_operations'
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
### Done when WU2 is complete
|
|
307
|
+
- `s3_uploader.rb` created
|
|
308
|
+
- `s3_operations.rb` upload delegates to S3Uploader; upload_file + detect_content_type removed from s3_operations.rb
|
|
309
|
+
- `lib/appydave/tools.rb` has s3_uploader require
|
|
310
|
+
- `RUBYOPT="-W0" bundle exec rspec` → 870 examples, 0 failures
|
|
311
|
+
- `bundle exec rubocop --format clang` → 0 offenses
|
|
312
|
+
- Commit: `kfix "extract S3Uploader from S3Operations; upload delegates to focused class"`
|
|
313
|
+
|
|
314
|
+
---
|
|
315
|
+
|
|
316
|
+
## WU3: Extract S3Downloader
|
|
317
|
+
|
|
318
|
+
**Prerequisite:** WU2 complete.
|
|
319
|
+
**File to create:** `lib/appydave/tools/dam/s3_downloader.rb`
|
|
320
|
+
**File to modify:** `lib/appydave/tools/dam/s3_operations.rb`
|
|
321
|
+
|
|
322
|
+
### What to build
|
|
323
|
+
|
|
324
|
+
```ruby
|
|
325
|
+
# frozen_string_literal: true
|
|
326
|
+
|
|
327
|
+
module Appydave
|
|
328
|
+
module Tools
|
|
329
|
+
module Dam
|
|
330
|
+
# Handles S3 download operations.
|
|
331
|
+
class S3Downloader < S3Base
|
|
332
|
+
def download(dry_run: false)
|
|
333
|
+
# MOVE exactly from s3_operations.rb L188-L257
|
|
334
|
+
end
|
|
335
|
+
|
|
336
|
+
private
|
|
337
|
+
|
|
338
|
+
def download_file(s3_key, local_file, dry_run: false)
|
|
339
|
+
# MOVE exactly from s3_operations.rb L790-L815
|
|
340
|
+
end
|
|
341
|
+
end
|
|
342
|
+
end
|
|
343
|
+
end
|
|
344
|
+
end
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
### Modify S3Operations
|
|
348
|
+
|
|
349
|
+
```ruby
|
|
350
|
+
def download(dry_run: false)
|
|
351
|
+
S3Downloader.new(brand, project_id, brand_info: brand_info, brand_path: brand_path, s3_client: @s3_client_override).download(dry_run: dry_run)
|
|
352
|
+
end
|
|
353
|
+
```
|
|
354
|
+
|
|
355
|
+
Remove `download_file` from S3Operations.
|
|
356
|
+
|
|
357
|
+
### Modify lib/appydave/tools.rb (WU3)
|
|
358
|
+
|
|
359
|
+
Add s3_downloader require:
|
|
360
|
+
```ruby
|
|
361
|
+
require 'appydave/tools/dam/s3_base'
|
|
362
|
+
require 'appydave/tools/dam/s3_uploader'
|
|
363
|
+
require 'appydave/tools/dam/s3_downloader' # ADD THIS
|
|
364
|
+
require 'appydave/tools/dam/s3_operations'
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
### Done when WU3 is complete
|
|
368
|
+
- `s3_downloader.rb` created, delegation in place, `download_file` removed from s3_operations.rb
|
|
369
|
+
- `lib/appydave/tools.rb` has s3_downloader require
|
|
370
|
+
- `RUBYOPT="-W0" bundle exec rspec` → 870 examples, 0 failures
|
|
371
|
+
- `bundle exec rubocop --format clang` → 0 offenses
|
|
372
|
+
- Commit: `kfix "extract S3Downloader from S3Operations; download delegates to focused class"`
|
|
373
|
+
|
|
374
|
+
---
|
|
375
|
+
|
|
376
|
+
## WU4: Extract S3StatusChecker
|
|
377
|
+
|
|
378
|
+
**Prerequisite:** WU3 complete.
|
|
379
|
+
**File to create:** `lib/appydave/tools/dam/s3_status_checker.rb`
|
|
380
|
+
**File to modify:** `lib/appydave/tools/dam/s3_operations.rb`
|
|
381
|
+
|
|
382
|
+
### What to build
|
|
383
|
+
|
|
384
|
+
```ruby
|
|
385
|
+
# frozen_string_literal: true
|
|
386
|
+
|
|
387
|
+
module Appydave
|
|
388
|
+
module Tools
|
|
389
|
+
module Dam
|
|
390
|
+
# Handles S3 status checking, sync status calculation, and timestamp queries.
|
|
391
|
+
# Used directly by project_listing.rb for dam list S3 column.
|
|
392
|
+
class S3StatusChecker < S3Base
|
|
393
|
+
def status
|
|
394
|
+
# MOVE exactly from s3_operations.rb L260-L351
|
|
395
|
+
end
|
|
396
|
+
|
|
397
|
+
def calculate_sync_status
|
|
398
|
+
# MOVE exactly from s3_operations.rb L495-L558
|
|
399
|
+
end
|
|
400
|
+
|
|
401
|
+
def sync_timestamps
|
|
402
|
+
# MOVE exactly from s3_operations.rb L562-L587
|
|
403
|
+
end
|
|
404
|
+
|
|
405
|
+
private
|
|
406
|
+
|
|
407
|
+
def list_local_files(staging_dir)
|
|
408
|
+
# MOVE exactly from s3_operations.rb L890-L900
|
|
409
|
+
end
|
|
410
|
+
end
|
|
411
|
+
end
|
|
412
|
+
end
|
|
413
|
+
end
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
### Modify S3Operations
|
|
417
|
+
|
|
418
|
+
```ruby
|
|
419
|
+
def status
|
|
420
|
+
S3StatusChecker.new(brand, project_id, brand_info: brand_info, brand_path: brand_path, s3_client: @s3_client_override).status
|
|
421
|
+
end
|
|
422
|
+
|
|
423
|
+
def calculate_sync_status
|
|
424
|
+
S3StatusChecker.new(brand, project_id, brand_info: brand_info, brand_path: brand_path, s3_client: @s3_client_override).calculate_sync_status
|
|
425
|
+
end
|
|
426
|
+
|
|
427
|
+
def sync_timestamps
|
|
428
|
+
S3StatusChecker.new(brand, project_id, brand_info: brand_info, brand_path: brand_path, s3_client: @s3_client_override).sync_timestamps
|
|
429
|
+
end
|
|
430
|
+
```
|
|
431
|
+
|
|
432
|
+
Remove `status`, `calculate_sync_status`, `sync_timestamps`, `list_local_files` from S3Operations.
|
|
433
|
+
|
|
434
|
+
**Note:** `project_listing.rb` calls `S3Operations.new(brand_arg, project, brand_info: brand_info).calculate_sync_status` — this still works because S3Operations delegates. No change needed in project_listing.rb.
|
|
435
|
+
|
|
436
|
+
### Modify lib/appydave/tools.rb (WU4)
|
|
437
|
+
|
|
438
|
+
Add s3_status_checker require:
|
|
439
|
+
```ruby
|
|
440
|
+
require 'appydave/tools/dam/s3_base'
|
|
441
|
+
require 'appydave/tools/dam/s3_uploader'
|
|
442
|
+
require 'appydave/tools/dam/s3_downloader'
|
|
443
|
+
require 'appydave/tools/dam/s3_status_checker' # ADD THIS
|
|
444
|
+
require 'appydave/tools/dam/s3_operations'
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
### Done when WU4 is complete
|
|
448
|
+
- `s3_status_checker.rb` created, delegation in place, 4 methods removed from s3_operations.rb
|
|
449
|
+
- `lib/appydave/tools.rb` has s3_status_checker require
|
|
450
|
+
- `RUBYOPT="-W0" bundle exec rspec` → 870 examples, 0 failures
|
|
451
|
+
- `bundle exec rubocop --format clang` → 0 offenses
|
|
452
|
+
- Commit: `kfix "extract S3StatusChecker from S3Operations; status methods delegate to focused class"`
|
|
453
|
+
|
|
454
|
+
---
|
|
455
|
+
|
|
456
|
+
## WU5: Extract S3Archiver + Final Cleanup
|
|
457
|
+
|
|
458
|
+
**Prerequisite:** WU4 complete.
|
|
459
|
+
**File to create:** `lib/appydave/tools/dam/s3_archiver.rb`
|
|
460
|
+
**Files to modify:** `lib/appydave/tools/dam/s3_operations.rb`, `lib/appydave/tools/tools.rb`
|
|
461
|
+
|
|
462
|
+
### What to build
|
|
463
|
+
|
|
464
|
+
```ruby
|
|
465
|
+
# frozen_string_literal: true
|
|
466
|
+
|
|
467
|
+
module Appydave
|
|
468
|
+
module Tools
|
|
469
|
+
module Dam
|
|
470
|
+
# Handles archive to SSD and S3/local cleanup operations.
|
|
471
|
+
class S3Archiver < S3Base
|
|
472
|
+
def cleanup(force: false, dry_run: false)
|
|
473
|
+
# MOVE exactly from s3_operations.rb L354-L390
|
|
474
|
+
end
|
|
475
|
+
|
|
476
|
+
def cleanup_local(force: false, dry_run: false)
|
|
477
|
+
# MOVE exactly from s3_operations.rb L393-L445
|
|
478
|
+
end
|
|
479
|
+
|
|
480
|
+
def archive(force: false, dry_run: false)
|
|
481
|
+
# MOVE exactly from s3_operations.rb L448-L491
|
|
482
|
+
end
|
|
483
|
+
|
|
484
|
+
private
|
|
485
|
+
|
|
486
|
+
def delete_s3_file(s3_key, dry_run: false)
|
|
487
|
+
# MOVE exactly from s3_operations.rb L818-L833
|
|
488
|
+
end
|
|
489
|
+
|
|
490
|
+
def delete_local_file(file_path, dry_run: false)
|
|
491
|
+
# MOVE exactly from s3_operations.rb L836-L847
|
|
492
|
+
end
|
|
493
|
+
|
|
494
|
+
def copy_to_ssd(source_dir, dest_dir, dry_run: false)
|
|
495
|
+
# MOVE exactly from s3_operations.rb L915-L947
|
|
496
|
+
end
|
|
497
|
+
|
|
498
|
+
def copy_with_exclusions(source_dir, dest_dir)
|
|
499
|
+
# MOVE exactly from s3_operations.rb L949-L971
|
|
500
|
+
# Note: calls excluded_path? which is in S3Base — still accessible via inheritance
|
|
501
|
+
end
|
|
502
|
+
|
|
503
|
+
def delete_local_project(project_dir, dry_run: false)
|
|
504
|
+
# MOVE exactly from s3_operations.rb L990-L1013
|
|
505
|
+
end
|
|
506
|
+
|
|
507
|
+
def calculate_directory_size(dir_path)
|
|
508
|
+
# MOVE exactly from s3_operations.rb L1015-L1021
|
|
509
|
+
end
|
|
510
|
+
end
|
|
511
|
+
end
|
|
512
|
+
end
|
|
513
|
+
end
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
### Modify S3Operations (final form ~70 lines)
|
|
517
|
+
|
|
518
|
+
After removing all methods, S3Operations should look like:
|
|
519
|
+
|
|
520
|
+
```ruby
|
|
521
|
+
# frozen_string_literal: true
|
|
522
|
+
|
|
523
|
+
module Appydave
|
|
524
|
+
module Tools
|
|
525
|
+
module Dam
|
|
526
|
+
# Thin facade for S3 operations.
|
|
527
|
+
# Inherits shared helpers from S3Base for backward-compatible spec access via send().
|
|
528
|
+
# Delegates all operations to focused sub-classes.
|
|
529
|
+
class S3Operations < S3Base
|
|
530
|
+
def upload(dry_run: false)
|
|
531
|
+
S3Uploader.new(brand, project_id, **delegated_opts).upload(dry_run: dry_run)
|
|
532
|
+
end
|
|
533
|
+
|
|
534
|
+
def download(dry_run: false)
|
|
535
|
+
S3Downloader.new(brand, project_id, **delegated_opts).download(dry_run: dry_run)
|
|
536
|
+
end
|
|
537
|
+
|
|
538
|
+
def status
|
|
539
|
+
S3StatusChecker.new(brand, project_id, **delegated_opts).status
|
|
540
|
+
end
|
|
541
|
+
|
|
542
|
+
def calculate_sync_status
|
|
543
|
+
S3StatusChecker.new(brand, project_id, **delegated_opts).calculate_sync_status
|
|
544
|
+
end
|
|
545
|
+
|
|
546
|
+
def sync_timestamps
|
|
547
|
+
S3StatusChecker.new(brand, project_id, **delegated_opts).sync_timestamps
|
|
548
|
+
end
|
|
549
|
+
|
|
550
|
+
def cleanup(force: false, dry_run: false)
|
|
551
|
+
S3Archiver.new(brand, project_id, **delegated_opts).cleanup(force: force, dry_run: dry_run)
|
|
552
|
+
end
|
|
553
|
+
|
|
554
|
+
def cleanup_local(force: false, dry_run: false)
|
|
555
|
+
S3Archiver.new(brand, project_id, **delegated_opts).cleanup_local(force: force, dry_run: dry_run)
|
|
556
|
+
end
|
|
557
|
+
|
|
558
|
+
def archive(force: false, dry_run: false)
|
|
559
|
+
S3Archiver.new(brand, project_id, **delegated_opts).archive(force: force, dry_run: dry_run)
|
|
560
|
+
end
|
|
561
|
+
|
|
562
|
+
private
|
|
563
|
+
|
|
564
|
+
def delegated_opts
|
|
565
|
+
{ brand_info: brand_info, brand_path: brand_path, s3_client: @s3_client_override }
|
|
566
|
+
end
|
|
567
|
+
end
|
|
568
|
+
end
|
|
569
|
+
end
|
|
570
|
+
end
|
|
571
|
+
```
|
|
572
|
+
|
|
573
|
+
### Modify lib/appydave/tools.rb (WU5)
|
|
574
|
+
|
|
575
|
+
Add s3_archiver require (WU1-4 already added the others):
|
|
576
|
+
```ruby
|
|
577
|
+
require 'appydave/tools/dam/s3_base'
|
|
578
|
+
require 'appydave/tools/dam/s3_uploader'
|
|
579
|
+
require 'appydave/tools/dam/s3_downloader'
|
|
580
|
+
require 'appydave/tools/dam/s3_status_checker'
|
|
581
|
+
require 'appydave/tools/dam/s3_archiver' # ADD THIS
|
|
582
|
+
require 'appydave/tools/dam/s3_operations' # existing
|
|
583
|
+
```
|
|
584
|
+
|
|
585
|
+
### Done when WU5 is complete
|
|
586
|
+
- `s3_archiver.rb` created
|
|
587
|
+
- `s3_operations.rb` is now ~70 lines (thin facade)
|
|
588
|
+
- `lib/appydave/tools.rb` has 5 new require lines before `s3_operations`
|
|
589
|
+
- `RUBYOPT="-W0" bundle exec rspec` → 870 examples, 0 failures
|
|
590
|
+
- `bundle exec rubocop --format clang` → 0 offenses
|
|
591
|
+
- Commit: `kfix "extract S3Archiver; S3Operations is now a thin delegation facade"`
|
|
592
|
+
|
|
593
|
+
---
|
|
594
|
+
|
|
595
|
+
## Success Criteria (Every Work Unit)
|
|
596
|
+
|
|
597
|
+
- [ ] `RUBYOPT="-W0" bundle exec rspec` → 870 examples, 0 failures
|
|
598
|
+
- [ ] `bundle exec rubocop --format clang` → 0 offenses
|
|
599
|
+
- [ ] Coverage ≥ 86.47%
|
|
600
|
+
- [ ] Working tree clean before calling kfix (git status check)
|
|
601
|
+
|
|
602
|
+
---
|
|
603
|
+
|
|
604
|
+
## Reference Patterns
|
|
605
|
+
|
|
606
|
+
### S3Base inheritance pattern (how sub-classes access shared state)
|
|
607
|
+
|
|
608
|
+
Methods in S3Uploader, S3Downloader, etc. can call any method defined in S3Base via `self`:
|
|
609
|
+
|
|
610
|
+
```ruby
|
|
611
|
+
class S3Uploader < S3Base
|
|
612
|
+
def upload(dry_run: false)
|
|
613
|
+
staging_dir = File.join(project_directory_path, 's3-staging') # from S3Base
|
|
614
|
+
files = list_s3_files # from S3Base
|
|
615
|
+
s3_key = build_s3_key(relative_path) # from S3Base
|
|
616
|
+
s3_client.put_object(...) # from S3Base
|
|
617
|
+
file_size_human(bytes) # from S3Base
|
|
618
|
+
end
|
|
619
|
+
end
|
|
620
|
+
```
|
|
621
|
+
|
|
622
|
+
### Delegation opts helper (in S3Operations)
|
|
623
|
+
|
|
624
|
+
```ruby
|
|
625
|
+
private
|
|
626
|
+
|
|
627
|
+
def delegated_opts
|
|
628
|
+
{ brand_info: brand_info, brand_path: brand_path, s3_client: @s3_client_override }
|
|
629
|
+
end
|
|
630
|
+
```
|
|
631
|
+
|
|
632
|
+
Use `**delegated_opts` in every delegation call to avoid repetition.
|
|
633
|
+
|
|
634
|
+
### instance_double — Always Full Constant
|
|
635
|
+
|
|
636
|
+
```ruby
|
|
637
|
+
instance_double(Appydave::Tools::Configuration::Models::BrandsConfig) # ✅
|
|
638
|
+
instance_double('BrandsConfig') # ❌ fails CI
|
|
639
|
+
```
|
|
640
|
+
|
|
641
|
+
### Typed Exceptions
|
|
642
|
+
|
|
643
|
+
```ruby
|
|
644
|
+
raise Appydave::Tools::Dam::BrandNotFoundError.new(brand, available, suggestions)
|
|
645
|
+
raise Appydave::Tools::Dam::ProjectNotFoundError, 'message'
|
|
646
|
+
raise Appydave::Tools::Dam::UsageError, 'message'
|
|
647
|
+
```
|
|
648
|
+
|
|
649
|
+
---
|
|
650
|
+
|
|
651
|
+
## Anti-Patterns to Avoid
|
|
652
|
+
|
|
653
|
+
- ❌ Don't add `require` statements inside individual `lib/` files — only in `lib/appydave/tools.rb`
|
|
654
|
+
- ❌ Don't change any method signatures — exact copies only (no refactoring during extraction)
|
|
655
|
+
- ❌ Don't remove `rubocop:disable` comments unless you're fixing the actual offense
|
|
656
|
+
- ❌ `exit 1` in library code — use typed exceptions (already done in prior campaigns)
|
|
657
|
+
- ❌ `instance_double('StringForm')` — fails CI on Ubuntu
|
|
658
|
+
- ❌ Multiple `before` blocks in same context — merge them (RSpec/ScatteredSetup)
|
|
659
|
+
- ❌ `$?` for subprocess status — use `$CHILD_STATUS`
|
|
660
|
+
- ❌ Don't touch spec files during WU1-WU5 — all existing specs test through S3Operations facade
|
|
661
|
+
- ❌ Don't try to parallelize — wave size is 1, these WUs MUST be sequential
|
|
662
|
+
|
|
663
|
+
---
|
|
664
|
+
|
|
665
|
+
## Learnings
|
|
666
|
+
|
|
667
|
+
### From batch-a-features (2026-03-20)
|
|
668
|
+
|
|
669
|
+
- **kfix runs `git add .` internally.** Clean the working tree before calling kfix. Check `git status` first.
|
|
670
|
+
- **`instance_double` string form fails CI on Ubuntu.** Always use full constant.
|
|
671
|
+
- **`warn` not `$stderr.puts`** — rubocop Style/StderrPuts cop
|
|
672
|
+
- **`not_to raise_error` is a weak assertion.** Prefer field-value assertions.
|
|
673
|
+
|
|
674
|
+
### From env-dead-code-cleanup (2026-03-20)
|
|
675
|
+
|
|
676
|
+
- **`exit 1` in library code → use typed exceptions.** VatCLI rescue blocks catch StandardError.
|
|
677
|
+
- **Config.brands needs separate mock** from shared filesystem context.
|
|
678
|
+
|
|
679
|
+
### From library-boundary-cleanup (2026-03-20)
|
|
680
|
+
|
|
681
|
+
- **S3ScanCommand#scan_all rescues per-brand** — per-brand failures are isolated.
|
|
682
|
+
- **`not_to raise_error` is a weak assertion.** Prefer field-value or method-spy assertions.
|
|
683
|
+
|
|
684
|
+
### Key s3_operations.rb note: `rubocop:disable Metrics/BlockLength`
|
|
685
|
+
|
|
686
|
+
The `upload` and `download` methods have `# rubocop:disable Metrics/BlockLength` / `# rubocop:enable Metrics/BlockLength` comments around their `each` blocks. When you MOVE these methods to S3Uploader/S3Downloader, carry those comments along. Rubocop will still flag them without the disable.
|