belt 0.6.1 → 0.6.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 66c35b71a47f74ba7805bce24b37eb02fa6ded7246872df30a951a35be22d21f
4
- data.tar.gz: 5ddc3bb756fdf782da5074abb72bd0c7275c6a90f4ba98c81963b92cb9cd6bb9
3
+ metadata.gz: 8a728d2756664cff1204c7926b8db4cbe7d327e1aacf6d544b9d39db2e2e258c
4
+ data.tar.gz: 5b4eb1d119a8343abbdfe5fc329f167546bc963ecb7ef415d0c54b5c049cf084
5
5
  SHA512:
6
- metadata.gz: 44320ad96e29e29a6bb9b88d2de94822f982dad01e92bc639204c38d2bb7a306463b7b570ac8c35cbf13cc00a223a55442b52171ac64f3357de98a9662a5e3e9
7
- data.tar.gz: 8e176982bec58892623685d2b3bfd02cde1ab22d9fcac2c6eb1bfe58f5ae27f02d28b708cced560b803379b36bcd2bea5463fce749bde4ebd9b169cbfff878ba
6
+ metadata.gz: bad0ef5562fe52c213e58b8ae0451e8fe48ae1f1b347f2cb6662e891e9b5893b59210002d5fedf0c3e3b03fcb72ed24ab05c5485c2156e034e1207a5c0f54316
7
+ data.tar.gz: 2a92ccf0e821c055f0cbf661c5d5446ba98fd7bd45fb25b7fda6a774027f29aba07411fc962afb0b3f150c6685a768790c2981ba8548f9549b3828c969ed88ec
data/CHANGELOG.md CHANGED
@@ -1,5 +1,31 @@
1
1
  # Changelog
2
2
 
3
+ ## Unreleased
4
+
5
+ ### Added
6
+
7
+ - **Data migrations.** `db/migrate/<timestamp>_<name>.rb` files define `Belt::Migration`
8
+ subclasses whose `up` runs in the booted app context (same as `belt console`).
9
+ `belt deploy <env>` runs pending ones after terraform apply and the nested-env data copy
10
+ (also after `--rebuild`; opt out with `--skip-migrations`). Each version runs once per
11
+ environment, nested envs included, recorded in SSM at
12
+ `/belt/<app>/<env>/schema_migrations/<version>`. A failed migration stays pending, the
13
+ deploy carries on and exits non-zero. New `belt db:migrate [env]` and
14
+ `belt db:migrate:status [env]` run or check them by hand. See `belt explain migrations`.
15
+
16
+ ## 0.6.2
17
+
18
+ ### Fixed
19
+
20
+ - **First deploy of a new env with a `domain` no longer fails at plan.** The generated
21
+ module keyed `aws_route53_record.cert_validation` by the ACM validation CNAME name,
22
+ which is unknown until the cert exists, so `belt plan`/`belt deploy` on a brand-new
23
+ standalone env failed with `Invalid for_each argument` and needed a `-target` apply
24
+ of the cert first. The record is now `aws_route53_record.acm_validation`, keyed by
25
+ the env's domain (known at plan). A `removed` block forgets old `cert_validation`
26
+ instances without deleting the live CNAME. The module now needs Terraform >= 1.7.
27
+ Existing apps: see [UPGRADING.md](UPGRADING.md#acm-validation-record-062).
28
+
3
29
  ## 0.6.1
4
30
 
5
31
  ### Fixed
data/README.md CHANGED
@@ -731,6 +731,29 @@ puts "Created post: #{post.id}"
731
731
 
732
732
  Refuses to run against an environment that already has data (pass `--force` to override). `belt new` scaffolds a starter `config/seeds.rb`. See `belt explain data_seeding` for details.
733
733
 
734
+ ### `belt db:migrate` — data migrations
735
+
736
+ For changes to rows that already exist (backfilling an attribute, populating a new GSI for old records), add a migration to `db/migrate/`:
737
+
738
+ ```ruby
739
+ # db/migrate/20261030120000_index_usernames.rb
740
+ class IndexUsernames < Belt::Migration
741
+ def up
742
+ PoshNosh::MatchStore.new.index_usernames!
743
+ end
744
+ end
745
+ ```
746
+
747
+ `belt deploy <env>` runs pending migrations after terraform apply, once per environment (nested envs included). Applied versions are recorded in SSM under `/belt/<app>/<env>/schema_migrations/`.
748
+
749
+ ```bash
750
+ belt db:migrate dev01 # run pending migrations by hand
751
+ belt db:migrate:status prod # which have run
752
+ belt deploy dev --skip-migrations
753
+ ```
754
+
755
+ Migrations must be idempotent and batch-safe. See `belt explain migrations`.
756
+
734
757
  ## Testing
735
758
 
736
759
  Belt ships an in-process **end-to-end harness** that drives the real router — no AWS, no HTTP server, no browser. It's the tier between controller unit tests (dispatch an action directly, never touch routing) and full cloud e2e (needs a deployed stack).
data/SKILL.md CHANGED
@@ -8,7 +8,7 @@ description: >-
8
8
  Belt routing DSL, writing BeltController actions, modeling DynamoDB data
9
9
  with ActiveItem, wiring Cognito auth, configuring the Lambda handler,
10
10
  running `belt` CLI commands (new, generate, deploy, console, routes, logs,
11
- server, setup, plugin, explain, db:copy/db:seed), Terraform via belt,
11
+ server, setup, plugin, explain, db:copy/db:seed/db:migrate), Terraform via belt,
12
12
  backups, observability, or authoring Belt plugins. Triggers on "belt new",
13
13
  "belt deploy", "belt generate", "belt routes", "BeltController", "ActiveItem",
14
14
  "Belt::LambdaHandler", "belt console", "belt server", "conveyor-belt". Do NOT
@@ -0,0 +1,208 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'optparse'
4
+ require 'stringio'
5
+ require_relative 'app_detection'
6
+ require_relative 'environment_config'
7
+ require_relative 'migration_store'
8
+ require_relative '../inflector'
9
+ require_relative '../migration'
10
+
11
+ module Belt
12
+ module CLI
13
+ # `belt db:migrate` / `belt db:migrate:status` — data migrations.
14
+ #
15
+ # Each `db/migrate/<timestamp>_<name>.rb` defines a `Belt::Migration`
16
+ # subclass named after the file (`20261030120000_index_usernames.rb` →
17
+ # `IndexUsernames`). Pending ones run oldest-first in the same booted app
18
+ # context `belt console` uses, and each is recorded in SSM (see
19
+ # MigrationStore) only after its `up` returns, so a failure is retried on
20
+ # the next run. `belt deploy` runs this after terraform apply.
21
+ class DbMigrateCommand
22
+ include AppDetection
23
+
24
+ MIGRATIONS_DIR = File.join('db', 'migrate')
25
+ FILENAME_PATTERN = /\A(\d+)_([a-z0-9_]+)\.rb\z/
26
+
27
+ MigrationFile = Struct.new(:version, :name, :path) do
28
+ def class_name
29
+ Belt::Inflector.camelize(name)
30
+ end
31
+
32
+ def label
33
+ name.tr('_', ' ').capitalize
34
+ end
35
+ end
36
+
37
+ def self.run(args, status: false)
38
+ new(args, status: status).run
39
+ end
40
+
41
+ def self.migrations?(root = Dir.pwd)
42
+ Dir.glob(File.join(root, MIGRATIONS_DIR, '*.rb')).any?
43
+ end
44
+
45
+ def initialize(args, status: false)
46
+ @options = { status: status, yes: false }
47
+ parse_options(args)
48
+ end
49
+
50
+ def run
51
+ ENV['BUNDLE_GEMFILE'] ||= File.join(Belt.root, 'Gemfile')
52
+ unless File.exist?(ENV['BUNDLE_GEMFILE'])
53
+ abort "Error: No Gemfile found at #{ENV['BUNDLE_GEMFILE']}. Are you in a Belt project?"
54
+ end
55
+
56
+ @environment = @env_arg || ENV.fetch('BELT_ENV', nil) || 'dev'
57
+ ENV['ENVIRONMENT'] = @environment
58
+ apply_env_config!
59
+
60
+ @store = MigrationStore.new(app_name: detect_app_name, env: @environment)
61
+ @options[:status] ? print_status : migrate
62
+ rescue MigrationStore::Error => e
63
+ abort "Error: #{e.message}"
64
+ end
65
+
66
+ private
67
+
68
+ def parse_options(args)
69
+ OptionParser.new do |opts|
70
+ opts.banner = if @options[:status]
71
+ 'Usage: belt db:migrate:status [environment]'
72
+ else
73
+ 'Usage: belt db:migrate [environment] [options]'
74
+ end
75
+
76
+ unless @options[:status]
77
+ opts.on('-y', '--yes', 'Skip the production confirmation prompt') { @options[:yes] = true }
78
+ end
79
+
80
+ opts.on('-h', '--help', 'Show this help') do
81
+ puts opts
82
+ exit
83
+ end
84
+ end.parse!(args)
85
+
86
+ @env_arg = args.shift
87
+ end
88
+
89
+ def apply_env_config!
90
+ env_config = EnvironmentConfig.load(@environment)
91
+ env_config.apply!
92
+ puts " 🔑 Using AWS profile: #{env_config.aws_profile}" if env_config.aws_profile?
93
+ end
94
+
95
+ def migrate
96
+ pending = pending_migrations
97
+ if pending.empty?
98
+ puts "belt → no pending migrations for #{@environment}"
99
+ return
100
+ end
101
+
102
+ production_guard!
103
+ boot_app
104
+
105
+ puts "belt → running #{pending.size} migration(s) against #{@environment}"
106
+ pending.each { |file| run_migration(file) }
107
+ puts "✅ Migrations complete (#{@environment})"
108
+ end
109
+
110
+ def run_migration(file)
111
+ puts " == #{file.version} #{file.label}"
112
+ started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
113
+ instantiate(file).up
114
+ @store.record!(file.version, file.name)
115
+ elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started
116
+ puts format(' == %<version>s migrated (%<elapsed>.1fs)', version: file.version, elapsed: elapsed)
117
+ rescue MigrationStore::Error
118
+ raise
119
+ rescue StandardError, ScriptError => e
120
+ abort "\n❌ Migration #{file.version} (#{file.name}) failed: #{e.class}: #{e.message}\n" \
121
+ "#{Array(e.backtrace).first(5).join("\n")}\n\n" \
122
+ 'Later migrations were not run. Fix it and re-run ' \
123
+ "`belt db:migrate #{@environment}` (migrations must be safe to re-run)."
124
+ end
125
+
126
+ def instantiate(file)
127
+ load File.expand_path(file.path)
128
+ klass = Object.const_get(file.class_name) if Object.const_defined?(file.class_name)
129
+ unless klass.is_a?(Class) && klass < Belt::Migration
130
+ raise NameError, "#{file.path} must define `class #{file.class_name} < Belt::Migration`"
131
+ end
132
+
133
+ klass.new(version: file.version, name: file.name)
134
+ end
135
+
136
+ def print_status
137
+ applied = @store.applied_versions
138
+ files = migration_files
139
+
140
+ puts "\nenvironment: #{@environment}\n\n"
141
+ puts ' Status Migration ID Migration Name'
142
+ puts '-' * 50
143
+ files.each do |file|
144
+ puts status_line(applied.include?(file.version) ? 'up' : 'down', file.version, file.label)
145
+ end
146
+ (applied - files.map(&:version)).sort.each do |version|
147
+ puts status_line('up', version, '** NO FILE **')
148
+ end
149
+ puts ' (no migrations in db/migrate)' if files.empty? && applied.empty?
150
+ end
151
+
152
+ def status_line(state, version, label)
153
+ format(' %<state>-6s %<version>-14s %<label>s', state: state, version: version, label: label)
154
+ end
155
+
156
+ def pending_migrations
157
+ applied = @store.applied_versions
158
+ migration_files.reject { |file| applied.include?(file.version) }
159
+ end
160
+
161
+ def migration_files
162
+ Dir.glob(File.join(MIGRATIONS_DIR, '*.rb')).map do |path|
163
+ match = File.basename(path).match(FILENAME_PATTERN)
164
+ abort "Error: #{path} must be named <timestamp>_<snake_case_name>.rb" unless match
165
+
166
+ MigrationFile.new(match[1], match[2], path)
167
+ end.sort_by(&:version)
168
+ end
169
+
170
+ def production_guard!
171
+ return unless @environment == 'prod' && !@options[:yes]
172
+
173
+ $stdout.write "\n⚠️ WARNING: You are about to migrate PRODUCTION data!\nType 'yes' to continue: "
174
+ response = $stdin.gets&.chomp
175
+ abort "\n❌ Cancelled." unless response&.downcase == 'yes'
176
+ end
177
+
178
+ def boot_app
179
+ suppress_warnings { require 'bundler/setup' }
180
+
181
+ environment_file = File.join(Belt.root, 'lambda', 'config', 'environment.rb')
182
+ if File.exist?(environment_file)
183
+ load environment_file
184
+ else
185
+ require 'belt'
186
+ load_dir('lib')
187
+ load_dir('models')
188
+ end
189
+ end
190
+
191
+ def load_dir(subdir)
192
+ dir = File.join(Belt.root, 'lambda', subdir)
193
+ Dir.glob(File.join(dir, '**', '*.rb')).each { |f| require f } if Dir.exist?(dir)
194
+ end
195
+
196
+ def suppress_warnings
197
+ original_verbose = $VERBOSE
198
+ $VERBOSE = nil
199
+ original_stderr = $stderr
200
+ $stderr = StringIO.new
201
+ yield
202
+ ensure
203
+ $stderr = original_stderr
204
+ $VERBOSE = original_verbose
205
+ end
206
+ end
207
+ end
208
+ end
@@ -16,11 +16,13 @@ require_relative 'cognito_sharer'
16
16
  require_relative 'dynamo_copier'
17
17
  require_relative 'dns_command'
18
18
  require_relative 'apex_dns_sync'
19
+ require_relative 'db_migrate_command'
19
20
 
20
21
  module Belt
21
22
  module CLI
22
23
  class DeployCommand
23
24
  DEFAULT_DOCKER_BUILD_IMAGE = 'public.ecr.aws/sam/build-ruby3.4:latest-x86_64'
25
+ BELT_EXE = File.expand_path('../../../exe/belt', __dir__)
24
26
  def self.run(args)
25
27
  # Handle `belt deploy frontend <env>` as before
26
28
  if args.first == 'frontend'
@@ -33,6 +35,7 @@ module Belt
33
35
  rebuild = false
34
36
  skip_backup = false
35
37
  backup_only = false
38
+ skip_migrations = false
36
39
  filtered_args = []
37
40
 
38
41
  args.each do |arg|
@@ -45,6 +48,8 @@ module Belt
45
48
  skip_backup = true
46
49
  when '--backup-only'
47
50
  backup_only = true
51
+ when '--skip-migrations'
52
+ skip_migrations = true
48
53
  when '-h', '--help'
49
54
  puts help_text
50
55
  exit 0
@@ -79,12 +84,13 @@ module Belt
79
84
  end
80
85
  end
81
86
 
87
+ options = { auto_approve: auto_approve, skip_migrations: skip_migrations, extra_args: filtered_args }
82
88
  if backup_only
83
- new(env, auto_approve: auto_approve, skip_backup: false, extra_args: filtered_args).run_backup_only
89
+ new(env, **options, skip_backup: false).run_backup_only
84
90
  elsif rebuild
85
- new(env, auto_approve: auto_approve, skip_backup: skip_backup, extra_args: filtered_args).run_rebuild
91
+ new(env, **options, skip_backup: skip_backup).run_rebuild
86
92
  else
87
- new(env, auto_approve: auto_approve, skip_backup: skip_backup, extra_args: filtered_args).run
93
+ new(env, **options, skip_backup: skip_backup).run
88
94
  end
89
95
  end
90
96
 
@@ -105,6 +111,7 @@ module Belt
105
111
  7. Prompt for confirmation (unless --auto)
106
112
  8. terraform apply (deploy changes)
107
113
  9. Nested envs: share parent Cognito + copy DynamoDB if empty
114
+ 10. Run pending data migrations in db/migrate (see `belt db:migrate`)
108
115
 
109
116
  Options:
110
117
  --auto, --yes, -y Skip confirmation prompt (auto-approve)
@@ -114,6 +121,7 @@ module Belt
114
121
  --skip-backup Skip pre-deploy backup phase
115
122
  --no-backup Alias for --skip-backup
116
123
  --backup-only Run backups only, do not deploy
124
+ --skip-migrations Don't run pending db/migrate data migrations
117
125
  -h, --help Show this help
118
126
 
119
127
  Environment:
@@ -144,10 +152,11 @@ module Belt
144
152
  HELP
145
153
  end
146
154
 
147
- def initialize(env, auto_approve: false, skip_backup: false, extra_args: [])
155
+ def initialize(env, auto_approve: false, skip_backup: false, skip_migrations: false, extra_args: [])
148
156
  @env = env
149
157
  @auto_approve = auto_approve
150
158
  @skip_backup = skip_backup
159
+ @skip_migrations = skip_migrations
151
160
  @extra_args = extra_args
152
161
  @infra_dir = TerraformCommand.find_infrastructure_dir
153
162
  @project_root = find_project_root
@@ -188,9 +197,12 @@ module Belt
188
197
 
189
198
  run_nested_env_hooks
190
199
 
200
+ migrations_ok = run_migrations
201
+
191
202
  deploy_frontend_if_exists
192
203
 
193
204
  puts "\n Run `belt server` to view your app locally (auto-connects to the deployed API)."
205
+ exit 1 unless migrations_ok
194
206
  end
195
207
 
196
208
  def run_backup_only
@@ -232,6 +244,7 @@ module Belt
232
244
  ensure_lockfile_consistent!
233
245
  generate_routes(lambda_dir)
234
246
  build_and_deploy(lambda_dir, function_name)
247
+ exit 1 unless run_migrations
235
248
  end
236
249
 
237
250
  private
@@ -901,6 +914,22 @@ module Belt
901
914
  [raw, sanitized].uniq
902
915
  end
903
916
 
917
+ # Runs pending db/migrate data migrations in a child `belt db:migrate`
918
+ # so the app's code is booted in its own process, not the deployer's.
919
+ # Runs after the nested-env DynamoDB copy so copied rows get migrated
920
+ # too. Returns false on failure: the infra is already live, so the rest
921
+ # of the deploy continues and the exit status reports the failure.
922
+ def run_migrations # rubocop:disable Naming/PredicateMethod
923
+ return true if @skip_migrations || !DbMigrateCommand.migrations?(@project_root)
924
+
925
+ puts "\n━━━ data migrations ━━━"
926
+ return true if system(Gem.ruby, BELT_EXE, 'db:migrate', @env, '--yes', chdir: @project_root)
927
+
928
+ puts "\n ⚠ Data migrations failed for #{@env}. The deploy itself succeeded."
929
+ puts " Fix the migration and run `belt db:migrate #{@env}` to retry."
930
+ false
931
+ end
932
+
904
933
  def deploy_frontend_if_exists
905
934
  require_relative 'frontend_registry'
906
935
  require_relative 'frontend_deploy_command'
@@ -52,6 +52,10 @@ module Belt
52
52
  'db:copy' => 'data_seeding',
53
53
  'copy' => 'data_seeding',
54
54
  'db_copy' => 'data_seeding',
55
+ 'migration' => 'migrations',
56
+ 'migrate' => 'migrations',
57
+ 'db:migrate' => 'migrations',
58
+ 'data_migrations' => 'migrations',
55
59
  'e2e' => 'testing',
56
60
  'test' => 'testing',
57
61
  'tests' => 'testing',
@@ -120,6 +124,7 @@ module Belt
120
124
  irb, repl → console
121
125
  seeds, seed, db:seed → data_seeding
122
126
  db:copy, copy → data_seeding
127
+ migrate, db:migrate → migrations
123
128
  e2e, test, harness → testing
124
129
 
125
130
  Examples:
@@ -0,0 +1,54 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'json'
4
+ require 'open3'
5
+ require 'time'
6
+
7
+ module Belt
8
+ module CLI
9
+ # Records which data migrations have run against an environment, as one
10
+ # SSM parameter per version:
11
+ #
12
+ # /belt/<app>/<env>/schema_migrations/<version>
13
+ #
14
+ # SSM rather than a DynamoDB table so the record isn't part of the
15
+ # `<app>-<env>-*` tables that `belt db:copy` and the nested-env copy move
16
+ # between environments — each stack keeps its own history.
17
+ class MigrationStore
18
+ class Error < StandardError; end
19
+
20
+ def initialize(app_name:, env:)
21
+ @app_name = app_name
22
+ @env = env
23
+ end
24
+
25
+ def path
26
+ "/belt/#{@app_name}/#{@env}/schema_migrations"
27
+ end
28
+
29
+ # Versions already applied to this environment, as strings.
30
+ def applied_versions
31
+ data = aws_json('ssm', 'get-parameters-by-path', '--path', path, '--output', 'json')
32
+ Array(data['Parameters']).map { |param| File.basename(param['Name']) }
33
+ end
34
+
35
+ def record!(version, name)
36
+ value = JSON.generate('name' => name, 'applied_at' => Time.now.utc.iso8601)
37
+ aws_json('ssm', 'put-parameter', '--name', "#{path}/#{version}", '--type', 'String',
38
+ '--value', value, '--overwrite', '--output', 'json')
39
+ end
40
+
41
+ private
42
+
43
+ def aws_json(*args)
44
+ command = "aws #{args.first(2).join(' ')}"
45
+ output, error, status = Open3.capture3('aws', *args)
46
+ raise Error, "#{command} failed: #{error.strip}" unless status.success?
47
+
48
+ output.strip.empty? ? {} : JSON.parse(output)
49
+ rescue JSON::ParserError => e
50
+ raise Error, "#{command} returned invalid JSON: #{e.message}"
51
+ end
52
+ end
53
+ end
54
+ end
data/lib/belt/cli.rb CHANGED
@@ -26,6 +26,7 @@ require_relative 'cli/tasks_command'
26
26
  require_relative 'cli/console_command'
27
27
  require_relative 'cli/db_copy_command'
28
28
  require_relative 'cli/db_seed_command'
29
+ require_relative 'cli/db_migrate_command'
29
30
  require_relative 'cli/logs_command'
30
31
  require_relative 'cli/doctor_command'
31
32
  require_relative 'cli/plugin_command'
@@ -43,6 +44,8 @@ module Belt
43
44
  %w[console c] => Belt::CLI::ConsoleCommand,
44
45
  'db:copy' => Belt::CLI::DbCopyCommand,
45
46
  'db:seed' => Belt::CLI::DbSeedCommand,
47
+ 'db:migrate' => Belt::CLI::DbMigrateCommand,
48
+ 'db:migrate:status' => ->(args) { Belt::CLI::DbMigrateCommand.run(args, status: true) },
46
49
  'logs' => Belt::CLI::LogsCommand,
47
50
  %w[tasks --tasks -T] => Belt::CLI::TasksCommand,
48
51
  'setup' => Belt::CLI::SetupCommand,
@@ -131,6 +134,8 @@ module Belt
131
134
  c Alias for console
132
135
  db:copy <from-env> <to-env> [--force] Copy DynamoDB data between environments
133
136
  db:seed [environment] [--force] Run config/seeds.rb against an environment
137
+ db:migrate [environment] [--yes] Run pending db/migrate data migrations
138
+ db:migrate:status [environment] Show which data migrations have run
134
139
  logs [lambda] [-f] [-s 5m] [-e env] View Lambda function logs
135
140
  tasks [-g PATTERN] [-a] List available rake tasks
136
141
  -T [-g PATTERN] [-a] Alias for tasks
@@ -180,6 +185,8 @@ module Belt
180
185
  belt db:copy prod dev --force # overwrite dev tables even if non-empty
181
186
  belt db:seed # run config/seeds.rb against dev (or BELT_ENV)
182
187
  belt db:seed dev01
188
+ belt db:migrate prod # run pending data migrations against prod
189
+ belt db:migrate:status dev01
183
190
  USAGE
184
191
  end
185
192
 
@@ -126,4 +126,5 @@ file, no generator required.
126
126
 
127
127
  - `belt explain backups` — recovery-point snapshots, not data seeding
128
128
  - `belt explain console` — the app-booting mechanism `db:seed` reuses
129
+ - `belt explain migrations` — changing rows already in a deployed env
129
130
  - `belt explain deployment` — nested environments and the parent → child copy hook
@@ -11,6 +11,7 @@ belt deploy <env> # init → plan → apply (interactive)
11
11
  belt deploy prod --auto # skip confirmation prompt
12
12
  belt deploy prod --skip-backup # skip pre-deploy backup
13
13
  belt deploy --backup-only # create recovery point without deploying
14
+ belt deploy dev --skip-migrations # don't run pending db/migrate migrations
14
15
  ```
15
16
 
16
17
  ## First-Time Setup
@@ -147,6 +148,14 @@ end
147
148
  Backups run automatically before each deploy (DynamoDB snapshots, Cognito
148
149
  exports, S3 syncs). See `belt explain backups` for full documentation.
149
150
 
151
+ ## Data Migrations
152
+
153
+ After terraform apply (and the nested-env data copy), `belt deploy` runs any
154
+ pending `db/migrate/<timestamp>_<name>.rb` data migrations against the env,
155
+ each exactly once per environment. A failing migration doesn't roll back the
156
+ deploy; it's reported and the command exits non-zero. Run them by hand with
157
+ `belt db:migrate <env>`. See `belt explain migrations`.
158
+
150
159
  ## Environment Variables
151
160
 
152
161
  Set `BELT_ENV` to avoid typing the environment every time:
@@ -176,4 +185,5 @@ deploys every configured frontend after terraform apply.
176
185
 
177
186
  - `belt explain routing` — how routes map to infrastructure
178
187
  - `belt explain backups` — pre-deploy backup configuration
188
+ - `belt explain migrations` — data migrations run after deploy
179
189
  - `belt doctor` — check system dependencies before deploying
@@ -0,0 +1,104 @@
1
+ # Data Migrations
2
+
3
+ `belt setup tables` handles the *shape* of a table (keys, GSIs). Data
4
+ migrations handle the *rows already in it*: backfilling a new attribute, or
5
+ populating a new index for records written before it existed.
6
+
7
+ They live in `db/migrate/`, run automatically after `belt deploy <env>`, and
8
+ each one runs **exactly once per environment** — prod, dev and every nested
9
+ (PR-preview) env keep their own history.
10
+
11
+ ## Writing a migration
12
+
13
+ One file per migration, named `<timestamp>_<snake_case_name>.rb`, defining a
14
+ `Belt::Migration` subclass named after the file:
15
+
16
+ ```ruby
17
+ # db/migrate/20261030120000_index_usernames.rb
18
+ class IndexUsernames < Belt::Migration
19
+ def up
20
+ PoshNosh::MatchStore.new.index_usernames!
21
+ say 'usernames indexed'
22
+ end
23
+ end
24
+ ```
25
+
26
+ `up` runs in the same booted app context as `belt console`
27
+ (`lambda/config/environment.rb`, or `lambda/lib` + `lambda/models`), with
28
+ `ENVIRONMENT` set to the target env, so models talk to that env's tables.
29
+ `say` prints an indented progress line.
30
+
31
+ Use `date -u +%Y%m%d%H%M%S` for the timestamp. Migrations run in timestamp
32
+ order.
33
+
34
+ ## Running them
35
+
36
+ ```bash
37
+ belt deploy dev # runs pending migrations after terraform apply
38
+ belt deploy dev --skip-migrations
39
+
40
+ belt db:migrate # dev, or $BELT_ENV if set
41
+ belt db:migrate dev01
42
+ belt db:migrate prod # prompts for confirmation (--yes to skip)
43
+ belt db:migrate:status prod # which versions are up / down
44
+ ```
45
+
46
+ ```
47
+ Status Migration ID Migration Name
48
+ --------------------------------------------------
49
+ up 20261030120000 Index usernames
50
+ down 20261102090000 Backfill display names
51
+ ```
52
+
53
+ During `belt deploy`, migrations run after terraform apply and the
54
+ nested-environment data copy (so rows copied from the parent get migrated
55
+ too), and before the frontend deploy. They also run after
56
+ `belt deploy --rebuild`. Nothing happens if `db/migrate/` is empty.
57
+
58
+ ## Where history is kept
59
+
60
+ Each applied version is an SSM parameter:
61
+
62
+ ```
63
+ /belt/<app>/<env>/schema_migrations/<version> → {"name": "...", "applied_at": "..."}
64
+ ```
65
+
66
+ It lives in SSM rather than a DynamoDB table so `belt db:copy` and the
67
+ nested-env copy (which move every `<app>-<env>-*` table) don't carry one env's
68
+ history into another. The deploying credentials need
69
+ `ssm:GetParametersByPath` and `ssm:PutParameter` on `/belt/<app>/*`.
70
+
71
+ To re-run a migration, delete its parameter:
72
+
73
+ ```bash
74
+ aws ssm delete-parameter --name /belt/myapp/dev/schema_migrations/20261030120000
75
+ ```
76
+
77
+ ## Failures
78
+
79
+ A version is recorded only after `up` returns. If one raises, it and every
80
+ later migration stay pending, and the command exits non-zero. In
81
+ `belt deploy` the infrastructure is already live, so the deploy carries on
82
+ (frontend included), prints a warning and exits 1 at the end. Fix the
83
+ migration and run `belt db:migrate <env>`.
84
+
85
+ ## Write them idempotent and batch-safe
86
+
87
+ Migrations run against live data, often while the previous code is still
88
+ serving requests:
89
+
90
+ - **Idempotent.** A migration that dies halfway is re-run from the start, and
91
+ two overlapping deploys can both run it. Only write rows that still need
92
+ the change, and make writes safe to repeat (set a value, don't increment).
93
+ - **Batch-safe.** A GSI created in the same deploy may still be backfilling,
94
+ so don't query the index you're populating — page through the base table
95
+ and write in small batches. Expect throttling on big tables.
96
+ - **Data only.** Keys, GSIs and table settings belong in Terraform
97
+ (`belt setup tables`). The deployed code should work both before and
98
+ after a migration runs.
99
+
100
+ ## See Also
101
+
102
+ - `belt explain data_seeding` — `db:copy` and `db:seed`
103
+ - `belt explain console` — the app-booting mechanism migrations reuse
104
+ - `belt explain deployment` — the full deploy lifecycle
@@ -0,0 +1,37 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Belt
4
+ # Base class for data migrations in `db/migrate/<timestamp>_<name>.rb`.
5
+ #
6
+ # # db/migrate/20261030120000_index_usernames.rb
7
+ # class IndexUsernames < Belt::Migration
8
+ # def up
9
+ # PoshNosh::MatchStore.new.index_usernames!
10
+ # end
11
+ # end
12
+ #
13
+ # `belt deploy <env>` runs pending migrations after terraform apply, and
14
+ # `belt db:migrate <env>` runs them by hand. Each version runs once per
15
+ # environment (nested envs included); applied versions are recorded in SSM.
16
+ #
17
+ # Migrations touch live data, possibly while a new GSI is still backfilling
18
+ # or the previous code is still serving requests, so `up` must be
19
+ # idempotent (safe to re-run after a partial failure) and batch-safe (page
20
+ # through the base table rather than querying a GSI it is populating).
21
+ class Migration
22
+ attr_reader :version, :name
23
+
24
+ def initialize(version:, name:)
25
+ @version = version
26
+ @name = name
27
+ end
28
+
29
+ def up
30
+ raise NotImplementedError, "#{self.class.name} must define #up"
31
+ end
32
+
33
+ def say(message)
34
+ puts " #{message}"
35
+ end
36
+ end
37
+ end
data/lib/belt/version.rb CHANGED
@@ -1,7 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Belt
4
- VERSION = '0.6.1'
4
+ VERSION = '0.6.3'
5
5
 
6
6
  # Ruby version new apps run on in Lambda. Rendered into the conveyor-belt
7
7
  # provider block (`ruby_version`) by `belt new` / `belt generate environment`.
@@ -62,33 +62,45 @@ resource "aws_acm_certificate" "app" {
62
62
  }
63
63
  }
64
64
 
65
- # DNS validation record for the certificate
66
- # Key by resource_record_name (not domain_name) because ACM uses the same
67
- # validation CNAME for both base domain and wildcard — keying by domain_name
68
- # creates duplicate records that Route 53 rejects.
69
- # Use ellipsis (...) to group duplicates, then take the first element since
70
- # they're identical anyway.
71
- resource "aws_route53_record" "cert_validation" {
72
- for_each = local.dns_enabled && !local.is_nested ? {
73
- for dvo in aws_acm_certificate.app[0].domain_validation_options : dvo.resource_record_name => {
74
- name = dvo.resource_record_name
75
- type = dvo.resource_record_type
76
- record = dvo.resource_record_value
77
- }...
78
- } : {}
65
+ # DNS validation record for the certificate.
66
+ # ACM issues one identical validation CNAME for both the base domain and the
67
+ # wildcard, so one record covers the cert. Key it by app_domain (known at plan)
68
+ # rather than by anything from domain_validation_options: those values are
69
+ # unknown until the cert exists, and for_each over them fails the first plan
70
+ # of a new env with "Invalid for_each argument".
71
+ resource "aws_route53_record" "acm_validation" {
72
+ for_each = local.dns_enabled && !local.is_nested ? toset([local.app_domain]) : toset([])
79
73
 
80
74
  zone_id = aws_route53_zone.app[0].zone_id
81
- name = each.value[0].name
82
- type = each.value[0].type
75
+ name = local.acm_validation_options[each.key].resource_record_name
76
+ type = local.acm_validation_options[each.key].resource_record_type
83
77
  ttl = 300
84
- records = [each.value[0].record]
78
+ records = [local.acm_validation_options[each.key].resource_record_value]
85
79
  allow_overwrite = true
86
80
  }
87
81
 
82
+ locals {
83
+ acm_validation_options = local.dns_enabled && !local.is_nested ? {
84
+ for dvo in aws_acm_certificate.app[0].domain_validation_options : dvo.domain_name => dvo
85
+ } : {}
86
+ }
87
+
88
+ # Older Belt modules created this record as cert_validation, keyed by the
89
+ # validation CNAME name. Forget those instances without deleting them:
90
+ # acm_validation upserts the same CNAME (allow_overwrite), and a delete would
91
+ # pull the record ACM needs for renewal. No-op for envs that never had it.
92
+ removed {
93
+ from = aws_route53_record.cert_validation
94
+
95
+ lifecycle {
96
+ destroy = false
97
+ }
98
+ }
99
+
88
100
  resource "aws_acm_certificate_validation" "app" {
89
101
  count = local.dns_enabled && !local.is_nested ? 1 : 0
90
102
  certificate_arn = aws_acm_certificate.app[0].arn
91
- validation_record_fqdns = [for record in aws_route53_record.cert_validation : record.fqdn]
103
+ validation_record_fqdns = [for record in aws_route53_record.acm_validation : record.fqdn]
92
104
  }
93
105
 
94
106
  # =============================================================================
@@ -2,6 +2,9 @@
2
2
  # Environments call this module with different variables.
3
3
 
4
4
  terraform {
5
+ # >= 1.7 for the removed block in dns.tf
6
+ required_version = ">= 1.7"
7
+
5
8
  required_providers {
6
9
  aws = {
7
10
  source = "hashicorp/aws"
@@ -137,6 +137,7 @@ belt output <env> # terraform output
137
137
  belt routes [-g PATTERN] [-f json] # Show route definitions
138
138
  belt contracts [-g PATTERN] [-f json] # Show API contracts
139
139
  belt console <env> # Interactive Ruby console
140
+ belt db:migrate <env> # Run pending db/migrate data migrations (deploy does this too)
140
141
  belt logs <lambda> [-f] [-s 5m] [-e env] # View Lambda logs
141
142
  belt doctor # Check dependencies
142
143
  ```
@@ -269,7 +270,8 @@ belt deploy <env> # Deploy everything (init → plan → apply)
269
270
  | `rails routes` | `belt routes` |
270
271
  | `rails console` | `belt console <env>` |
271
272
  | `config/routes.rb` → URL paths | `config/routes.rb` → URL paths + infrastructure |
272
- | Migrations | `belt setup tables <env>` |
273
+ | Schema migrations | `belt setup tables <env>` |
274
+ | Data migrations (`rails db:migrate`) | `db/migrate/*.rb` + `belt db:migrate <env>` (run by `belt deploy`) |
273
275
  | `has_many` / `belongs_to` | Manual (no joins in DynamoDB) |
274
276
  | Template rendering | JSON responses (default) |
275
277
 
data/references/cli.md CHANGED
@@ -21,13 +21,15 @@ default environment so you can omit the `<env>` argument.
21
21
  | `belt doctor` | Check system deps + AWS config. |
22
22
  | `belt plugin new <name>` | Scaffold a Belt plugin gem. |
23
23
  | `belt explain <topic>` | Explain a concept (see topic list below). |
24
- | `belt deploy [env] [--auto] [--skip-backup] [--backup-only]` | Deploy to AWS (init → plan → apply, runs backups first if configured). |
24
+ | `belt deploy [env] [--auto] [--skip-backup] [--backup-only] [--skip-migrations]` | Deploy to AWS (init → plan → apply, runs backups first if configured, data migrations after). |
25
25
  | `belt deploy frontend <env> [--frontend NAME]` | Build + deploy frontend(s). |
26
26
  | `belt dns <deploy\|add <env>\|show>` | Manage the root DNS zone. |
27
27
  | `belt frontend <env <env>\|list>` | Write `<frontend>/.env` from TF outputs, or list frontends. |
28
28
  | `belt server [--frontend NAME]` (alias `s`) | Start local dev server. |
29
29
  | `belt db:copy <from> <to> [--force]` | Copy DynamoDB data between environments. |
30
30
  | `belt db:seed [env] [--force]` | Run `config/seeds.rb` against an environment. |
31
+ | `belt db:migrate [env] [--yes]` | Run pending `db/migrate` data migrations (also runs after `belt deploy`). |
32
+ | `belt db:migrate:status [env]` | Show which data migrations have run against an environment. |
31
33
  | `belt version` | Show Belt version. |
32
34
 
33
35
  ### Terraform shorthand
@@ -64,6 +64,18 @@ belt db:seed [env] [--force] # run config/seeds.rb in the booted console con
64
64
  `db:copy` skips non-empty destination tables by default. `db:seed` refuses to
65
65
  run against an env that already has data unless `--force`.
66
66
 
67
+ ## Data migrations
68
+
69
+ ```bash
70
+ belt db:migrate [env] [--yes] # run pending db/migrate/<timestamp>_<name>.rb
71
+ belt db:migrate:status [env] # up/down per version
72
+ ```
73
+
74
+ `belt deploy` runs pending migrations after apply (`--skip-migrations` to
75
+ opt out). Each version runs once per env, recorded in SSM at
76
+ `/belt/<app>/<env>/schema_migrations/<version>`. Write them idempotent and
77
+ batch-safe: see `belt explain migrations`.
78
+
67
79
  ## Observability
68
80
 
69
81
  `Belt::LambdaHandler` wires these global facades automatically:
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: belt
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.1
4
+ version: 0.6.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Stowzilla
@@ -121,6 +121,7 @@ files:
121
121
  - lib/belt/cli/console_command.rb
122
122
  - lib/belt/cli/contracts_command.rb
123
123
  - lib/belt/cli/db_copy_command.rb
124
+ - lib/belt/cli/db_migrate_command.rb
124
125
  - lib/belt/cli/db_seed_command.rb
125
126
  - lib/belt/cli/deploy_command.rb
126
127
  - lib/belt/cli/destroy_command.rb
@@ -142,6 +143,7 @@ files:
142
143
  - lib/belt/cli/index_command.rb
143
144
  - lib/belt/cli/lambda_config_command.rb
144
145
  - lib/belt/cli/logs_command.rb
146
+ - lib/belt/cli/migration_store.rb
145
147
  - lib/belt/cli/nested_environment.rb
146
148
  - lib/belt/cli/new_command.rb
147
149
  - lib/belt/cli/path_gem_materializer.rb
@@ -168,6 +170,7 @@ files:
168
170
  - lib/belt/docs/frontend.md
169
171
  - lib/belt/docs/generators.md
170
172
  - lib/belt/docs/lambda_handler.md
173
+ - lib/belt/docs/migrations.md
171
174
  - lib/belt/docs/models.md
172
175
  - lib/belt/docs/observability.md
173
176
  - lib/belt/docs/plugins.md
@@ -181,6 +184,7 @@ files:
181
184
  - lib/belt/http_status.rb
182
185
  - lib/belt/inflector.rb
183
186
  - lib/belt/lambda_handler.rb
187
+ - lib/belt/migration.rb
184
188
  - lib/belt/observability.rb
185
189
  - lib/belt/parameters.rb
186
190
  - lib/belt/rendering.rb