migsupo 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +58 -0
- data/lib/migsupo/coherent.rb +71 -0
- data/lib/migsupo/generator/migration_builder.rb +7 -1
- data/lib/migsupo/generator/schemafile_dumper.rb +18 -0
- data/lib/migsupo/tasks/migsupo.rake +79 -0
- data/lib/migsupo/version.rb +1 -1
- data/lib/migsupo.rb +1 -0
- metadata +5 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f6bd6bc4627dd5769bb11ae7f872a7234a09eccf52b4ed9661e162d7049dc87a
|
|
4
|
+
data.tar.gz: 796a9c73842e770248796ce1f35a5def6e006fe9c44b31485fd9ca21f98ac31f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 88c10374ed224574113524e17ea4856f8671f04022a669899136691c87387580f57f49d91458887261a962d143d033875a633b8e829ccff4fdaa9e8e338ed9a5
|
|
7
|
+
data.tar.gz: 14719373ed30592817a1b0e9107dd613fb3936c1c47a977fd0e38d8067bcf0ef982b2bae5df1cac74308a7bcecfa480559fa069b988ea3d32f378f7d641d3250
|
data/README.md
CHANGED
|
@@ -122,6 +122,64 @@ Exit with code 1 if the Schemafile and the current database are not in sync. Use
|
|
|
122
122
|
rails db:generate_migration:check
|
|
123
123
|
```
|
|
124
124
|
|
|
125
|
+
### `rails db:coherent` / `rails db:coherent:apply`
|
|
126
|
+
|
|
127
|
+
Adopt a hand-made database change as the truth. See [coherent](#coherent--when-the-db-was-changed-by-hand).
|
|
128
|
+
|
|
129
|
+
## `coherent` — when the DB was changed by hand
|
|
130
|
+
|
|
131
|
+
Sometimes someone edits the database directly — an emergency `ALTER TABLE` in production, say — and that state is the one you want to keep. The normal flow cannot express this: the Schemafile is behind, and running a migration to catch up would touch columns that are already correct.
|
|
132
|
+
|
|
133
|
+
`coherent` takes the **DB as the source of truth** and moves everything else up to it: it writes the migration file that *would* have produced the change (so every other environment gets it through the normal `rails db:migrate`), updates the Schemafile, and then records the migration as applied on this database **without executing it**.
|
|
134
|
+
|
|
135
|
+
```
|
|
136
|
+
DB (hand-edited, = truth)
|
|
137
|
+
├─ db:coherent → db/migrate/*.rb (generated) + Schemafile (updated)
|
|
138
|
+
└─ db:coherent:apply → INSERT INTO schema_migrations ← columns untouched
|
|
139
|
+
db/schema.rb re-dumped
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### 1. Generate
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
rails db:coherent
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
Adopting the following from the database:
|
|
150
|
+
add_column users.nickname (string)
|
|
151
|
+
|
|
152
|
+
Generated migration(s):
|
|
153
|
+
db/migrate/20260901120000_add_columns_to_users.rb
|
|
154
|
+
Updated Schemafile: Schemafile
|
|
155
|
+
|
|
156
|
+
Review them, then record the history without touching the database:
|
|
157
|
+
rails db:coherent:apply VERSION=20260901120000
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
The diff runs in the opposite direction from `db:generate_migration`: the DB is the desired state and the Schemafile is the current one, so a column that exists only in the DB comes out as `add_column`, not `remove_column`.
|
|
161
|
+
|
|
162
|
+
### 2. Review, then rewrite history only
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
rails db:coherent:apply VERSION=20260901120000
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
This inserts the version into `schema_migrations` and re-dumps `db/schema.rb`. No DDL is executed — the columns are already where they should be.
|
|
169
|
+
|
|
170
|
+
`VERSION` is required (comma-separate several). Run without it to list the pending migrations. Before writing anything, `apply` re-checks that the DB and the Schemafile match: if they do not, the migration genuinely needs to run and the task aborts.
|
|
171
|
+
|
|
172
|
+
### On other environments
|
|
173
|
+
|
|
174
|
+
Nothing special. `db/migrate/*.rb` is a normal migration, so staging and production pick it up with `rails db:migrate`.
|
|
175
|
+
|
|
176
|
+
### Caveats
|
|
177
|
+
|
|
178
|
+
- `db:coherent` regenerates the whole Schemafile from the DB, so hand-written comments and ordering in it are lost. Review the diff (`git diff Schemafile`) before committing.
|
|
179
|
+
- `rename_hints` are not applied — a column renamed by hand comes out as `remove_column` + `add_column`.
|
|
180
|
+
- Only what migsupo models is tracked: tables, columns, indexes. Foreign keys, extensions and check constraints are neither diffed nor written to the Schemafile.
|
|
181
|
+
- Primary key options (`id: :uuid`, `primary_key: "uid"`) are not read back from the database, so the regenerated Schemafile drops them. Diffs are unaffected — table options are not compared — but re-add them by hand if you ever recreate the table from the Schemafile.
|
|
182
|
+
|
|
125
183
|
## Environment Variables
|
|
126
184
|
|
|
127
185
|
| Variable | Default | Description |
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
require_relative "parser/schemafile_parser"
|
|
2
|
+
require_relative "loader/active_record_loader"
|
|
3
|
+
require_relative "differ/diff_calculator"
|
|
4
|
+
require_relative "generator/schemafile_dumper"
|
|
5
|
+
|
|
6
|
+
module Migsupo
|
|
7
|
+
# For the case where the DB was changed by hand and that state is now the
|
|
8
|
+
# truth. Coherent brings the Schemafile and the migration history up to the
|
|
9
|
+
# DB without touching a single column.
|
|
10
|
+
module Coherent
|
|
11
|
+
module_function
|
|
12
|
+
|
|
13
|
+
# desired/current are swapped on purpose: the migration has to move the
|
|
14
|
+
# declared state up to what the DB already looks like.
|
|
15
|
+
# ponytail: rename_hints point the other way here, so they are dropped -
|
|
16
|
+
# a hand-renamed column comes out as remove + add.
|
|
17
|
+
def diff(schemafile_path: nil)
|
|
18
|
+
schemafile_path ||= Migsupo.configuration.schemafile_path
|
|
19
|
+
declared = Parser::SchemafileParser.parse(schemafile_path)
|
|
20
|
+
|
|
21
|
+
Differ::DiffCalculator.new.calculate(desired: actual_schema, current: declared)
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def dump_schemafile(path: nil)
|
|
25
|
+
path ||= Migsupo.configuration.schemafile_path
|
|
26
|
+
File.write(path, Generator::SchemafileDumper.new.dump(actual_schema))
|
|
27
|
+
path
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# Writes history only - no DDL is executed. Returns the versions inserted.
|
|
31
|
+
def mark_applied(versions)
|
|
32
|
+
applied = applied_versions
|
|
33
|
+
inserted = versions.map(&:to_s) - applied
|
|
34
|
+
|
|
35
|
+
table = connection.quote_table_name("schema_migrations")
|
|
36
|
+
inserted.each do |version|
|
|
37
|
+
connection.execute("INSERT INTO #{table} (version) VALUES (#{connection.quote(version)})")
|
|
38
|
+
end
|
|
39
|
+
inserted
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def applied_versions
|
|
43
|
+
connection.select_values(
|
|
44
|
+
"SELECT version FROM #{connection.quote_table_name('schema_migrations')}"
|
|
45
|
+
).map(&:to_s)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
# [version, filename] for every migration file not yet in schema_migrations.
|
|
49
|
+
def pending(migrations_dir = nil)
|
|
50
|
+
migrations_dir ||= Migsupo.configuration.migrations_dir
|
|
51
|
+
applied = applied_versions
|
|
52
|
+
|
|
53
|
+
migration_files(migrations_dir).reject { |version, _| applied.include?(version) }
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def migration_files(migrations_dir)
|
|
57
|
+
Dir.glob(File.join(migrations_dir, "*.rb")).sort.filter_map do |path|
|
|
58
|
+
version = File.basename(path)[/\A\d+/]
|
|
59
|
+
[version, File.basename(path)] if version
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def actual_schema
|
|
64
|
+
Loader::ActiveRecordLoader.new(ignored_tables: Migsupo.configuration.ignored_tables).load_schema
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def connection
|
|
68
|
+
ActiveRecord::Base.connection
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
@@ -32,6 +32,12 @@ module Migsupo
|
|
|
32
32
|
RUBY
|
|
33
33
|
end
|
|
34
34
|
|
|
35
|
+
# Renders one create_table block (plus its add_index lines) - the same shape
|
|
36
|
+
# a Schemafile uses.
|
|
37
|
+
def build_table(table)
|
|
38
|
+
render_create_table(Differ::Operations::CreateTable.new(table))
|
|
39
|
+
end
|
|
40
|
+
|
|
35
41
|
private
|
|
36
42
|
|
|
37
43
|
def version_suffix
|
|
@@ -102,7 +108,7 @@ module Migsupo
|
|
|
102
108
|
|
|
103
109
|
def render_drop_table(op, direction:)
|
|
104
110
|
if direction == :down
|
|
105
|
-
render_create_table(Operations::CreateTable.new(op.table))
|
|
111
|
+
render_create_table(Differ::Operations::CreateTable.new(op.table))
|
|
106
112
|
else
|
|
107
113
|
"drop_table #{op.table_name.inspect}"
|
|
108
114
|
end
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
require_relative "migration_builder"
|
|
2
|
+
|
|
3
|
+
module Migsupo
|
|
4
|
+
module Generator
|
|
5
|
+
# Renders a SchemaDefinition back out as Schemafile DSL.
|
|
6
|
+
# Only what migsupo models (tables / columns / indexes) is emitted, so the
|
|
7
|
+
# output always round-trips through SchemafileParser.
|
|
8
|
+
class SchemafileDumper
|
|
9
|
+
def initialize
|
|
10
|
+
@builder = MigrationBuilder.new
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
def dump(schema)
|
|
14
|
+
"#{schema.tables.values.map { |table| @builder.build_table(table) }.join("\n\n")}\n"
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
@@ -54,3 +54,82 @@ namespace :db do
|
|
|
54
54
|
end
|
|
55
55
|
end
|
|
56
56
|
end
|
|
57
|
+
|
|
58
|
+
namespace :db do
|
|
59
|
+
desc "Take the current DB as truth: generate migrations for hand-made changes and update the Schemafile"
|
|
60
|
+
task coherent: :environment do
|
|
61
|
+
schemafile_path = ENV.fetch("SCHEMAFILE", Migsupo.configuration.schemafile_path)
|
|
62
|
+
output_dir = ENV.fetch("OUTPUT_DIR", Migsupo.configuration.migrations_dir)
|
|
63
|
+
dry_run = ENV["DRY_RUN"] == "true"
|
|
64
|
+
|
|
65
|
+
diff = Migsupo::Coherent.diff(schemafile_path: schemafile_path)
|
|
66
|
+
|
|
67
|
+
if diff.empty?
|
|
68
|
+
puts "No differences between the database and the Schemafile. Nothing for coherent to do."
|
|
69
|
+
next
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
puts "Adopting the following from the database:"
|
|
73
|
+
puts diff.to_s
|
|
74
|
+
puts
|
|
75
|
+
|
|
76
|
+
files = Migsupo.generate_migrations(diff, output_dir: output_dir, dry_run: dry_run)
|
|
77
|
+
next if dry_run
|
|
78
|
+
|
|
79
|
+
Migsupo::Coherent.dump_schemafile(path: schemafile_path)
|
|
80
|
+
versions = files.map { |f| File.basename(f)[/\A\d+/] }
|
|
81
|
+
|
|
82
|
+
puts "Generated migration(s):"
|
|
83
|
+
files.each { |f| puts " #{f}" }
|
|
84
|
+
puts "Updated Schemafile: #{schemafile_path}"
|
|
85
|
+
puts
|
|
86
|
+
puts "Review them, then record the history without touching the database:"
|
|
87
|
+
puts " rails db:coherent:apply VERSION=#{versions.join(',')}"
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
namespace :coherent do
|
|
91
|
+
desc "Mark the given migrations as applied without running them (VERSION=ts[,ts...])"
|
|
92
|
+
task apply: :environment do
|
|
93
|
+
schemafile_path = ENV.fetch("SCHEMAFILE", Migsupo.configuration.schemafile_path)
|
|
94
|
+
output_dir = ENV.fetch("OUTPUT_DIR", Migsupo.configuration.migrations_dir)
|
|
95
|
+
versions = ENV["VERSION"].to_s.split(",").map(&:strip).reject(&:empty?)
|
|
96
|
+
|
|
97
|
+
if versions.empty?
|
|
98
|
+
puts "VERSION is required (e.g. VERSION=20260901120000)."
|
|
99
|
+
pending = Migsupo::Coherent.pending(output_dir)
|
|
100
|
+
if pending.empty?
|
|
101
|
+
puts "No pending migrations."
|
|
102
|
+
else
|
|
103
|
+
puts "Pending migrations:"
|
|
104
|
+
pending.each { |version, name| puts " #{version} #{name}" }
|
|
105
|
+
end
|
|
106
|
+
exit 1
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
known = Migsupo::Coherent.migration_files(output_dir).to_h
|
|
110
|
+
unknown = versions.reject { |v| known.key?(v) }
|
|
111
|
+
unless unknown.empty?
|
|
112
|
+
puts "No migration found in #{output_dir} for: #{unknown.join(', ')}"
|
|
113
|
+
exit 1
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
# Recording history only is sound when the DB already matches the
|
|
117
|
+
# Schemafile. A remaining diff means the migration genuinely needs to run.
|
|
118
|
+
diff = Migsupo::Coherent.diff(schemafile_path: schemafile_path)
|
|
119
|
+
unless diff.empty?
|
|
120
|
+
puts "The database and the Schemafile do not match. Run rails db:coherent first:"
|
|
121
|
+
puts diff.to_s
|
|
122
|
+
exit 1
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
inserted = Migsupo::Coherent.mark_applied(versions)
|
|
126
|
+
skipped = versions - inserted
|
|
127
|
+
|
|
128
|
+
inserted.each { |v| puts "Recorded as applied: #{v} #{known[v]}" }
|
|
129
|
+
skipped.each { |v| puts "Already recorded, skipped: #{v} #{known[v]}" }
|
|
130
|
+
|
|
131
|
+
Rake::Task["db:schema:dump"].invoke
|
|
132
|
+
puts "Updated db/schema.rb."
|
|
133
|
+
end
|
|
134
|
+
end
|
|
135
|
+
end
|
data/lib/migsupo/version.rb
CHANGED
data/lib/migsupo.rb
CHANGED
|
@@ -9,6 +9,7 @@ require_relative "migsupo/loader/active_record_loader"
|
|
|
9
9
|
require_relative "migsupo/loader/schema_rb_loader"
|
|
10
10
|
require_relative "migsupo/differ/diff_calculator"
|
|
11
11
|
require_relative "migsupo/generator/migration_generator"
|
|
12
|
+
require_relative "migsupo/coherent"
|
|
12
13
|
|
|
13
14
|
module Migsupo
|
|
14
15
|
class << self
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: migsupo
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- masak1yu
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 1980-01-01 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activerecord
|
|
@@ -103,6 +103,7 @@ extra_rdoc_files: []
|
|
|
103
103
|
files:
|
|
104
104
|
- README.md
|
|
105
105
|
- lib/migsupo.rb
|
|
106
|
+
- lib/migsupo/coherent.rb
|
|
106
107
|
- lib/migsupo/configuration.rb
|
|
107
108
|
- lib/migsupo/differ/diff.rb
|
|
108
109
|
- lib/migsupo/differ/diff_calculator.rb
|
|
@@ -117,6 +118,7 @@ files:
|
|
|
117
118
|
- lib/migsupo/generator/migration_builder.rb
|
|
118
119
|
- lib/migsupo/generator/migration_generator.rb
|
|
119
120
|
- lib/migsupo/generator/naming.rb
|
|
121
|
+
- lib/migsupo/generator/schemafile_dumper.rb
|
|
120
122
|
- lib/migsupo/loader/active_record_loader.rb
|
|
121
123
|
- lib/migsupo/loader/schema_rb_loader.rb
|
|
122
124
|
- lib/migsupo/parser/dsl_context.rb
|
|
@@ -149,7 +151,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
149
151
|
- !ruby/object:Gem::Version
|
|
150
152
|
version: '0'
|
|
151
153
|
requirements: []
|
|
152
|
-
rubygems_version: 3.
|
|
154
|
+
rubygems_version: 3.4.19
|
|
153
155
|
signing_key:
|
|
154
156
|
specification_version: 4
|
|
155
157
|
summary: Generate Rails migrations from a Schemafile diff
|