schema_ferry 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (30) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +20 -1
  3. data/README.md +34 -32
  4. data/lib/schema_ferry/cli.rb +19 -10
  5. data/lib/schema_ferry/config.rb +109 -0
  6. data/lib/schema_ferry/{converter → core}/column_converter.rb +4 -4
  7. data/lib/schema_ferry/{converter → core}/enum_check_builder.rb +8 -6
  8. data/lib/schema_ferry/{converter/identifier_shortener.rb → core/identifier_shortenable.rb} +4 -14
  9. data/lib/schema_ferry/core/schema_model.rb +60 -0
  10. data/lib/schema_ferry/core/schemafile_renderer.rb +109 -0
  11. data/lib/schema_ferry/core/table_converter.rb +159 -0
  12. data/lib/schema_ferry/core/translator.rb +53 -0
  13. data/lib/schema_ferry/{converter → core}/type_mapper.rb +2 -2
  14. data/lib/schema_ferry/errors.rb +2 -0
  15. data/lib/schema_ferry/io/mysql_reader.rb +80 -0
  16. data/lib/schema_ferry/{target/ridgepole_runner.rb → io/postgres_writer.rb} +11 -6
  17. data/lib/schema_ferry/pipeline.rb +24 -7
  18. data/lib/schema_ferry/support/drop_detectable.rb +17 -0
  19. data/lib/schema_ferry/support/warnings.rb +13 -0
  20. data/lib/schema_ferry/version.rb +1 -1
  21. data/lib/schema_ferry.rb +14 -14
  22. metadata +20 -14
  23. data/lib/schema_ferry/converter/schema_converter.rb +0 -172
  24. data/lib/schema_ferry/dsl/config.rb +0 -77
  25. data/lib/schema_ferry/dsl/table_rule.rb +0 -34
  26. data/lib/schema_ferry/schema_model.rb +0 -56
  27. data/lib/schema_ferry/source/connection_registry.rb +0 -29
  28. data/lib/schema_ferry/source/mysql_reader.rb +0 -88
  29. data/lib/schema_ferry/target/schemafile_renderer.rb +0 -143
  30. data/lib/schema_ferry/warnings.rb +0 -13
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e954e39c0309126d404ec1a7b95cfcaf062cbdf18525e6d179e8747c96d95aaf
4
- data.tar.gz: 29c7495b19fb18d7cb9ea1a17fb64b8778dc7deeefeeee50858b0584adc57fe8
3
+ metadata.gz: 689fd7901be3470c89447bcf83a61fb4185880e53f5723afa3385d2c23850a29
4
+ data.tar.gz: e74cd92c6c2b0c446b4b57e5460311f9885044f3f1f1ad2b86b23fc926cb0671
5
5
  SHA512:
6
- metadata.gz: 939a4335e6ad17b7eabbde3d4f3979e09009493cef5b474ec033cfbf853f4def56fb505d82aa9ab4366ab6ff6158196aed1f9a4ea762470cc5e3938a148f181e
7
- data.tar.gz: d7e983858d24f45938fbda3a8c76d4a0de06d496c949a91320903b0c49b1ff440450570440e7726860e6c2158bbddec080d3a92c78918f62182f84321ad21b42
6
+ metadata.gz: c918782073cfb88339251517b2985a022b7035eb813c21045afe9ba66b252adaf32350820ac756c035e795411963af678747a5aeb00ec82404ab1bc10b006ee7
7
+ data.tar.gz: 8a8d656502ee2ad147212f3920b19136ec31ca88f7b99d2391ade83046f9a0db476bbc163fdc496034e2d3048ff58219674aee03f8315a07b15b5ab4b576f9ac
data/CHANGELOG.md CHANGED
@@ -5,7 +5,26 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
- ## [Unreleased]
8
+ ## [0.3.0] - 2026-07-11
9
+
10
+ ### Added
11
+
12
+ - `apply!(allow_drops: false)` — refuses to apply anything (raising `SchemaFerry::DropNotAllowedError` instead) whenever the diff contains a column, index, foreign key, or constraint drop. The default remains `allow_drops: true`, matching the gem's "MySQL is the source of truth" philosophy, but this gives an escape hatch for handoff periods where a target-only addition might otherwise get swept up in a legitimate MySQL-driven drop. The CLI exposes the same behavior via `schema_ferry apply --disable-drops`.
13
+
14
+ ### Changed
15
+
16
+ - `map_column :col, type: :type` is now `column :col, map_type_to: :type`. `map_column` implied a mapping between two columns, which isn't what it does — it overrides one column's own type in place. `column` pairs with the existing `table` block, and `map_type_to:` mirrors `map_type`'s `to:` keyword for the same kind of type override.
17
+ - `apply!` now wraps the whole migration in a single transaction (via ridgepole's `--pre-query`/`--post-query`), so a failure partway through no longer leaves a half-applied schema on the target.
18
+ - A table name over 63 bytes now raises `SchemaFerry::ConversionError` instead of just warning. `apply!` was never actually going to succeed for it anyway — ActiveRecord's own PostgreSQL schema statements reject an over-length `create_table` outright — so the warning was misleading about how far the pipeline would actually get. `ignore_table` it, or rename it in MySQL.
19
+
20
+ ### Removed
21
+
22
+ - `Pipeline#schemafile` — it exposed the generated ridgepole Schemafile as a string without connecting to the target. That text is an internal hand-off format between schema_ferry and ridgepole, and keeping it public meant committing to a stable, human-friendly rendering of it forever. Pipeline's API is now just `dry_run` and `apply!`; to preview what a run would do, use `dry_run` — it shows the actual diff against the target rather than the full declared schema.
23
+
24
+ ### Fixed
25
+
26
+ - A MySQL `MULTIPOINT` column silently passed through as a plain PostgreSQL integer instead of raising. ActiveRecord misdetects `MULTIPOINT` as `:integer`, the same bug that already affected `POINT` — the check only matched `POINT` by name, so `MULTIPOINT` slipped past it. All eight MySQL spatial types (`GEOMETRY`, `POINT`, `LINESTRING`, `POLYGON`, `MULTIPOINT`, `MULTILINESTRING`, `MULTIPOLYGON`, `GEOMETRYCOLLECTION`) are now matched directly by `sql_type`, so they all raise the same explicit "no PostgreSQL equivalent without PostGIS" error instead of some going through a generic "unknown type" message.
27
+ - `Pipeline#apply!(allow_drops: false)` read the MySQL schema twice — once for the drop pre-check and once for the actual apply — so a schema change landing on MySQL between the two reads could apply a drop the pre-check never saw. The schema is now read and converted once, and the same Schemafile is used for both the pre-check and the apply, matching what the CLI already did.
9
28
 
10
29
  ## [0.2.0] - 2026-07-06
11
30
 
data/README.md CHANGED
@@ -6,7 +6,7 @@ You're migrating a production MySQL database to PostgreSQL. Moving the data take
6
6
  - **Sensible defaults, fully customizable** — built-in type mappings handle most cases; override anything with a few DSL rules
7
7
  - **Safe to iterate** — `dry_run` shows the exact changes that would be applied, before touching anything
8
8
 
9
- schema_ferry is designed to run repeatedly (e.g. via cron). Data migration is out of scope — pair it with [pgloader](https://github.com/dimitri/pgloader) (one-shot bulk copy) or CDC replication (AWS DMS, [Debezium](https://github.com/debezium/debezium), …), which load rows into the tables schema_ferry keeps in sync.
9
+ schema_ferry is designed to run repeatedly — as a step in whatever CI/CD pipeline you already have (Jenkins, Step Functions, GitHub Actions, …). Data migration is out of scope — pair it with [pgloader](https://github.com/dimitri/pgloader) (one-shot bulk copy) or CDC replication (AWS DMS, [Debezium](https://github.com/debezium/debezium), …), which load rows into the tables schema_ferry keeps in sync.
10
10
 
11
11
  ## Requirements
12
12
 
@@ -41,13 +41,11 @@ pipeline.dry_run # returns the changes that would be applied, without applying
41
41
  pipeline.apply! # applies the schema to PostgreSQL
42
42
  ```
43
43
 
44
- There is also `pipeline.schemafile`, which returns the generated schema as a string without connecting to the target.
45
-
46
- `apply!` makes the target match the generated schema — including **dropping** columns and indexes from the target that are not part of it. Before running against a target that holds data, read [Coverage](#coverage) below.
44
+ `apply!` makes the target match the generated schema including **dropping** columns, indexes, foreign keys, and constraints from the target that are not part of it. Before running against a target that holds data, read [Handoff](#handoff) below.
47
45
 
48
46
  ### CLI
49
47
 
50
- For cron jobs or whenever you'd rather not write a runner script there is a small CLI. Put the same DSL (without the `SchemaFerry.define` wrapper) in a `Ferryfile`:
48
+ As a step in an existing pipeline, or whenever you'd rather not write a runner script, there is a small CLI. Put the same DSL (without the `SchemaFerry.define` wrapper) in a `Ferryfile`:
51
49
 
52
50
  ```ruby
53
51
  source "mysql2://user:password@host:3306/source_db"
@@ -60,9 +58,10 @@ Then:
60
58
  schema_ferry dry-run # show what would change (reads ./Ferryfile)
61
59
  schema_ferry apply # apply to PostgreSQL
62
60
  schema_ferry apply -c path/to/Ferryfile # explicit definition file path
61
+ schema_ferry apply --disable-drops # refuse instead of applying if the diff contains a drop
63
62
  ```
64
63
 
65
- Each command prints the changes it applied (or would apply) followed by a one-line summary (`118 tables synced, 3 changes applied`). The exit status is 0 on success and 1 on any error, so cron mail and monitoring can rely on it.
64
+ Each command prints the changes it applied (or would apply) followed by a one-line summary (`118 tables synced, 3 changes applied`). The exit status is 0 on success and 1 on any error, so your monitoring can rely on it.
66
65
 
67
66
  ### Custom conversion rules
68
67
 
@@ -75,7 +74,7 @@ pipeline = SchemaFerry.define do
75
74
  map_type :json, to: :json # e.g. opt out of the default json → jsonb conversion
76
75
 
77
76
  table :users do
78
- map_column :is_admin, type: :boolean # override a specific column's type
77
+ column :is_admin, map_type_to: :boolean # override a specific column's type
79
78
  ignore_column :legacy_field # exclude a column
80
79
  ignore_index :idx_old_legacy # exclude an index
81
80
  end
@@ -84,25 +83,7 @@ pipeline = SchemaFerry.define do
84
83
  end
85
84
  ```
86
85
 
87
- The same rules work in a CLI definition file.
88
-
89
- ### Coverage
90
-
91
- schema_ferry syncs what can be done automatically — exactly where possible, or as an approximation with a warning where it isn't — and leaves the rest to add by hand, later. Where there's no reasonable equivalent at all, it raises instead of guessing. Add the column or index by hand once you're fully cut over to PostgreSQL — not before, since `apply!` delegates to [ridgepole](https://github.com/ridgepole/ridgepole), which drops anything missing from the generated schema on a managed table, including something added by hand as a stand-in — but never drops a table that's absent from it entirely.
92
-
93
- Review `dry_run` output before your first `apply!` and whenever you change the conversion rules — those are the moments that introduce drops. Unattended runs in between only mirror changes made to the MySQL schema; if even those need review, schedule `dry-run` instead and apply by hand.
94
-
95
- Normalized automatically, with a warning to stderr:
96
-
97
- - **Index prefix lengths** (`KEY (col(10))`) are dropped silently — PostgreSQL indexes the full column.
98
- - **Identifiers over 63 bytes** (MySQL allows 64): index and foreign key names are shortened deterministically (`first 54 bytes + _ + 8-char digest`), so repeated runs stay stable. Overlong table names are only warned about — rename those yourself.
99
- - **Zero-date defaults** (`'0000-00-00 00:00:00'`) are invalid in PostgreSQL and are dropped.
100
- - **BIGINT UNSIGNED columns on a foreign key** (either side) become signed `bigint` instead of `numeric(20)` — a numeric column cannot reference a bigint primary key. Values above 2⁶³−1 will not fit, the same trade-off as for `BIGINT UNSIGNED` primary keys.
101
-
102
- Raises instead:
103
-
104
- - **FULLTEXT indexes** — PostgreSQL has no equivalent construct (a `pg_trgm` GIN index is a common approximation, but it's not the same search semantics, so schema_ferry doesn't create one for you). Because of the drop behavior above, you can't pre-create a replacement during the sync period — add one once you're fully cut over to PostgreSQL. `ignore_index` them.
105
- - **Spatial columns** (`POINT`, `GEOMETRY`, `POLYGON`, `LINESTRING`, …) — PostgreSQL has no built-in equivalent without PostGIS, which schema_ferry does not manage. `ignore_column` them.
86
+ The same rules work in a `Ferryfile`.
106
87
 
107
88
  ## DSL reference
108
89
 
@@ -121,14 +102,14 @@ Raises instead:
121
102
 
122
103
  | Method | Description |
123
104
  |---|---|
124
- | `map_column :col, type: :type` | Override a column's type |
125
- | `map_column :col, type: :type, default: value` | …and give it an explicit default |
105
+ | `column :col, map_type_to: :type` | Override a column's type |
106
+ | `column :col, map_type_to: :type, default: value` | …and give it an explicit default |
126
107
  | `ignore_column :col` | Exclude a column |
127
108
  | `ignore_index :index_name` | Exclude an index |
128
109
 
129
- Ignoring a column also drops indexes and foreign keys that reference it. Renaming tables or columns is out of scope — clean up names after the cutover with a regular migration.
110
+ Ignoring a column also drops indexes and foreign keys that reference it.
130
111
 
131
- **tinyint(1) caveat:** ActiveRecord reads `tinyint(1)` as boolean, including its default (`DEFAULT 2` is read as `true`). If a `tinyint(1)` column actually holds 0/1/2-style values, override both the type and the default: `map_column :flags, type: :integer, default: 2`. Without an explicit default, schema_ferry drops the unreliable boolean default and warns.
112
+ **tinyint(1) caveat:** ActiveRecord reads `tinyint(1)` as boolean, including its default (`DEFAULT 2` is read as `true`). If a `tinyint(1)` column actually holds 0/1/2-style values, override both the type and the default: `column :flags, map_type_to: :integer, default: 2`. Without an explicit default, schema_ferry drops the unreliable boolean default and warns.
132
113
 
133
114
  ## Default type mapping
134
115
 
@@ -139,7 +120,7 @@ Ignoring a column also drops indexes and foreign keys that reference it. Renamin
139
120
  | `TINYINT(1)` | `boolean` | see the caveat above if a column holds more than 0/1 |
140
121
  | `TINYINT`…`BIGINT` (signed) | `smallint` / `integer` / `bigint` | widths normalized to PostgreSQL's three integer sizes |
141
122
  | `TINYINT`…`INT` `UNSIGNED` | one size larger | e.g. `INT UNSIGNED` → `bigint` |
142
- | `BIGINT UNSIGNED` | `numeric(20)` | PostgreSQL has no unsigned 8-byte integer; emitted with a warning. Columns on a foreign key become signed `bigint` instead — see above |
123
+ | `BIGINT UNSIGNED` | `numeric(20)` | PostgreSQL has no unsigned 8-byte integer; emitted with a warning. Columns on a foreign key become signed `bigint` instead — see [Handoff](#handoff) below |
143
124
  | `FLOAT` / `DOUBLE` | `double precision` | |
144
125
  | `DECIMAL(p,s)` | `numeric(p,s)` | |
145
126
  | `DATETIME` / `TIMESTAMP` | `timestamp` | use `map_type :datetime, to: :timestamptz` for `timestamptz` |
@@ -148,7 +129,28 @@ Ignoring a column also drops indexes and foreign keys that reference it. Renamin
148
129
  | `JSON` | `jsonb` | opt out with `map_type :json, to: :json` |
149
130
  | `ENUM(...)` | `varchar` | add `enum_as :check` to enforce the values with a CHECK constraint |
150
131
 
151
- `map_type` / `map_column` take Rails-style abstract type symbols (`:string`, `:integer`, `:jsonb`, …), not raw SQL type names.
132
+ `map_type` / `column`'s `map_type_to:` take Rails-style abstract type symbols (`:string`, `:integer`, `:jsonb`, …), not raw SQL type names.
133
+
134
+ ## Handoff
135
+
136
+ MySQL is the source of truth: `apply!` makes PostgreSQL match the generated schema exactly, so anything else on the target — including a column or index added by hand as an early stand-in — gets dropped. That's intentional. Add the real thing by hand once you're fully cut over to PostgreSQL, not before. The one exception is a table absent from the generated schema entirely — that's left alone.
137
+
138
+ Though that's the philosophy, if you don't want it, you can switch it off with `apply!(allow_drops: false)`: `apply!` then only goes through when the diff is drop-free; otherwise it raises `SchemaFerry::DropNotAllowedError` instead of applying anything to the target.
139
+
140
+ Within that generated schema, schema_ferry syncs what can be done automatically — exactly where possible, or as an approximation with a warning where it isn't — and leaves the rest to add by hand, later. Where there's no reasonable equivalent at all, it raises instead of guessing.
141
+
142
+ Normalized automatically, with a warning to stderr:
143
+
144
+ - **Index prefix lengths** (`KEY (col(10))`) are dropped silently — PostgreSQL indexes the full column.
145
+ - **Index and foreign key names over 63 bytes** (MySQL allows 64) are shortened deterministically (`first 54 bytes + _ + 8-char digest`), so repeated runs stay stable.
146
+ - **Zero-date defaults** (`'0000-00-00 00:00:00'`) are invalid in PostgreSQL and are dropped.
147
+ - **BIGINT UNSIGNED columns on a foreign key** (either side) become signed `bigint` instead of `numeric(20)` — a numeric column cannot reference a bigint primary key. Values above 2⁶³−1 will not fit, the same trade-off as for `BIGINT UNSIGNED` primary keys.
148
+
149
+ Raises instead:
150
+
151
+ - **FULLTEXT indexes** — PostgreSQL has no equivalent construct (a `pg_trgm` GIN index is a common approximation, but it's not the same search semantics, so schema_ferry doesn't create one for you). Because of the drop behavior above, you can't pre-create a replacement during the sync period — add one once you're fully cut over to PostgreSQL. `ignore_index` them.
152
+ - **Spatial columns** (`POINT`, `GEOMETRY`, `POLYGON`, `LINESTRING`, …) — PostgreSQL has no built-in equivalent without PostGIS, which schema_ferry does not manage. `ignore_column` them.
153
+ - **Table names over 63 bytes** (MySQL allows 64) — unlike index and foreign key names, schema_ferry won't rename a table for you (too invasive to do silently). `ignore_table` it, or rename it in MySQL.
152
154
 
153
155
  ## How it works
154
156
 
@@ -3,16 +3,16 @@
3
3
  require "optparse"
4
4
 
5
5
  module SchemaFerry
6
- # Minimal command-line interface: `schema_ferry <apply|dry-run> [-c FILE]`.
7
- # #run returns a process exit code instead of calling Kernel#exit so the
8
- # class stays testable.
9
6
  class CLI
7
+ include Support::DropDetectable
8
+
10
9
  DEFAULT_CONFIG_PATH = "Ferryfile"
11
10
  COMMANDS = %w[apply dry-run].freeze
12
11
 
13
12
  def initialize(stdout: $stdout, stderr: $stderr)
14
- @stdout = stdout
15
- @stderr = stderr
13
+ @stdout = stdout
14
+ @stderr = stderr
15
+ @allow_drops = true
16
16
  end
17
17
 
18
18
  def run(argv)
@@ -33,10 +33,18 @@ module SchemaFerry
33
33
  private
34
34
 
35
35
  def run_command(command)
36
- config = load_config
37
- schemafile = Pipeline.new(config).schemafile
38
- dry_run = command == "dry-run"
39
- output = Target::RidgepoleRunner.new(config.target_url).run(schemafile, dry_run: dry_run)
36
+ config = load_config
37
+ mysql_tables = IO::MysqlReader.new(config.source_url).read_all
38
+ schemafile = Core::Translator.new(config).translate(mysql_tables)
39
+ dry_run = command == "dry-run"
40
+ writer = IO::PostgresWriter.new(config.target_url)
41
+
42
+ if command == "apply" && !@allow_drops
43
+ drops = detect_drops(writer.run(schemafile, dry_run: true))
44
+ raise DropNotAllowedError, "refused: the diff contains drop(s):\n#{drops.join("\n")}" unless drops.empty?
45
+ end
46
+
47
+ output = writer.run(schemafile, dry_run: dry_run)
40
48
 
41
49
  @stdout.puts output
42
50
  @stdout.puts summary(schemafile, output, dry_run: dry_run)
@@ -72,13 +80,14 @@ module SchemaFerry
72
80
  path = @config_path || DEFAULT_CONFIG_PATH
73
81
  raise ConfigError, "definition file not found: #{path}" unless File.exist?(path)
74
82
 
75
- DSL::Config.load_file(path)
83
+ Config.load_file(path)
76
84
  end
77
85
 
78
86
  def parser
79
87
  @parser ||= OptionParser.new do |o|
80
88
  o.banner = "Usage: schema_ferry <apply|dry-run> [options]"
81
89
  o.on("-c", "--config FILE", "Definition file (default: #{DEFAULT_CONFIG_PATH})") { |v| @config_path = v }
90
+ o.on("--disable-drops", "apply: refuse if the diff contains any drop") { @allow_drops = false }
82
91
  o.on("-h", "--help", "Show this help") { @mode = :help }
83
92
  o.on("--version", "Show version") { @mode = :version }
84
93
  end
@@ -0,0 +1,109 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SchemaFerry
4
+ class Config
5
+ attr_reader :source_url, :target_url,
6
+ :global_type_overrides,
7
+ :table_rules,
8
+ :ignored_tables,
9
+ :enum_mode
10
+
11
+ class << self
12
+ # Ruby entry point
13
+ def build(&block)
14
+ evaluate { |config| config.instance_eval(&block) }
15
+ end
16
+
17
+ # CLI entry point
18
+ def load_file(path)
19
+ evaluate { |config| config.instance_eval(File.read(path), path.to_s, 1) }
20
+ end
21
+
22
+ private
23
+
24
+ def evaluate
25
+ new.tap do |config|
26
+ yield config
27
+ config.validate!
28
+ end
29
+ end
30
+ end
31
+
32
+ def initialize
33
+ @global_type_overrides = {}
34
+ @table_rules = Hash.new(TableRule::EMPTY)
35
+ @ignored_tables = []
36
+ @enum_mode = :string
37
+ end
38
+
39
+ def source(url)
40
+ @source_url = url
41
+ end
42
+
43
+ def target(url)
44
+ @target_url = url
45
+ end
46
+
47
+ def map_type(mysql_type, to:)
48
+ @global_type_overrides[mysql_type.to_sym] = to.to_sym
49
+ end
50
+
51
+ def table(table_name, &)
52
+ rule = TableRule.new(table_name)
53
+ rule.instance_eval(&)
54
+ @table_rules[table_name.to_s] = rule
55
+ end
56
+
57
+ def ignore_table(table_name)
58
+ @ignored_tables << table_name.to_s
59
+ end
60
+
61
+ # How to convert MySQL enum columns:
62
+ # :string — plain varchar, values not enforced (default)
63
+ # :check — varchar plus a CHECK constraint restricting the values
64
+ def enum_as(mode)
65
+ modes = %i[string check]
66
+ unless modes.include?(mode.to_sym)
67
+ raise ConfigError, "enum_as accepts #{modes.map(&:inspect).join(" or ")}, got #{mode.inspect}"
68
+ end
69
+
70
+ @enum_mode = mode.to_sym
71
+ end
72
+
73
+ def validate!
74
+ raise ConfigError, 'source is not configured. Add: source "mysql2://..."' unless @source_url
75
+ raise ConfigError, 'target is not configured. Add: target "postgresql://..."' unless @target_url
76
+ end
77
+
78
+ # Vocabulary available inside a `table :name do ... end` block.
79
+ class TableRule
80
+ attr_reader :table_name, :column_type_overrides,
81
+ :column_default_overrides, :ignored_columns, :ignored_indexes
82
+
83
+ def initialize(table_name)
84
+ @table_name = table_name.to_s
85
+ @column_type_overrides = {}
86
+ @column_default_overrides = {}
87
+ @ignored_columns = []
88
+ @ignored_indexes = []
89
+ end
90
+
91
+ EMPTY = new("").freeze # Null object for tables without a `table ... do` block.
92
+ DEFAULT_OMITTED = Object.new.freeze
93
+ private_constant :DEFAULT_OMITTED
94
+
95
+ def column(column_name, map_type_to:, default: DEFAULT_OMITTED)
96
+ @column_type_overrides[column_name.to_s] = map_type_to.to_sym
97
+ @column_default_overrides[column_name.to_s] = default unless default.equal?(DEFAULT_OMITTED)
98
+ end
99
+
100
+ def ignore_column(column_name)
101
+ @ignored_columns << column_name.to_s
102
+ end
103
+
104
+ def ignore_index(index_name)
105
+ @ignored_indexes << index_name.to_s
106
+ end
107
+ end
108
+ end
109
+ end
@@ -1,9 +1,9 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SchemaFerry
4
- module Converter
4
+ module Core
5
5
  class ColumnConverter
6
- include Warnings
6
+ include Support::Warnings
7
7
 
8
8
  # PG has no unsigned integers: bump to the next size that holds the full
9
9
  # unsigned range. Keys/values are AR byte limits.
@@ -16,7 +16,7 @@ module SchemaFerry
16
16
  def call(raw, table_name, rule, fk_columns = [])
17
17
  raw = bump_unsigned_integer(raw, table_name, fk_columns.include?(raw[:name]))
18
18
  raw = drop_zero_date_default(raw, table_name)
19
- override = rule&.column_type_overrides&.[](raw[:name])
19
+ override = rule.column_type_overrides[raw[:name]]
20
20
  col_opts = raw.slice(:limit, :precision, :scale, :null, :default, :default_function, :comment)
21
21
 
22
22
  if override
@@ -71,7 +71,7 @@ module SchemaFerry
71
71
  emit_warning "#{rule.table_name}.#{raw[:name]}: dropping default #{default.inspect} — " \
72
72
  "MySQL reported a tinyint(1) default as boolean, which is unreliable under " \
73
73
  "a type override. Restore it explicitly: " \
74
- "map_column :#{raw[:name]}, type: :#{override}, default: <value>"
74
+ "column :#{raw[:name]}, map_type_to: :#{override}, default: <value>"
75
75
  return nil
76
76
  end
77
77
  default
@@ -1,16 +1,18 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SchemaFerry
4
- module Converter
4
+ module Core
5
5
  # Builds CHECK constraints enforcing MySQL enum values on varchar columns
6
6
  # (enum_as :check).
7
7
  class EnumCheckBuilder
8
- def call(raw_table, rule, ignored)
8
+ include IdentifierShortenable
9
+
10
+ def call(raw_table, rule)
9
11
  raw_table[:columns].filter_map do |col|
10
- next if ignored.include?(col[:name])
12
+ next if rule.ignored_columns.include?(col[:name])
11
13
  # A type override takes the column away from varchar; the caller owns
12
14
  # any constraint then.
13
- next if rule&.column_type_overrides&.key?(col[:name])
15
+ next if rule.column_type_overrides.key?(col[:name])
14
16
 
15
17
  values = enum_values(col[:sql_type])
16
18
  next if values.nil?
@@ -24,8 +26,8 @@ module SchemaFerry
24
26
  def build_constraint(table_name, col_name, values)
25
27
  CheckConstraintSchema.new(
26
28
  expression: expression(col_name, values),
27
- name: IdentifierShortener.shorten("chk_#{table_name}_#{col_name}",
28
- kind: "check constraint", table: table_name)
29
+ name: shorten_identifier("chk_#{table_name}_#{col_name}",
30
+ kind: "check constraint", table: table_name)
29
31
  )
30
32
  end
31
33
 
@@ -3,19 +3,17 @@
3
3
  require "digest"
4
4
 
5
5
  module SchemaFerry
6
- module Converter
6
+ module Core
7
7
  # PostgreSQL truncates identifiers to 63 bytes (MySQL allows 64). A silently
8
8
  # truncated index name makes ridgepole see a diff on every run, so names that
9
9
  # would overflow are shortened deterministically instead.
10
- module IdentifierShortener
11
- extend Warnings
10
+ module IdentifierShortenable
11
+ include Support::Warnings
12
12
 
13
13
  MAX_BYTES = 63
14
14
  HASH_LENGTH = 8
15
15
 
16
- module_function
17
-
18
- def shorten(name, kind:, table:)
16
+ def shorten_identifier(name, kind:, table:)
19
17
  return name if name.nil? || name.bytesize <= MAX_BYTES
20
18
 
21
19
  prefix = name.byteslice(0, MAX_BYTES - HASH_LENGTH - 1)
@@ -24,14 +22,6 @@ module SchemaFerry
24
22
  "#{MAX_BYTES}-byte identifier limit; renamed to #{short.inspect}."
25
23
  short
26
24
  end
27
-
28
- def warn_long_table_name(name)
29
- return if name.bytesize <= MAX_BYTES
30
-
31
- emit_warning "table name #{name.inspect} exceeds PostgreSQL's #{MAX_BYTES}-byte " \
32
- "identifier limit and will be truncated by PostgreSQL. Rename the " \
33
- "table before applying to avoid ridgepole re-creating it on every run."
34
- end
35
25
  end
36
26
  end
37
27
  end
@@ -0,0 +1,60 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Plain data structs for the converted (PostgreSQL-side) schema: TableConverter
4
+ # builds these, SchemafileRenderer reads them.
5
+ module SchemaFerry
6
+ module Core
7
+ TableSchema = Struct.new(
8
+ :name, # String
9
+ :primary_key, # String | Array<String> | nil
10
+ :pk_type, # Symbol: :bigint, :integer, :string, etc. (single-column PK only)
11
+ :pk_limit, # Integer | nil (limit of a string PK column)
12
+ :comment, # String | nil
13
+ :columns, # Array<ColumnSchema>
14
+ :indexes, # Array<IndexSchema>
15
+ :foreign_keys, # Array<ForeignKeySchema>
16
+ :check_constraints, # Array<CheckConstraintSchema>
17
+ keyword_init: true
18
+ )
19
+
20
+ ColumnSchema = Struct.new(
21
+ :name, # String
22
+ :type, # Symbol (:string, :integer, :jsonb, …)
23
+ :limit, # Integer | nil
24
+ :precision, # Integer | nil
25
+ :scale, # Integer | nil
26
+ :null, # Boolean
27
+ :default, # Object | nil
28
+ :default_function, # String | nil (e.g. "CURRENT_TIMESTAMP")
29
+ :comment, # String | nil
30
+ keyword_init: true
31
+ )
32
+
33
+ IndexSchema = Struct.new(
34
+ :name, # String
35
+ :columns, # Array<String>
36
+ :unique, # Boolean
37
+ :using, # Symbol | nil
38
+ :lengths, # Integer | Hash | nil
39
+ :orders, # Symbol | Hash | nil
40
+ keyword_init: true
41
+ )
42
+
43
+ CheckConstraintSchema = Struct.new(
44
+ :expression, # String (e.g. "kind IN ('a', 'b')")
45
+ :name, # String
46
+ keyword_init: true
47
+ )
48
+
49
+ ForeignKeySchema = Struct.new(
50
+ :from_table, # String
51
+ :to_table, # String
52
+ :column, # String
53
+ :primary_key, # String
54
+ :on_update, # Symbol | nil
55
+ :on_delete, # Symbol | nil
56
+ :name, # String | nil
57
+ keyword_init: true
58
+ )
59
+ end
60
+ end
@@ -0,0 +1,109 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SchemaFerry
4
+ module Core
5
+ class SchemafileRenderer
6
+ Call = Struct.new(:name, :args, :opts, :children)
7
+ Raw = Struct.new(:source)
8
+
9
+ def render(tables)
10
+ calls = tables.map { |table| table_call(table) }
11
+ calls += tables.flat_map { |table| table.foreign_keys.map { |fk| foreign_key_call(fk) } }
12
+ calls.map { |call| serialize(call) }.join("\n\n")
13
+ end
14
+
15
+ private
16
+
17
+ def table_call(table)
18
+ children = table.columns.map { |col| column_call(col) }
19
+ children += table.indexes.map { |idx| index_call(idx) }
20
+ children += Array(table.check_constraints).map { |chk| check_constraint_call(chk) }
21
+ Call.new("create_table", [table.name], table_options(table), children)
22
+ end
23
+
24
+ def table_options(table)
25
+ opts = { force: :cascade }
26
+ case table.primary_key
27
+ when nil
28
+ opts[:id] = false
29
+ when String
30
+ opts[:primary_key] = table.primary_key if table.primary_key != "id"
31
+ if table.pk_type && table.pk_type != :bigint
32
+ opts[:id] = table.pk_type
33
+ opts[:limit] = table.pk_limit
34
+ end
35
+ when Array
36
+ opts[:primary_key] = table.primary_key
37
+ end
38
+ opts[:comment] = table.comment
39
+ opts
40
+ end
41
+
42
+ def column_call(col)
43
+ Call.new("t.#{col.type}", [col.name], {
44
+ limit: col.limit,
45
+ precision: col.precision,
46
+ scale: col.scale,
47
+ default: default_option(col),
48
+ null: (false if col.null == false),
49
+ comment: col.comment
50
+ })
51
+ end
52
+
53
+ def default_option(col)
54
+ return col.default unless col.default.nil?
55
+
56
+ Raw.new("-> { #{col.default_function.inspect} }") if col.default_function
57
+ end
58
+
59
+ def index_call(idx)
60
+ Call.new("t.index", [idx.columns], {
61
+ name: idx.name,
62
+ unique: (true if idx.unique),
63
+ using: idx.using,
64
+ order: idx.orders
65
+ })
66
+ end
67
+
68
+ def check_constraint_call(chk)
69
+ Call.new("t.check_constraint", [chk.expression], { name: chk.name })
70
+ end
71
+
72
+ def foreign_key_call(foreign_key)
73
+ Call.new("add_foreign_key", [foreign_key.from_table, foreign_key.to_table], {
74
+ column: (foreign_key.column if custom_fk_column?(foreign_key)),
75
+ primary_key: (foreign_key.primary_key if foreign_key.primary_key && foreign_key.primary_key != "id"),
76
+ name: foreign_key.name,
77
+ on_update: foreign_key.on_update,
78
+ on_delete: foreign_key.on_delete
79
+ })
80
+ end
81
+
82
+ # ridgepole's PostgreSQL export omits column: when it follows the Rails
83
+ # convention, and it compares foreign keys by their literal options —
84
+ # emitting column: for conventional names re-creates the FK every run.
85
+ def custom_fk_column?(foreign_key)
86
+ foreign_key.column && foreign_key.column != "#{foreign_key.to_table.singularize}_id"
87
+ end
88
+
89
+ def serialize(call, indent = "")
90
+ head = "#{indent}#{[call.name, arguments(call)].reject(&:empty?).join(" ")}"
91
+ return head unless call.children
92
+
93
+ ["#{head} do |t|",
94
+ *call.children.map { |child| serialize(child, "#{indent} ") },
95
+ "#{indent}end"].join("\n")
96
+ end
97
+
98
+ def arguments(call)
99
+ positional = call.args.map { |arg| literal(arg) }
100
+ keyword = call.opts.filter_map { |key, value| "#{key}: #{literal(value)}" unless value.nil? }
101
+ (positional + keyword).join(", ")
102
+ end
103
+
104
+ def literal(value)
105
+ value.is_a?(Raw) ? value.source : value.inspect
106
+ end
107
+ end
108
+ end
109
+ end