pg_ha_migrations 2.0.0 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: cd0329ccdae5b3bf68ac759bc0eca2dbc5089c3d91f9ee6444c6791a2bd42e93
4
- data.tar.gz: ecf840233b36ede3ef278411bac124cefa32036b43d62119792a8d74ac126b5e
3
+ metadata.gz: 7b2e6bb12c48cffbc58e2f182f6d147f9952fd5167b82feaebee77190d0c71f0
4
+ data.tar.gz: f438b6095e6493bae0545cca0f9b3b3e5aa6d2dfc14d80a8c851e80f5b90e09d
5
5
  SHA512:
6
- metadata.gz: f744006470a25bff85e10026f750e1cada2b49a19809cc2279208f9ec10ac86bfacf38dbbe32bd839f655475d0dde773d1c219b584c4a5cccc7c911f570c10bd
7
- data.tar.gz: 5626ac149515ef764e63797cceed133c87d6032f22100a4940573a42a0b1ae483cbebc8a26596cc77494c4420e94e55d08e74b44ebe6fb83cd7f18e1a502d731
6
+ metadata.gz: 85b57ebf531422f9b14ff7d3e91a0759b9e4c714dd8b53dff2e5e45e0b44fe583ab726ad9a160f5d34de0bab102da0770caae5dad622e5294d4d8eba1b178a52
7
+ data.tar.gz: 2992367de5df4a1b57bcd024c39edebf40ecd5cc595a2b2b8034cd89af60d9bb068a4029e8771f66abf0fc7c0a73966d7c06bb0959c678df264e874276b5e73b
@@ -9,6 +9,7 @@ jobs:
9
9
  - 14
10
10
  - 15
11
11
  - 16
12
+ - 17
12
13
  ruby:
13
14
  - "3.2"
14
15
  - "3.3"
@@ -17,7 +18,15 @@ jobs:
17
18
  - rails_7.1
18
19
  - rails_7.2
19
20
  - rails_8.0
20
- name: PostgreSQL ${{ matrix.pg }} - Ruby ${{ matrix.ruby }} - ${{ matrix.gemfile }}
21
+ partman:
22
+ - 4
23
+ - 5
24
+ exclude:
25
+ - pg: 17
26
+ partman: 4 # Partman 4.x is not available in PGDG for PG 17
27
+ - pg: 13
28
+ partman: 5 # Partman 5.x is not available in PGDG for PG 13
29
+ name: PostgreSQL ${{ matrix.pg }} - Partman ${{ matrix.partman }} - Ruby ${{ matrix.ruby }} - ${{ matrix.gemfile }}
21
30
  runs-on: ubuntu-latest
22
31
  env: # $BUNDLE_GEMFILE must be set at the job level, so it is set for all steps
23
32
  BUNDLE_GEMFILE: gemfiles/${{ matrix.gemfile }}.gemfile
@@ -28,6 +37,7 @@ jobs:
28
37
  run: docker compose up -d --build
29
38
  env:
30
39
  PGVERSION: ${{ matrix.pg }}
40
+ PARTMAN_VERSION: ${{ matrix.partman }}
31
41
  - name: Setup Ruby using .ruby-version file
32
42
  uses: ruby/setup-ruby@v1
33
43
  with:
data/Dockerfile CHANGED
@@ -1,6 +1,7 @@
1
1
  ARG PGVERSION
2
+ ARG PARTMAN_VERSION
2
3
 
3
- FROM postgres:$PGVERSION-bullseye
4
+ FROM postgres:$PGVERSION-bookworm AS base
4
5
 
5
6
  RUN apt-get update && apt-get install -y curl ca-certificates gnupg lsb-release
6
7
 
@@ -8,4 +9,11 @@ RUN curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor | te
8
9
 
9
10
  RUN echo "deb https://apt-archive.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg-archive main" > /etc/apt/sources.list.d/pgdg.list
10
11
 
11
- RUN apt update && apt-get install -y postgresql-$PG_MAJOR-partman=4.7.4-2.pgdg110+1
12
+ FROM base AS partman-4-branch
13
+ ENV PARTMAN_VERSION=4.7.4-2.pgdg120+1
14
+
15
+ FROM base AS partman-5-branch
16
+ ENV PARTMAN_VERSION=5.2.4-1.pgdg120+1
17
+
18
+ FROM partman-$PARTMAN_VERSION-branch AS final
19
+ RUN apt update && apt-get install -y postgresql-$PG_MAJOR-partman=$PARTMAN_VERSION
data/README.md CHANGED
@@ -144,9 +144,7 @@ Unsafely change the value of an enum type entry.
144
144
  unsafe_rename_enum_value(:enum, "old_value", "new_value")
145
145
  ```
146
146
 
147
- Note:
148
-
149
- Changing an enum value does not issue any long-running scans or acquire locks on usages of the enum type. Therefore multiple queries within a transaction concurrent with the change may see both the old and new values. To highlight these potential pitfalls no `safe_rename_enum_value` equivalent exists. Before modifying an enum type entry you should verify that no concurrently executing queries will attempt to write the old value and that read queries understand the new value.
147
+ > **Note:** Changing an enum value does not issue any long-running scans or acquire locks on usages of the enum type. Therefore multiple queries within a transaction concurrent with the change may see both the old and new values. To highlight these potential pitfalls no `safe_rename_enum_value` equivalent exists. Before modifying an enum type entry you should verify that no concurrently executing queries will attempt to write the old value and that read queries understand the new value.
150
148
 
151
149
  #### safe\_add\_column
152
150
 
@@ -178,7 +176,7 @@ safe_change_column_default :table, :column, -> { "NOW()" }
178
176
  safe_change_column_default :table, :column, -> { "'NOW()'" }
179
177
  ```
180
178
 
181
- Note: On Postgres 11+ adding a column with a constant default value does not rewrite or scan the table (under a lock or otherwise). In that case a migration adding a column with a default should do so in a single operation rather than the two-step `safe_add_column` followed by `safe_change_column_default`. We enforce this best practice with the error `PgHaMigrations::BestPracticeError`, but if your prefer otherwise (or are running in a mixed Postgres version environment), you may opt out by setting `config.prefer_single_step_column_addition_with_default = false` [in your configuration initializer](#configuration).
179
+ > **Note:** On Postgres 11+ adding a column with a constant default value does not rewrite or scan the table (under a lock or otherwise). In that case a migration adding a column with a default should do so in a single operation rather than the two-step `safe_add_column` followed by `safe_change_column_default`. We enforce this best practice with the error `PgHaMigrations::BestPracticeError`, but if your prefer otherwise (or are running in a mixed Postgres version environment), you may opt out by setting `config.prefer_single_step_column_addition_with_default = false` [in your configuration initializer](#configuration).
182
180
 
183
181
  #### safe\_make\_column\_nullable
184
182
 
@@ -189,15 +187,15 @@ safe_make_column_nullable :table, :column
189
187
  ```
190
188
  #### safe\_make\_column\_not\_nullable
191
189
 
192
- Safely make the column not nullable - adds a temporary constraint and uses that constraint to validate no values are null before altering the column, then removes the temporary constraint.
190
+ Safely make the column not nullable. This method uses a `CHECK column IS NOT NULL` constraint to validate no values are null before altering the column. If such a constraint exists already, it is re-used, if it does not, a temporary constraint is added. Whether or not the constraint already existed, the constraint will be validated, if necessary, and removed after the column is marked `NOT NULL`.
193
191
 
194
192
  ```ruby
195
193
  safe_make_column_not_nullable :table, :column
196
194
  ```
197
195
 
198
196
  > **Note:**
199
- > - This method performs a full table scan to validate that no NULL values exist in the column. While no exclusive lock is held for this scan, on large tables the scan may take a long time.
200
- > - The method runs multiple DDL statements non-transactionally. Validating the constraint can fail. In such cases an exception will be raised, and an INVALID constraint will be left on the table.
197
+ > - This method may perform a full table scan to validate that no NULL values exist in the column. While no exclusive lock is held for this scan, on large tables the scan may take a long time.
198
+ > - The method runs multiple DDL statements non-transactionally. Validating the constraint can fail. In such cases an INVALID constraint will be left on the table. Calling `safe_make_column_not_nullable` again is safe.
201
199
 
202
200
  If you want to avoid a full table scan and have already added and validated a suitable CHECK constraint, consider using [`safe_make_column_not_nullable_from_check_constraint`](#safe_make_column_not_nullable_from_check_constraint) instead.
203
201
 
@@ -224,7 +222,7 @@ You should use [`safe_make_column_not_nullable`](#safe_make_column_not_nullable)
224
222
 
225
223
  This method will raise an error if the constraint does not exist, is not validated, or does not strictly enforce non-null values for the column.
226
224
 
227
- > **Note:** We do not attempt to catch all possible proofs of `column IS NOT NULL` by means of an existing constraint; only a constraint with the exact definition `column IS NOT NULL` will be recognized.
225
+ > **Note:** We do not attempt to catch all possible proofs of `column IS NOT NULL` by means of an existing constraint; only a constraint with the exact definition `column IS NOT NULL` will be recognized.
228
226
 
229
227
  #### safe\_add\_index\_on\_empty\_table
230
228
 
@@ -285,11 +283,10 @@ Add a composite index using the `hash` index type with custom name for the paren
285
283
  safe_add_concurrent_partitioned_index :partitioned_table, [:column1, :column2], name: "custom_name_idx", using: :hash
286
284
  ```
287
285
 
288
- Note:
289
-
290
- This method runs multiple DDL statements non-transactionally.
291
- Creating or attaching an index on a child table could fail.
292
- In such cases an exception will be raised, and an `INVALID` index will be left on the parent table.
286
+ > **Note:**
287
+ > This method runs multiple DDL statements non-transactionally.
288
+ > Creating or attaching an index on a child table could fail.
289
+ > In such cases an exception will be raised, and an `INVALID` index will be left on the parent table.
293
290
 
294
291
  #### safe\_add\_unvalidated\_check\_constraint
295
292
 
@@ -383,7 +380,7 @@ The rest are keyword args with the following mappings:
383
380
  - `premake` -> `p_premake`. Required: `false`. Partman defaults to `4`.
384
381
  - `start_partition` -> `p_start_partition`. Required: `false`. Partman defaults to the current timestamp.
385
382
 
386
- Note that we have chosen to require PostgreSQL 11+ and hardcode `p_type` to `native` for simplicity, as previous PostgreSQL versions are end-of-life.
383
+ > **Note:** We have chosen to require PostgreSQL 11+ and hardcode `p_type` to `native` (`range` in the case of Partman 5) for simplicity, as previous PostgreSQL versions are end-of-life.
387
384
 
388
385
  Additionally, this method allows you to configure a subset of attributes on the record stored in the [part\_config](https://github.com/pgpartman/pg_partman/blob/master/doc/pg_partman.md#tables) table.
389
386
  These options are delegated to the `unsafe_partman_update_config` method to update the record:
@@ -400,7 +397,7 @@ safe_create_partitioned_table :table, type: :range, partition_key: :created_at d
400
397
  t.timestamps null: false
401
398
  end
402
399
 
403
- safe_partman_create_parent :table, partition_key: :created_at, interval: "weekly"
400
+ safe_partman_create_parent :table, partition_key: :created_at, interval: "1 week"
404
401
  ```
405
402
 
406
403
  With custom overrides:
@@ -418,7 +415,7 @@ end
418
415
 
419
416
  safe_partman_create_parent :table,
420
417
  partition_key: :created_at,
421
- interval: "weekly",
418
+ interval: "1 week",
422
419
  template_table: :table_template,
423
420
  premake: 10,
424
421
  start_partition: Time.current + 1.month,
@@ -442,7 +439,7 @@ Allowed keyword args:
442
439
  - `retention`
443
440
  - `retention_keep_table`
444
441
 
445
- Note that we detect if the value of `inherit_privileges` is changing and will automatically call `safe_partman_reapply_privileges` to ensure permissions are propagated to existing child partitions.
442
+ > **Note:** If `inherit_privileges` will change then `safe_partman_reapply_privileges` will be automatically called to ensure permissions are propagated to existing child partitions.
446
443
 
447
444
  ```ruby
448
445
  safe_partman_update_config :table,
@@ -471,6 +468,29 @@ If your partitioned table is configured with `inherit_privileges` set to `true`,
471
468
  safe_partman_reapply_privileges :table
472
469
  ```
473
470
 
471
+ #### unsafe\_partman\_standardize\_partition\_naming
472
+
473
+ This method provides functionality to standardize existing Partman 4 tables such that naming is consistent with Partman 5.
474
+ The logic follows the guidelines in the [Partman upgrade docs](https://github.com/pgpartman/pg_partman/blob/v5.2.4/doc/pg_partman_5.0.1_upgrade.md).
475
+
476
+ Technically, only `weekly` and `quarterly` partitioned tables _need_ to be standardized prior to the upgrade.
477
+ _However_, Partman 5 changes the default [datetime_string](https://github.com/pgpartman/pg_partman/blob/v5.2.4/sql/functions/calculate_time_partition_info.sql#L13-L17) that is used for _all_ intervals (`YYYYMMDD` and `YYYYMMDD_HH24MISS`).
478
+ Compare that to the Partman 4 logic for [datetime_string](https://github.com/pgpartman/pg_partman/blob/v4.7.4/sql/functions/create_parent.sql#L434-L459).
479
+ So, this method supports standardization for _all_ Partman 4 intervals.
480
+
481
+ > **Note:** This method is safe from a database perspective, but is only safe from an application perspective if child tables are not directly referenced (child tables are renamed during this operation)
482
+
483
+ ```ruby
484
+ unsafe_partman_standardize_partition_naming :table
485
+ ```
486
+
487
+ This method uses a default statement timeout of 1 second.
488
+ If the target table has many partitions (hundreds of thousands), you may need to increase the statement timeout for the operation to succeed.
489
+
490
+ ```ruby
491
+ unsafe_partman_standardize_partition_naming :table, statement_timeout: 2
492
+ ```
493
+
474
494
  ### Utilities
475
495
 
476
496
  #### safely\_acquire\_lock\_for\_table
@@ -505,10 +525,8 @@ safely_acquire_lock_for_table(:table_a, :table_b, mode: :exclusive) do
505
525
  end
506
526
  ```
507
527
 
508
- Note:
509
-
510
- We enforce that only one set of tables can be locked at a time.
511
- Attempting to acquire a nested lock on a different set of tables will result in an error.
528
+ > **Note:** We enforce that only one set of tables can be locked at a time.
529
+ > Attempting to acquire a nested lock on a different set of tables will result in an error.
512
530
 
513
531
  #### adjust\_lock\_timeout
514
532
 
@@ -571,6 +589,7 @@ end
571
589
  - `prefer_single_step_column_addition_with_default`: If true, raise an error when adding a column and separately setting a constant default value for that column in the same migration. Default: `true`
572
590
  - `allow_force_create_table`: If false, the `force: true` option to ActiveRecord's `create_table` method is disallowed. Default: `false`
573
591
  - `infer_primary_key_on_partitioned_tables`: If true, the primary key for partitioned tables will be inferred on PostgreSQL 11+ databases (identifier column + partition key columns). Default: `true`
592
+ - `partman_5_compatibility_mode`: If true, `safe_partman_create_parent` will raise an error if the user provides an interval that is [not supported by Partman 5](https://github.com/pgpartman/pg_partman/blob/v5.2.4/sql/functions/create_parent.sql#L86-L96). If the interval is supported, the method will ensure table name suffixes match the Partman 5 format (`YYYYMMDD`, `YYYYMMDD_HTH24MISS`). Default: `false`
574
593
 
575
594
  ### Rake Tasks
576
595
 
@@ -606,7 +625,7 @@ To install this gem onto your local machine, run `bundle exec rake install`.
606
625
 
607
626
  To release a new version, update the version number in `version.rb`, commit the change, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
608
627
 
609
- Note: if while releasing the gem you get the error ``Your rubygems.org credentials aren't set. Run `gem push` to set them.`` you can more simply run `gem signin`.
628
+ > **Note:** If while releasing the gem you get the error ``Your rubygems.org credentials aren't set. Run `gem push` to set them.`` you can more simply run `gem signin`.
610
629
 
611
630
  ## Contributing
612
631
 
data/docker-compose.yml CHANGED
@@ -4,7 +4,8 @@ services:
4
4
  build:
5
5
  context: .
6
6
  args:
7
- - PGVERSION=${PGVERSION:-16}
7
+ - PGVERSION=${PGVERSION:-17}
8
+ - PARTMAN_VERSION=${PARTMAN_VERSION:-5}
8
9
  ports:
9
10
  - "5432:5432"
10
11
  environment:
@@ -1 +1,8 @@
1
- PgHaMigrations::CheckConstraint = Struct.new(:name, :definition, :validated)
1
+ PgHaMigrations::CheckConstraint = Struct.new(:name, :definition, :validated) do
2
+ def initialize(name, definition, validated)
3
+ # pg_get_constraintdef includes NOT VALID in the definition,
4
+ # but we return that as a separate attribute.
5
+ definition = definition&.gsub(/ NOT VALID\Z/, "")
6
+ super(name, definition, validated)
7
+ end
8
+ end
@@ -0,0 +1,35 @@
1
+ module PgHaMigrations
2
+ class Extension
3
+ attr_reader :name, :schema, :version
4
+
5
+ def initialize(name)
6
+ @name = name
7
+
8
+ @schema, @version = ActiveRecord::Base.connection.select_rows(<<~SQL).first
9
+ SELECT nspname, extversion
10
+ FROM pg_namespace JOIN pg_extension
11
+ ON pg_namespace.oid = pg_extension.extnamespace
12
+ WHERE pg_extension.extname = #{ActiveRecord::Base.connection.quote(name)}
13
+ LIMIT 1
14
+ SQL
15
+ end
16
+
17
+ def quoted_schema
18
+ return unless schema
19
+
20
+ PG::Connection.quote_ident(schema)
21
+ end
22
+
23
+ def major_version
24
+ return unless version
25
+
26
+ Gem::Version.new(version)
27
+ .segments
28
+ .first
29
+ end
30
+
31
+ def installed?
32
+ !!schema && !!version
33
+ end
34
+ end
35
+ end
@@ -1,11 +1,73 @@
1
1
  # This is an internal class that is not meant to be used directly
2
2
  class PgHaMigrations::PartmanConfig < ActiveRecord::Base
3
+ SUPPORTED_PARTITION_TYPES = %w[native range]
4
+
5
+ delegate :connection, to: :class
6
+
3
7
  self.primary_key = :parent_table
4
8
 
5
- # This method is called by unsafe_partman_update_config to set the fully
6
- # qualified table name, as partman is often installed in a schema that
7
- # is not included the application's search path
8
- def self.schema=(schema)
9
- self.table_name = "#{schema}.part_config"
9
+ def self.find(parent_table, partman_extension:)
10
+ unless partman_extension.installed?
11
+ raise PgHaMigrations::MissingExtensionError, "The pg_partman extension is not installed"
12
+ end
13
+
14
+ self.table_name = "#{partman_extension.quoted_schema}.part_config"
15
+
16
+ super(parent_table)
17
+ end
18
+
19
+ # The actual column type is TEXT and the value is determined by the
20
+ # intervalstyle in Postgres at the time create_parent is called.
21
+ # Rails hard codes this config when it builds connections for ease
22
+ # of parsing by ActiveSupport::Duration.parse. So in theory, we
23
+ # really only need to do the interval casting, but we're doing the
24
+ # SET LOCAL to be absolutely sure intervalstyle is correct.
25
+ #
26
+ # https://github.com/rails/rails/blob/v8.0.3/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L979-L980
27
+ def partition_interval_iso_8601
28
+ transaction do
29
+ connection.execute("SET LOCAL intervalstyle TO 'iso_8601'")
30
+ connection.select_value("SELECT #{connection.quote(partition_interval)}::interval")
31
+ end
32
+ end
33
+
34
+ def partition_rename_adapter
35
+ unless SUPPORTED_PARTITION_TYPES.include?(partition_type)
36
+ raise PgHaMigrations::InvalidPartmanConfigError,
37
+ "Expected partition_type to be in #{SUPPORTED_PARTITION_TYPES.inspect} " \
38
+ "but received #{partition_type.inspect}"
39
+ end
40
+
41
+ duration = ActiveSupport::Duration.parse(partition_interval_iso_8601)
42
+
43
+ if duration.parts.size != 1
44
+ raise PgHaMigrations::InvalidPartmanConfigError,
45
+ "Partition renaming for complex partition_interval #{duration.iso8601.inspect} not supported"
46
+ end
47
+
48
+ # Quarterly and weekly have special meaning in Partman 4 with
49
+ # specific datetime strings that need to be handled separately.
50
+ #
51
+ # The intervals "1 week" and "3 months" will not match the first
52
+ # two conditionals and will fallthrough to standard adapters below.
53
+ if duration == 1.week && datetime_string == "IYYY\"w\"IW"
54
+ PgHaMigrations::WeeklyPartmanRenameAdapter.new(self)
55
+ elsif duration == 3.months && datetime_string == "YYYY\"q\"Q"
56
+ PgHaMigrations::QuarterlyPartmanRenameAdapter.new(self)
57
+ elsif duration >= 1.year
58
+ PgHaMigrations::YearToForeverPartmanRenameAdapter.new(self)
59
+ elsif duration >= 1.month && duration < 1.year
60
+ PgHaMigrations::MonthToYearPartmanRenameAdapter.new(self)
61
+ elsif duration >= 1.day && duration < 1.month
62
+ PgHaMigrations::DayToMonthPartmanRenameAdapter.new(self)
63
+ elsif duration >= 1.minute && duration < 1.day
64
+ PgHaMigrations::MinuteToDayPartmanRenameAdapter.new(self)
65
+ elsif duration >= 1.second && duration < 1.minute
66
+ PgHaMigrations::SecondToMinutePartmanRenameAdapter.new(self)
67
+ else
68
+ raise PgHaMigrations::InvalidPartmanConfigError,
69
+ "Expected partition_interval to be greater than 1 second " \
70
+ "but received #{duration.iso8601.inspect}"
71
+ end
10
72
  end
11
73
  end
@@ -0,0 +1,209 @@
1
+ module PgHaMigrations
2
+ class AbstractPartmanRenameAdapter
3
+ def initialize(part_config)
4
+ if part_config.datetime_string != source_datetime_string
5
+ raise PgHaMigrations::InvalidPartmanConfigError,
6
+ "Expected datetime_string to be #{source_datetime_string.inspect} " \
7
+ "but received #{part_config.datetime_string.inspect}"
8
+ end
9
+ end
10
+
11
+ def alter_table_sql(partitions)
12
+ sql = partitions.filter_map do |partition|
13
+ next if partition.name =~ /\A.+_default\z/
14
+
15
+ if partition.name !~ source_name_suffix_pattern
16
+ raise PgHaMigrations::InvalidIdentifierError,
17
+ "Expected #{partition.name.inspect} to match #{source_name_suffix_pattern.inspect}"
18
+ end
19
+
20
+ begin
21
+ "ALTER TABLE #{partition.fully_qualified_name} RENAME TO #{target_table_name(partition.name)};"
22
+ rescue Date::Error
23
+ raise PgHaMigrations::InvalidIdentifierError,
24
+ "Expected #{partition.name.inspect} suffix to be a parseable DateTime"
25
+ end
26
+ end.join("\n")
27
+
28
+ # This wraps the SQL in an anonymous function such that
29
+ # the statement timeout would apply to the entire batch of
30
+ # statements instead of each individual statement
31
+ "DO $$ BEGIN #{sql} END; $$;"
32
+ end
33
+
34
+ def target_table_name(table_name)
35
+ raise "#{__method__} should be implemented in subclass"
36
+ end
37
+
38
+ def source_datetime_string
39
+ raise "#{__method__} should be implemented in subclass"
40
+ end
41
+
42
+ def source_name_suffix_pattern
43
+ raise "#{__method__} should be implemented in subclass"
44
+ end
45
+
46
+ def target_datetime_string
47
+ raise "#{__method__} should be implemented in subclass"
48
+ end
49
+ end
50
+
51
+ class YearToForeverPartmanRenameAdapter < AbstractPartmanRenameAdapter
52
+ def target_table_name(table_name)
53
+ table_name + "0101"
54
+ end
55
+
56
+ def source_datetime_string
57
+ "YYYY"
58
+ end
59
+
60
+ def source_name_suffix_pattern
61
+ /\A.+_p\d{4}\z/
62
+ end
63
+
64
+ def target_datetime_string
65
+ "YYYYMMDD"
66
+ end
67
+ end
68
+
69
+ class QuarterlyPartmanRenameAdapter < AbstractPartmanRenameAdapter
70
+ QUARTER_MONTH_MAPPING = {
71
+ "1" => "01",
72
+ "2" => "04",
73
+ "3" => "07",
74
+ "4" => "10",
75
+ }
76
+
77
+ def target_table_name(table_name)
78
+ base_name = table_name[0...-6]
79
+
80
+ year = table_name.last(6).first(4)
81
+
82
+ month = QUARTER_MONTH_MAPPING.fetch(table_name.last(1))
83
+
84
+ base_name + year + month + "01"
85
+ end
86
+
87
+ def source_datetime_string
88
+ "YYYY\"q\"Q"
89
+ end
90
+
91
+ def source_name_suffix_pattern
92
+ /\A.+_p\d{4}q(1|2|3|4)\z/
93
+ end
94
+
95
+ def target_datetime_string
96
+ "YYYYMMDD"
97
+ end
98
+ end
99
+
100
+ class MonthToYearPartmanRenameAdapter < AbstractPartmanRenameAdapter
101
+ def target_table_name(table_name)
102
+ base_name = table_name[0...-7]
103
+
104
+ partition_datetime = DateTime.strptime(table_name.last(7), "%Y_%m")
105
+
106
+ base_name + partition_datetime.strftime("%Y%m%d")
107
+ end
108
+
109
+ def source_datetime_string
110
+ "YYYY_MM"
111
+ end
112
+
113
+ def source_name_suffix_pattern
114
+ /\A.+_p\d{4}_\d{2}\z/
115
+ end
116
+
117
+ def target_datetime_string
118
+ "YYYYMMDD"
119
+ end
120
+ end
121
+
122
+ class WeeklyPartmanRenameAdapter < AbstractPartmanRenameAdapter
123
+ def target_table_name(table_name)
124
+ base_name = table_name[0...-7]
125
+
126
+ partition_datetime = DateTime.strptime(table_name.last(7), "%Gw%V")
127
+
128
+ base_name + partition_datetime.strftime("%Y%m%d")
129
+ end
130
+
131
+ def source_datetime_string
132
+ "IYYY\"w\"IW"
133
+ end
134
+
135
+ def source_name_suffix_pattern
136
+ /\A.+_p\d{4}w\d{2}\z/
137
+ end
138
+
139
+ def target_datetime_string
140
+ "YYYYMMDD"
141
+ end
142
+ end
143
+
144
+ class DayToMonthPartmanRenameAdapter < AbstractPartmanRenameAdapter
145
+ def target_table_name(table_name)
146
+ base_name = table_name[0...-10]
147
+
148
+ partition_datetime = DateTime.strptime(table_name.last(10), "%Y_%m_%d")
149
+
150
+ base_name + partition_datetime.strftime("%Y%m%d")
151
+ end
152
+
153
+ def source_datetime_string
154
+ "YYYY_MM_DD"
155
+ end
156
+
157
+ def source_name_suffix_pattern
158
+ /\A.+_p\d{4}_\d{2}_\d{2}\z/
159
+ end
160
+
161
+ def target_datetime_string
162
+ "YYYYMMDD"
163
+ end
164
+ end
165
+
166
+ class MinuteToDayPartmanRenameAdapter < AbstractPartmanRenameAdapter
167
+ def target_table_name(table_name)
168
+ base_name = table_name[0...-15]
169
+
170
+ partition_datetime = DateTime.strptime(table_name.last(15), "%Y_%m_%d_%H%M")
171
+
172
+ base_name + partition_datetime.strftime("%Y%m%d_%H%M%S")
173
+ end
174
+
175
+ def source_datetime_string
176
+ "YYYY_MM_DD_HH24MI"
177
+ end
178
+
179
+ def source_name_suffix_pattern
180
+ /\A.+_p\d{4}_\d{2}_\d{2}_\d{4}\z/
181
+ end
182
+
183
+ def target_datetime_string
184
+ "YYYYMMDD_HH24MISS"
185
+ end
186
+ end
187
+
188
+ class SecondToMinutePartmanRenameAdapter < AbstractPartmanRenameAdapter
189
+ def target_table_name(table_name)
190
+ base_name = table_name[0...-17]
191
+
192
+ partition_datetime = DateTime.strptime(table_name.last(17), "%Y_%m_%d_%H%M%S")
193
+
194
+ base_name + partition_datetime.strftime("%Y%m%d_%H%M%S")
195
+ end
196
+
197
+ def source_datetime_string
198
+ "YYYY_MM_DD_HH24MISS"
199
+ end
200
+
201
+ def source_name_suffix_pattern
202
+ /\A.+_p\d{4}_\d{2}_\d{2}_\d{6}\z/
203
+ end
204
+
205
+ def target_datetime_string
206
+ "YYYYMMDD_HH24MISS"
207
+ end
208
+ end
209
+ end
@@ -139,6 +139,30 @@ module PgHaMigrations
139
139
  end
140
140
  end
141
141
 
142
+ class PartmanTable < Table
143
+ IDENTIFIER_REGEX = /^[a-z_][a-z_\d]*$/
144
+
145
+ def initialize(name, schema, mode=nil)
146
+ if name !~ IDENTIFIER_REGEX
147
+ raise InvalidIdentifierError, "Partman requires table names to be lowercase with underscores"
148
+ end
149
+
150
+ if schema !~ IDENTIFIER_REGEX
151
+ raise InvalidIdentifierError, "Partman requires schema names to be lowercase with underscores"
152
+ end
153
+
154
+ super
155
+ end
156
+
157
+ def fully_qualified_name
158
+ "#{schema}.#{name}"
159
+ end
160
+
161
+ def part_config(partman_extension:)
162
+ PgHaMigrations::PartmanConfig.find(fully_qualified_name, partman_extension: partman_extension)
163
+ end
164
+ end
165
+
142
166
  class Index < Relation
143
167
  MAX_NAME_SIZE = 63 # bytes
144
168
 
@@ -3,6 +3,10 @@ module PgHaMigrations::SafeStatements
3
3
  @safe_added_columns_without_default_value ||= []
4
4
  end
5
5
 
6
+ def partman_extension
7
+ @partman_extension ||= PgHaMigrations::Extension.new("pg_partman")
8
+ end
9
+
6
10
  def safe_create_table(table, **options, &block)
7
11
  if options[:force]
8
12
  raise PgHaMigrations::UnsafeMigrationError.new(":force is NOT SAFE! Explicitly call unsafe_drop_table first if you want to recreate an existing table")
@@ -139,23 +143,39 @@ module PgHaMigrations::SafeStatements
139
143
  end
140
144
 
141
145
  validated_table = PgHaMigrations::Table.from_table_name(table)
142
- tmp_constraint_name = "tmp_not_null_constraint_#{OpenSSL::Digest::SHA256.hexdigest(column.to_s).first(7)}"
146
+ quoted_column_name = connection.quote_column_name(column)
147
+ column_str = column.to_s
143
148
 
144
- if validated_table.check_constraints.any? { |c| c.name == tmp_constraint_name }
145
- raise PgHaMigrations::InvalidMigrationError, "A constraint #{tmp_constraint_name.inspect} already exists. " \
146
- "This implies that a previous invocation of this method failed and left behind a temporary constraint. " \
147
- "Please drop the constraint before attempting to run this method again."
148
- end
149
+ # First, look for existing constraints that match the IS NOT NULL pattern for this column
150
+ existing_constraint = validated_table.check_constraints.select do |c|
151
+ c.definition =~ /\ACHECK \(*(#{Regexp.escape(column_str)}|#{Regexp.escape(quoted_column_name)}) IS NOT NULL\)*\Z/i
152
+ end.first
153
+
154
+ constraint_name = nil
155
+ if existing_constraint
156
+ if existing_constraint.validated
157
+ say "Found existing validated constraint #{existing_constraint.inspect} for column #{column_str}, using it directly"
158
+ else
159
+ say "Found existing unvalidated constraint #{existing_constraint.inspect} for column #{column_str}, validating it first"
160
+ safe_validate_check_constraint(table, name: existing_constraint.name)
161
+ end
162
+ constraint_name = existing_constraint.name
163
+ else
164
+ # Create a temporary constraint if no matching constraints exist
165
+ constraint_name = "tmp_not_null_constraint_#{OpenSSL::Digest::SHA256.hexdigest(column.to_s).first(7)}"
149
166
 
150
- safe_add_unvalidated_check_constraint(table, "#{connection.quote_column_name(column)} IS NOT NULL", name: tmp_constraint_name)
151
- safe_validate_check_constraint(table, name: tmp_constraint_name)
167
+ safe_add_unvalidated_check_constraint(table, "#{quoted_column_name} IS NOT NULL", name: constraint_name)
168
+ safe_validate_check_constraint(table, name: constraint_name)
169
+ end
152
170
 
153
171
  # "Ordinarily this is checked during the ALTER TABLE by scanning the entire table; however, if a
154
172
  # valid CHECK constraint is found which proves no NULL can exist, then the table scan is
155
173
  # skipped."
156
174
  # See: https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-DESC-SET-DROP-NOT-NULL
157
175
  unsafe_make_column_not_nullable(table, column)
158
- unsafe_remove_constraint(table, name: tmp_constraint_name)
176
+
177
+ # Always drop the constraint at the end, whether it was existing or temporary
178
+ unsafe_remove_constraint(table, name: constraint_name)
159
179
  end
160
180
 
161
181
  # This method is a variant of `safe_make_column_not_nullable` that is expected to always be fast;
@@ -458,6 +478,18 @@ module PgHaMigrations::SafeStatements
458
478
  raise PgHaMigrations::InvalidMigrationError, "Native partitioning with partman not supported on Postgres databases before version 11"
459
479
  end
460
480
 
481
+ raise PgHaMigrations::MissingExtensionError, "The pg_partman extension is not installed" unless partman_extension.installed?
482
+
483
+ if partman_extension.major_version >= 5 || PgHaMigrations.config.partman_5_compatibility_mode
484
+ if PgHaMigrations::PARTMAN_UNSUPPORTED_INTERVALS.include?(interval)
485
+ raise PgHaMigrations::InvalidMigrationError,
486
+ "Special partition interval values (#{interval}) are no longer supported. " \
487
+ "Please use a supported interval time value from core PostgreSQL " \
488
+ "#{(partman_extension.major_version < 5 ? "or turn partman 5 compatibility mode off " : "")}" \
489
+ "(https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT)"
490
+ end
491
+ end
492
+
461
493
  formatted_start_partition = nil
462
494
 
463
495
  if start_partition.present?
@@ -472,16 +504,22 @@ module PgHaMigrations::SafeStatements
472
504
  end
473
505
  end
474
506
 
507
+ validated_table = PgHaMigrations::PartmanTable.from_table_name(table)
508
+ validated_template_table = template_table ? PgHaMigrations::PartmanTable.from_table_name(template_table) : nil
509
+
475
510
  create_parent_options = {
476
- parent_table: _fully_qualified_table_name_for_partman(table),
477
- template_table: template_table ? _fully_qualified_table_name_for_partman(template_table) : nil,
511
+ parent_table: validated_table.fully_qualified_name,
512
+ template_table: validated_template_table&.fully_qualified_name,
478
513
  control: partition_key,
479
- type: "native",
480
514
  interval: interval,
481
515
  premake: premake,
482
516
  start_partition: formatted_start_partition,
483
517
  }.compact
484
518
 
519
+ if partman_extension.major_version < 5
520
+ create_parent_options[:type] = "native"
521
+ end
522
+
485
523
  create_parent_sql = create_parent_options.map { |k, v| "p_#{k} := #{connection.quote(v)}" }.join(", ")
486
524
 
487
525
  log_message = "partman_create_parent(#{table.inspect}, " \
@@ -492,7 +530,7 @@ module PgHaMigrations::SafeStatements
492
530
  "template_table: #{template_table.inspect})"
493
531
 
494
532
  say_with_time(log_message) do
495
- connection.execute("SELECT #{_quoted_partman_schema}.create_parent(#{create_parent_sql})")
533
+ connection.execute("SELECT #{partman_extension.quoted_schema}.create_parent(#{create_parent_sql})")
496
534
  end
497
535
 
498
536
  update_config_options = {
@@ -503,6 +541,10 @@ module PgHaMigrations::SafeStatements
503
541
  }.compact
504
542
 
505
543
  unsafe_partman_update_config(table, **update_config_options)
544
+
545
+ if PgHaMigrations.config.partman_5_compatibility_mode && partman_extension.major_version < 5
546
+ unsafe_partman_standardize_partition_naming(table)
547
+ end
506
548
  end
507
549
 
508
550
  def safe_partman_update_config(table, **options)
@@ -514,32 +556,13 @@ module PgHaMigrations::SafeStatements
514
556
  end
515
557
 
516
558
  def safe_partman_reapply_privileges(table)
517
- say_with_time "partman_reapply_privileges(#{table.inspect})" do
518
- connection.execute("SELECT #{_quoted_partman_schema}.reapply_privileges('#{_fully_qualified_table_name_for_partman(table)}')")
519
- end
520
- end
521
-
522
- def _quoted_partman_schema
523
- schema = connection.select_value(<<~SQL)
524
- SELECT nspname
525
- FROM pg_namespace JOIN pg_extension
526
- ON pg_namespace.oid = pg_extension.extnamespace
527
- WHERE pg_extension.extname = 'pg_partman'
528
- SQL
559
+ raise PgHaMigrations::MissingExtensionError, "The pg_partman extension is not installed" unless partman_extension.installed?
529
560
 
530
- raise PgHaMigrations::InvalidMigrationError, "The pg_partman extension is not installed" unless schema.present?
561
+ validated_table = PgHaMigrations::PartmanTable.from_table_name(table)
531
562
 
532
- connection.quote_schema_name(schema)
533
- end
534
-
535
- def _fully_qualified_table_name_for_partman(table)
536
- table = PgHaMigrations::Table.from_table_name(table)
537
-
538
- [table.schema, table.name].each do |identifier|
539
- if identifier.to_s !~ /^[a-z_][a-z_\d]*$/
540
- raise PgHaMigrations::InvalidMigrationError, "Partman requires schema / table names to be lowercase with underscores"
541
- end
542
- end.join(".")
563
+ say_with_time "partman_reapply_privileges(#{table.inspect})" do
564
+ connection.execute("SELECT #{partman_extension.quoted_schema}.reapply_privileges('#{validated_table.fully_qualified_name}')")
565
+ end
543
566
  end
544
567
 
545
568
  def _per_migration_caller
@@ -196,21 +196,55 @@ module PgHaMigrations::UnsafeStatements
196
196
 
197
197
  raise ArgumentError, "Unrecognized argument(s): #{invalid_options}" unless invalid_options.empty?
198
198
 
199
- PgHaMigrations::PartmanConfig.schema = _quoted_partman_schema
199
+ part_config = PgHaMigrations::PartmanTable
200
+ .from_table_name(table)
201
+ .part_config(partman_extension: partman_extension)
200
202
 
201
- config = PgHaMigrations::PartmanConfig.find(_fully_qualified_table_name_for_partman(table))
203
+ part_config.assign_attributes(**options)
202
204
 
203
- config.assign_attributes(**options)
204
-
205
- inherit_privileges_changed = config.inherit_privileges_changed?
205
+ inherit_privileges_changed = part_config.inherit_privileges_changed?
206
206
 
207
207
  say_with_time "partman_update_config(#{table.inspect}, #{options.map { |k,v| "#{k}: #{v.inspect}" }.join(", ")})" do
208
- config.save!
208
+ part_config.save!
209
209
  end
210
210
 
211
211
  safe_partman_reapply_privileges(table) if inherit_privileges_changed
212
212
  end
213
213
 
214
+ def unsafe_partman_standardize_partition_naming(table, statement_timeout: 1)
215
+ raise PgHaMigrations::MissingExtensionError, "The pg_partman extension is not installed" unless partman_extension.installed?
216
+ raise PgHaMigrations::InvalidMigrationError, "This method is only available for pg_partman major version 4" unless partman_extension.major_version == 4
217
+
218
+ validated_table = PgHaMigrations::PartmanTable.from_table_name(table)
219
+
220
+ part_config = validated_table.part_config(partman_extension: partman_extension)
221
+ partition_rename_adapter = part_config.partition_rename_adapter
222
+
223
+ before_automatic_maintenance = part_config.automatic_maintenance
224
+
225
+ part_config.update!(automatic_maintenance: "off") if before_automatic_maintenance == "on"
226
+
227
+ begin
228
+ partitions = validated_table.partitions
229
+ alter_table_sql = partition_rename_adapter.alter_table_sql(partitions)
230
+
231
+ log_message = "partman_standardize_partition_naming(" \
232
+ "#{table.inspect}, statement_timeout: #{statement_timeout}) - " \
233
+ "Renaming #{partitions.size - 1} partition(s)" # excluding default partition
234
+
235
+ safely_acquire_lock_for_table(table) do
236
+ adjust_statement_timeout(statement_timeout) do
237
+ say_with_time(log_message) do
238
+ part_config.update!(datetime_string: partition_rename_adapter.target_datetime_string)
239
+ connection.execute(alter_table_sql)
240
+ end
241
+ end
242
+ end
243
+ ensure
244
+ part_config.reload.update!(automatic_maintenance: "on") if before_automatic_maintenance == "on"
245
+ end
246
+ end
247
+
214
248
  ruby2_keywords def execute_ancestor_statement(method_name, *args, &block)
215
249
  # Dispatching here is a bit complicated: we need to execute the method
216
250
  # belonging to the first member of the inheritance chain (besides
@@ -1,3 +1,3 @@
1
1
  module PgHaMigrations
2
- VERSION = "2.0.0"
2
+ VERSION = "2.1.0"
3
3
  end
@@ -14,6 +14,7 @@ module PgHaMigrations
14
14
  :allow_force_create_table,
15
15
  :prefer_single_step_column_addition_with_default,
16
16
  :infer_primary_key_on_partitioned_tables,
17
+ :partman_5_compatibility_mode,
17
18
  )
18
19
 
19
20
  def self.config
@@ -22,7 +23,8 @@ module PgHaMigrations
22
23
  true,
23
24
  false,
24
25
  true,
25
- true
26
+ true,
27
+ false,
26
28
  )
27
29
  end
28
30
 
@@ -44,6 +46,17 @@ module PgHaMigrations
44
46
  retention_keep_table
45
47
  ]
46
48
 
49
+ PARTMAN_UNSUPPORTED_INTERVALS = %w[
50
+ yearly
51
+ quarterly
52
+ monthly
53
+ weekly
54
+ daily
55
+ hourly
56
+ half-hour
57
+ quarter-hour
58
+ ]
59
+
47
60
  # Safe versus unsafe in this context specifically means the following:
48
61
  # - Safe operations will not block for long periods of time.
49
62
  # - Unsafe operations _may_ block for long periods of time.
@@ -69,13 +82,24 @@ module PgHaMigrations
69
82
  # Some methods need to inspect the attributes of a table. In such cases,
70
83
  # this error will be raised if the table does not exist
71
84
  UndefinedTableError = Class.new(StandardError)
85
+
86
+ # Some methods rely on certain extensions being installed (e.g. partman).
87
+ MissingExtensionError = Class.new(StandardError)
88
+
89
+ # Some methods require table / schema names to be in a specific format.
90
+ InvalidIdentifierError = Class.new(StandardError)
91
+
92
+ # Some methods require the part_config entry to be in a specific state.
93
+ InvalidPartmanConfigError = Class.new(StandardError)
72
94
  end
73
95
 
74
96
  require "pg_ha_migrations/constraint"
97
+ require "pg_ha_migrations/extension"
75
98
  require "pg_ha_migrations/relation"
76
99
  require "pg_ha_migrations/blocking_database_transactions"
77
100
  require "pg_ha_migrations/blocking_database_transactions_reporter"
78
101
  require "pg_ha_migrations/partman_config"
102
+ require "pg_ha_migrations/partman_rename_adapter"
79
103
  require "pg_ha_migrations/lock_mode"
80
104
  require "pg_ha_migrations/unsafe_statements"
81
105
  require "pg_ha_migrations/safe_statements"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pg_ha_migrations
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.0
4
+ version: 2.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - celeen
@@ -13,7 +13,7 @@ authors:
13
13
  - redneckbeard
14
14
  bindir: exe
15
15
  cert_chain: []
16
- date: 2025-04-17 00:00:00.000000000 Z
16
+ date: 2025-10-27 00:00:00.000000000 Z
17
17
  dependencies:
18
18
  - !ruby/object:Gem::Dependency
19
19
  name: rake
@@ -194,11 +194,13 @@ files:
194
194
  - lib/pg_ha_migrations/blocking_database_transactions_reporter.rb
195
195
  - lib/pg_ha_migrations/constraint.rb
196
196
  - lib/pg_ha_migrations/dependent_objects_checks.rb
197
+ - lib/pg_ha_migrations/extension.rb
197
198
  - lib/pg_ha_migrations/hacks/add_index_on_only.rb
198
199
  - lib/pg_ha_migrations/hacks/cleanup_unnecessary_output.rb
199
200
  - lib/pg_ha_migrations/hacks/disable_ddl_transaction.rb
200
201
  - lib/pg_ha_migrations/lock_mode.rb
201
202
  - lib/pg_ha_migrations/partman_config.rb
203
+ - lib/pg_ha_migrations/partman_rename_adapter.rb
202
204
  - lib/pg_ha_migrations/railtie.rb
203
205
  - lib/pg_ha_migrations/relation.rb
204
206
  - lib/pg_ha_migrations/safe_statements.rb