pg_party 1.3.0 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ee226c4523ca2b7099761ad517be99938078f7977281d88920beff47396f0c51
4
- data.tar.gz: 2843ee1eb9fbe92e9998d8da0a3a20dfa989fa6c6dfcf3a261b80bac0caa47da
3
+ metadata.gz: 4e3cebeaa10645ddde63782d4ec7c7ae4f053cbcf1070662928585ae48a2b347
4
+ data.tar.gz: cc11305b51a54b8aaa55adf05c9ff2f05f1dee642f751945356ffe3531e1ed9e
5
5
  SHA512:
6
- metadata.gz: 6cb4e5e42f08dcca3ec065c5d73656b6075e5a647d60722607cb4a9ff3cfeee24a94f4048544f3442284c816e7e37fe196579b9037e1ddacaaf1136e440efd4a
7
- data.tar.gz: fdb593463dace1bdcfd844cac214cb6e6489dc7cd533226b90d11e506955ff48ad959fa80320423f3485637f89e3f6dbaeec5812ec0c1f640eb2d620e91de6dc
6
+ metadata.gz: 6685b47219e38972531b0b846f09e1936eef95bcdbfbacb3275931026ce096ca8dadd79ff4a53efb1241649ef62e385c3e63185eaf4e01783ea1e9a153adc898
7
+ data.tar.gz: fc372ed2b040e5e275092e003306467d67f05693db203b50a3e75133e84cfa9a2c22524f892ce3a83e7d05c38abff491323c8f1778de1da3d8aa69afa05892eb
data/README.md CHANGED
@@ -61,6 +61,23 @@ These values can be accessed and set via `PgParty.config` and `PgParty.configure
61
61
  - `schema_exclude_partitions`
62
62
  - Whether to exclude child partitions in `rake db:structure:dump`
63
63
  - Default: `true`
64
+ - `create_template_tables`
65
+ - Whether to create template tables by default. Use the `template:` option when creating partitioned tables to override this default.
66
+ - Default: `true`
67
+ - `create_with_primary_key`
68
+ - Whether to add primary key constraints to partitioned (parent) tables by default.
69
+ * This behavior is disabled by default as this configuration usually requires composite primary keys to be specified
70
+ and ActiveRecord does not natively support composite primary keys. There are workarounds such as the
71
+ [composite_primary_keys gem](https://github.com/composite-primary-keys/composite_primary_keys).
72
+ * This is not supported for Postgres 10 (requires Postgres 11+)
73
+ * Primary key constraints must include all partition keys, for example: `primary_key: [:id, :created_at], partition_key: :created_at`
74
+ * Partition keys cannot use expressions
75
+ * Can be overridden via the `create_with_primary_key:` option when creating partitioned tables
76
+ - Default: `false`
77
+ - `include_subpartitions_in_partition_list`
78
+ - Whether to include nested subpartitions in the result of `YourModelClass.partiton_list` mby default.
79
+ You can always pass the `include_subpartitions:` option to override this.
80
+ - Default: `false` (for backward compatibility)
64
81
 
65
82
  Note that caching is done in-memory for each process of an application. Attaching / detaching partitions _will_ clear the cache, but only for the process that initiated the request. For multi-process web servers, it is recommended to use a TTL or disable caching entirely.
66
83
 
@@ -71,6 +88,11 @@ Note that caching is done in-memory for each process of an application. Attachin
71
88
  PgParty.configure do |c|
72
89
  c.caching_ttl = 60
73
90
  c.schema_exclude_partitions = false
91
+ c.include_subpartitions_in_partition_list = true
92
+ # Postgres 11+ users starting fresh may consider the below options to rely on Postgres' native features instead of
93
+ # this gem's template tables feature.
94
+ c.create_template_tables = false
95
+ c.create_with_primary_key = true
74
96
  end
75
97
  ```
76
98
 
@@ -88,25 +110,63 @@ These methods are available in migrations as well as `ActiveRecord::Base#connect
88
110
  - `create_list_partition`
89
111
  - Create partitioned table using the _list_ partitioning method
90
112
  - Required args: `table_name`, `partition_key:`
113
+ - `create_hash_partition` (Postgres 11+)
114
+ - Create partitioned table using the _hash_ partitioning method
115
+ - Required args: `table_name`, `partition_key:`
91
116
  - `create_range_partition_of`
92
117
  - Create partition in _range_ partitioned table with partition key between _range_ of values
93
118
  - Required args: `table_name`, `start_range:`, `end_range:`
119
+ - Create a subpartition by specifying a `partition_type:` of `:range`, `:list`, or `:hash` and a `partition_key:`
94
120
  - `create_list_partition_of`
95
121
  - Create partition in _list_ partitioned table with partition key in _list_ of values
96
122
  - Required args: `table_name`, `values:`
123
+ - Create a subpartition by specifying a `partition_type:` of `:range`, `:list`, or `:hash` and a `partition_key:`
124
+ - `create_hash_partition_of` (Postgres 11+)
125
+ - Create partition in _hash_ partitioned table for partition keys with hashed values having a specific remainder
126
+ - Required args: `table_name`, `modulus:`, `remainder`
127
+ - Create a subpartition by specifying a `partition_type:` of `:range`, `:list`, or `:hash` and a `partition_key:`
128
+ - Note that all partitions in a _hash_ partitioned table should have the same modulus. See [Examples](#examples) for more info.
129
+ - `create_default_partition_of` (Postgres 11+)
130
+ - Create a default partition for values not falling in the range or list constraints of any other partitions
131
+ - Required args: `table_name`
97
132
  - `attach_range_partition`
98
133
  - Attach existing table to _range_ partitioned table with partition key between _range_ of values
99
134
  - Required args: `parent_table_name`, `child_table_name`, `start_range:`, `end_range:`
100
135
  - `attach_list_partition`
101
136
  - Attach existing table to _list_ partitioned table with partition key in _list_ of values
102
137
  - Required args: `parent_table_name`, `child_table_name`, `values:`
138
+ - `attach_hash_partition` (Postgres 11+)
139
+ - Attach existing table to _hash_ partitioned table with partition key hashed values having a specific remainder
140
+ - Required args: `parent_table_name`, `child_table_name`, `modulus:`, `remainder`
141
+ - `attach_default_partition` (Postgres 11+)
142
+ - Attach existing table as the _default_ partition
143
+ - Required args: `parent_table_name`, `child_table_name`
103
144
  - `detach_partition`
104
145
  - Detach partition from both _range and list_ partitioned tables
105
146
  - Required args: `parent_table_name`, `child_table_name`
106
147
  - `create_table_like`
107
148
  - Clone _any_ existing table
108
149
  - Required args: `table_name`, `new_table_name`
109
-
150
+ - `partitions_for_table_name`
151
+ - List all attached partitions for a given table
152
+ - Required args: `table_name`, `include_subpartitions:` (true or false)
153
+ - `parent_for_table_name`
154
+ - Fetch the parent table for a partition
155
+ - Required args: `table_name`
156
+ - Pass optional `traverse: true` to return the top-level table in the hierarchy (for subpartitions)
157
+ - Returns `nil` if the table is not a partition / has no parent
158
+ - `table_partitioned?`
159
+ - Returns true if the table is partitioned (false for non-partitioned tables and partitions themselves)
160
+ - Required args: `table_name`
161
+ - `add_index_on_all_partitions`
162
+ - Recursively add an index to all partitions and subpartitions of `table_name` using Postgres's ADD INDEX CONCURRENTLY
163
+ algorithm which adds the index in a non-blocking manner.
164
+ - Required args: `table_name`, `column_name` (all `add_index` arguments are supported)
165
+ - Use the `in_threads:` option to add indexes in parallel threads when there are many partitions. A value of 2 to 4
166
+ may be reasonable for tables with many large partitions and hosts with 4+ CPUs/cores.
167
+ - Use `disable_ddl_transaction!` in your migration to disable transactions when using this command with `in_threads:`
168
+ or `algorithm: :concurrently`.
169
+
110
170
  #### Examples
111
171
 
112
172
  Create _range_ partitioned table on `created_at::date` with two partitions:
@@ -157,20 +217,127 @@ class CreateSomeListRecord < ActiveRecord::Migration[5.1]
157
217
  create_list_partition_of \
158
218
  :some_list_records,
159
219
  values: 101..200
220
+
221
+ # default partition support is available in Postgres 11 or higher
222
+ create_default_partition_of \
223
+ :some_list_records
224
+ end
225
+ end
226
+ ```
227
+
228
+ Create _hash_ partitioned table on `account_id` with two partitions (Postgres 11+ required):
229
+ * A hash partition can be used to spread keys evenly(ish) across partitions
230
+ * `modulus:` should always equal the total number of partitions planned for the table
231
+ * `remainder:` is an integer which should be in the range of 0 to modulus-1
232
+
233
+ ```ruby
234
+ class CreateSomeHashRecord < ActiveRecord::Migration[5.1]
235
+ def up
236
+ # symbol is used for partition keys referring to individual columns
237
+ # create_with_primary_key: true, template: false on Postgres 11 will rely on PostgreSQL's native partition schema
238
+ # management vs this gem's template tables
239
+ # Note composite primary keys will require a workaround in ActiveRecord, such as through the use of the composite_primary_keys gem
240
+ create_hash_partition :some_hash_records, partition_key: :account_id, primary_key: [:id, :account_id],
241
+ create_with_primary_key: true, template: false do |t|
242
+ t.bigserial :id, null: false
243
+ t.bigint :account_id, null: false
244
+ t.text :some_value
245
+ t.timestamps
246
+ end
247
+
248
+ # without name argument, child partition created as "some_list_records_<hash>"
249
+ create_hash_partition_of \
250
+ :some_hash_records,
251
+ modulus: 2,
252
+ remainder: 0
253
+
254
+ # without name argument, child partition created as "some_list_records_<hash>"
255
+ create_hash_partition_of \
256
+ :some_hash_records,
257
+ modulus: 2,
258
+ remainder: 1
259
+ end
260
+ end
261
+ ```
262
+
263
+ Advanced example with subpartitioning: Create _list_ partitioned table on `account_id` subpartitioned by _range_ on `created_at`
264
+ with default partitions. This example is for a table with no primary key... perhaps for some analytics use case.
265
+ * Default partitions are only supported in Postgres 11+
266
+
267
+ ```ruby
268
+ class CreateSomeListSubpartitionedRecord < ActiveRecord::Migration[5.1]
269
+ def up
270
+ create_list_partition :some_list_subpartitioned_records, partition_key: :account_id, id: false,
271
+ template: false do |t|
272
+ t.bigint :account_id, null: false
273
+ t.text :some_value
274
+ t.created_at
275
+ end
276
+
277
+ create_default_partition_of \
278
+ :some_list_subpartitioned_records,
279
+ name: :some_list_subpartitioned_records_default,
280
+ partition_type: :range,
281
+ partition_key: :created_at
282
+
283
+ create_range_partition_of \
284
+ :some_list_subpartitioned_records_default,
285
+ name: :some_list_subpartitioned_records_default_2019,
286
+ start_range: '2019-01-01',
287
+ end_range: '2019-12-31T23:59:59'
288
+
289
+ create_default_partition_of \
290
+ :some_list_subpartitioned_records_default
291
+
292
+ create_list_partition_of \
293
+ :some_list_subpartitioned_records,
294
+ name: :some_list_subpartitioned_records_1,
295
+ values: 1..100,
296
+ partition_type: :range,
297
+ partition_key: :created_at
298
+
299
+ create_range_partition_of \
300
+ :some_list_subpartitioned_records_1,
301
+ name: :some_list_subpartitioned_records_1_2019,
302
+ start_range: '2019-01-01',
303
+ end_range: '2019-12-31T23:59:59'
304
+
305
+ create_default_partition_of
306
+ :some_list_subpartitioned_records_1
307
+
308
+ create_list_partition_of \
309
+ :some_list_subpartitioned_records,
310
+ name: :some_list_subpartitioned_records_2,
311
+ values: 101..200,
312
+ partition_type: :range,
313
+ partition_key: :created_at
314
+
315
+ create_range_partition_of \
316
+ :some_list_subpartitioned_records_2,
317
+ name: :some_list_subpartitioned_records_2_2019,
318
+ start_range: '2019-01-01',
319
+ end_range: '2019-12-31T23:59:59'
320
+
321
+ create_default_partition_of \
322
+ :some_list_subpartitioned_records_2
160
323
  end
161
324
  end
162
325
  ```
163
326
 
327
+ #### Template Tables
164
328
  Unfortunately, PostgreSQL 10 doesn't support indexes on partitioned tables.
165
329
  However, individual _partitions_ can have indexes.
166
330
  To avoid explicit index creation for _every_ new partition, we've introduced the idea of template tables.
167
331
  For every call to `create_list_partition` and `create_range_partition`, a clone `<table_name>_template` is created.
168
332
  Indexes, constraints, etc. created on the template table will propagate to new partitions in calls to `create_list_partition_of` and `create_range_partition_of`:
333
+ * Subpartitions will correctly clone from template tables if a template table exists for the top-level ancestor
334
+ * When using Postgres 11 or higher, you may wish to disable template tables and use the native features instead, see [Configuration](#configuration)\
335
+ but this may result in you using composite primary keys, which is not natively supported by ActiveRecord.
169
336
 
170
337
  ```ruby
171
338
  class CreateSomeListRecord < ActiveRecord::Migration[5.1]
172
339
  def up
173
- # template table creation is enabled by default - use "template: false" to opt-out
340
+ # template table creation is enabled by default - use "template: false" or the config option to opt-out
174
341
  create_list_partition :some_list_records, partition_key: :id do |t|
175
342
  t.integer :some_foreign_id
176
343
  t.text :some_value
@@ -193,6 +360,8 @@ class CreateSomeListRecord < ActiveRecord::Migration[5.1]
193
360
  end
194
361
  ```
195
362
 
363
+ #### Attaching Existing Tables as Partitions
364
+
196
365
  Attach an existing table to a _range_ partitioned table:
197
366
 
198
367
  ```ruby
@@ -220,6 +389,20 @@ class AttachListPartition < ActiveRecord::Migration[5.1]
220
389
  end
221
390
  ```
222
391
 
392
+ Attach an existing table to a _hash_ partitioned table:
393
+
394
+ ```ruby
395
+ class AttachHashPartition < ActiveRecord::Migration[5.1]
396
+ def up
397
+ attach_hash_partition \
398
+ :some_hash_records,
399
+ :some_existing_table,
400
+ modulus: 2,
401
+ remainder: 1
402
+ end
403
+ end
404
+ ```
405
+
223
406
  Detach a partition from any partitioned table:
224
407
 
225
408
  ```ruby
@@ -230,6 +413,31 @@ class DetachPartition < ActiveRecord::Migration[5.1]
230
413
  end
231
414
  ```
232
415
 
416
+ #### Safely cascading `add_index` commands
417
+ Postgres 11+ will automatically cascade CREATE INDEX operations to partitions and subpartitions, however
418
+ CREATE INDEX CONCURRENTLY is not supported, meaning table locks will be taken on each table while the new index is built.
419
+ Postgres 10 provides no way to cascade index creation natively.
420
+ * The `add_index_on_all_partitions` method solves for these limitations by recursively creating the specified
421
+ index on all partitions and subpartitions. Index names on individual partitions will include a hash suffix to avoid conflicts.
422
+ * On Postgres 11+, the created indices are correctly attached to an index on the parent table
423
+ * On Postgres 10, if you are using [Template Tables](#template-tables-for-postgres-10), you will want to add the index to the template table separately.
424
+ * This command can also be used on subpartitions to cascade targeted indices starting at one level of the table hierarchy
425
+
426
+ ```ruby
427
+ class AddSomeValueIndexToSomeListRecord < ActiveRecord::Migration[5.1]
428
+ # add_index_on_all_partitions with in_threads option may not be used within a transaction
429
+ # (also, algorithm: :concurrently cannot be used within a transaction)
430
+ disable_ddl_transaction!
431
+
432
+ def up
433
+ add_index :some_records_template, :some_value # Only if using Postgres 10 with template tables
434
+
435
+ # Pass the `in_threads:` option to create indices in parallel across multiple Postgres connections
436
+ add_index_on_all_partitions :some_records, :some_value, algorithm: :concurrently, in_threads: 4
437
+ end
438
+ end
439
+ ```
440
+
233
441
  For more examples, take a look at the Combustion schema definition and integration specs:
234
442
 
235
443
  - https://github.com/rkrage/pg_party/blob/master/spec/dummy/db/schema.rb
@@ -250,12 +458,15 @@ Class methods available to _all_ ActiveRecord models:
250
458
  - `list_partition_by`
251
459
  - Configure a model backed by a _list_ partitioned table
252
460
  - Required arg: `key` (partition key column) or block returning partition key expression
461
+ - `hash_partition_by`
462
+ - Configure a model backed by a _hash_ partitioned table
463
+ - Required arg: `key` (partition key column) or block returning partition key expression
253
464
 
254
465
  Class methods available to both _range and list_ partitioned models:
255
466
 
256
467
  - `partitions`
257
468
  - Retrieve a list of currently attached partitions
258
- - No arguments
469
+ - Optional `include_subpartitions:` argument to include all subpartitions in the returned list
259
470
  - `in_partition`
260
471
  - Retrieve an ActiveRecord model scoped to an individual partition
261
472
  - Required arg: `child_table_name`
@@ -280,6 +491,16 @@ Class methods available to _list_ partitioned models:
280
491
  - `partition_key_in`
281
492
  - Query for records where partition key in _list_ of values
282
493
  - Required arg: list of `values`
494
+
495
+
496
+ Class methods available to _hash_ partitioned models:
497
+
498
+ - `create_partition`
499
+ - Dynamically create new partition with hashed partition key divided by _modulus_ equals _remainder_
500
+ - Required arg: `modulus:`, `remainder:`
501
+ - `partition_key_in`
502
+ - Query for records where partition key in _list_ of values (method operates the same as for _list_ partitions above)
503
+ - Required arg: list of `values`
283
504
 
284
505
  #### Examples
285
506
 
@@ -301,6 +522,15 @@ class SomeListRecord < ApplicationRecord
301
522
  end
302
523
  ```
303
524
 
525
+ Configure model backed by a _hash_ partitioned table to get access to the methods described above:
526
+
527
+ ```ruby
528
+ class SomeHashRecord < ApplicationRecord
529
+ # symbol is used for partition keys referring to individual columns
530
+ hash_partition_by :id
531
+ end
532
+ ```
533
+
304
534
  Dynamically create new partition from _range_ partitioned model:
305
535
 
306
536
  ```ruby
@@ -315,19 +545,26 @@ Dynamically create new partition from _list_ partitioned model:
315
545
  SomeListRecord.create_partition(values: 200..300)
316
546
  ```
317
547
 
548
+ Dynamically create new partition from _hash_ partitioned model:
549
+
550
+ ```ruby
551
+ # additional options include: "name:" and "primary_key:"
552
+ SomeHashRecord.create_partition(modulus: 2, remainder: 1)
553
+ ```
554
+
318
555
  For _range_ partitioned model, query for records where partition key in _range_ of values:
319
556
 
320
557
  ```ruby
321
558
  SomeRangeRecord.partition_key_in("2019-06-08", "2019-06-10")
322
559
  ```
323
560
 
324
- For _list_ partitioned model, query for records where partition key in _list_ of values:
561
+ For _list_ and _hash_ partitioned models, query for records where partition key in _list_ of values:
325
562
 
326
563
  ```ruby
327
564
  SomeListRecord.partition_key_in(1, 2, 3, 4)
328
565
  ```
329
566
 
330
- For both _range and list_ partitioned models, query for records matching partition key:
567
+ For all partitioned models, query for records matching partition key:
331
568
 
332
569
  ```ruby
333
570
  SomeRangeRecord.partition_key_eq(Date.current)
@@ -335,15 +572,15 @@ SomeRangeRecord.partition_key_eq(Date.current)
335
572
  SomeListRecord.partition_key_eq(100)
336
573
  ```
337
574
 
338
- For both _range and list_ partitioned models, retrieve currently attached partitions:
575
+ For all partitioned models, retrieve currently attached partitions:
339
576
 
340
577
  ```ruby
341
578
  SomeRangeRecord.partitions
342
579
 
343
- SomeListRecord.partitions
580
+ SomeListRecord.partitions(include_subpartitions: true) # Include nested subpartitions
344
581
  ```
345
582
 
346
- For both _range and list_ partitioned models, retrieve ActiveRecord model scoped to individual partition:
583
+ For both all partitioned models, retrieve ActiveRecord model scoped to individual partition:
347
584
 
348
585
  ```ruby
349
586
  SomeRangeRecord.in_partition(:some_range_records_partition_name)
@@ -11,6 +11,10 @@ module PgParty
11
11
  raise "#create_list_partition is not implemented"
12
12
  end
13
13
 
14
+ def create_hash_partition(*)
15
+ raise "#create_hash_partition is not implemented"
16
+ end
17
+
14
18
  def create_range_partition_of(*)
15
19
  raise "#create_range_partition_of is not implemented"
16
20
  end
@@ -19,6 +23,14 @@ module PgParty
19
23
  raise "#create_list_partition_of is not implemented"
20
24
  end
21
25
 
26
+ def create_hash_partition_of(*)
27
+ raise "#create_hash_partition_of is not implemented"
28
+ end
29
+
30
+ def create_default_partition_of(*)
31
+ raise "#create_default_partition_of is not implemented"
32
+ end
33
+
22
34
  def create_table_like(*)
23
35
  raise "#create_table_like is not implemented"
24
36
  end
@@ -31,9 +43,33 @@ module PgParty
31
43
  raise "#attach_list_partition is not implemented"
32
44
  end
33
45
 
46
+ def attach_hash_partition(*)
47
+ raise "#attach_hash_partition is not implemented"
48
+ end
49
+
50
+ def attach_default_partition(*)
51
+ raise "#attach_default_partition is not implemented"
52
+ end
53
+
34
54
  def detach_partition(*)
35
55
  raise "#detach_partition is not implemented"
36
56
  end
57
+
58
+ def parent_for_table_name(*)
59
+ raise "#parent_for_table_name is not implemented"
60
+ end
61
+
62
+ def partitions_for_table_name(*)
63
+ raise "#partitions_for_table_name is not implemented"
64
+ end
65
+
66
+ def add_index_on_all_partitions(*)
67
+ raise "#add_index_on_all_partitions is not implemented"
68
+ end
69
+
70
+ def table_partitioned?(*)
71
+ raise "#table_partitioned? is not implemented"
72
+ end
37
73
  end
38
74
  end
39
75
  end
@@ -14,6 +14,10 @@ module PgParty
14
14
  PgParty::AdapterDecorator.new(self).create_list_partition(*args, &blk)
15
15
  end
16
16
 
17
+ ruby2_keywords def create_hash_partition(*args, &blk)
18
+ PgParty::AdapterDecorator.new(self).create_hash_partition(*args, &blk)
19
+ end
20
+
17
21
  ruby2_keywords def create_range_partition_of(*args)
18
22
  PgParty::AdapterDecorator.new(self).create_range_partition_of(*args)
19
23
  end
@@ -22,6 +26,14 @@ module PgParty
22
26
  PgParty::AdapterDecorator.new(self).create_list_partition_of(*args)
23
27
  end
24
28
 
29
+ ruby2_keywords def create_hash_partition_of(*args)
30
+ PgParty::AdapterDecorator.new(self).create_hash_partition_of(*args)
31
+ end
32
+
33
+ ruby2_keywords def create_default_partition_of(*args)
34
+ PgParty::AdapterDecorator.new(self).create_default_partition_of(*args)
35
+ end
36
+
25
37
  ruby2_keywords def create_table_like(*args)
26
38
  PgParty::AdapterDecorator.new(self).create_table_like(*args)
27
39
  end
@@ -34,9 +46,33 @@ module PgParty
34
46
  PgParty::AdapterDecorator.new(self).attach_list_partition(*args)
35
47
  end
36
48
 
49
+ ruby2_keywords def attach_hash_partition(*args)
50
+ PgParty::AdapterDecorator.new(self).attach_hash_partition(*args)
51
+ end
52
+
53
+ ruby2_keywords def attach_default_partition(*args)
54
+ PgParty::AdapterDecorator.new(self).attach_default_partition(*args)
55
+ end
56
+
37
57
  ruby2_keywords def detach_partition(*args)
38
58
  PgParty::AdapterDecorator.new(self).detach_partition(*args)
39
59
  end
60
+
61
+ ruby2_keywords def partitions_for_table_name(*args)
62
+ PgParty::AdapterDecorator.new(self).partitions_for_table_name(*args)
63
+ end
64
+
65
+ ruby2_keywords def parent_for_table_name(*args)
66
+ PgParty::AdapterDecorator.new(self).parent_for_table_name(*args)
67
+ end
68
+
69
+ ruby2_keywords def add_index_on_all_partitions(*args)
70
+ PgParty::AdapterDecorator.new(self).add_index_on_all_partitions(*args)
71
+ end
72
+
73
+ ruby2_keywords def table_partitioned?(*args)
74
+ PgParty::AdapterDecorator.new(self).table_partitioned?(*args)
75
+ end
40
76
  end
41
77
  end
42
78
  end
@@ -1,9 +1,12 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "digest"
4
+ require 'parallel'
4
5
 
5
6
  module PgParty
6
7
  class AdapterDecorator < SimpleDelegator
8
+ SUPPORTED_PARTITION_TYPES = %i[range list hash].freeze
9
+
7
10
  def initialize(adapter)
8
11
  super(adapter)
9
12
 
@@ -18,6 +21,10 @@ module PgParty
18
21
  create_partition(table_name, :list, partition_key, **options, &blk)
19
22
  end
20
23
 
24
+ def create_hash_partition(table_name, partition_key:, **options, &blk)
25
+ create_partition(table_name, :hash, partition_key, **options, &blk)
26
+ end
27
+
21
28
  def create_range_partition_of(table_name, start_range:, end_range:, **options)
22
29
  create_partition_of(table_name, range_constraint_clause(start_range, end_range), **options)
23
30
  end
@@ -26,17 +33,42 @@ module PgParty
26
33
  create_partition_of(table_name, list_constraint_clause(values), **options)
27
34
  end
28
35
 
36
+ def create_hash_partition_of(table_name, modulus:, remainder:, **options)
37
+ create_partition_of(table_name, hash_constraint_clause(modulus, remainder), **options)
38
+ end
39
+
40
+ def create_default_partition_of(table_name, **options)
41
+ create_partition_of(table_name, nil, default_partition: true, **options)
42
+ end
43
+
29
44
  def create_table_like(table_name, new_table_name, **options)
30
- primary_key = options.fetch(:primary_key) { calculate_primary_key(table_name) }
45
+ primary_key = options.fetch(:primary_key) { calculate_primary_key(table_name) }
46
+ partition_key = options.fetch(:partition_key, nil)
47
+ partition_type = options.fetch(:partition_type, nil)
48
+ create_with_pks = options.fetch(
49
+ :create_with_primary_key,
50
+ PgParty.config.create_with_primary_key
51
+ )
52
+
53
+ validate_primary_key(primary_key) unless create_with_pks
54
+ if partition_type
55
+ validate_supported_partition_type!(partition_type)
56
+ raise ArgumentError, '`partition_key` is required when specifying a partition_type' unless partition_key
57
+ end
31
58
 
32
- validate_primary_key(primary_key)
59
+ like_option = if !partition_type || create_with_pks
60
+ 'INCLUDING ALL'
61
+ else
62
+ 'INCLUDING ALL EXCLUDING INDEXES'
63
+ end
33
64
 
34
65
  execute(<<-SQL)
35
66
  CREATE TABLE #{quote_table_name(new_table_name)} (
36
- LIKE #{quote_table_name(table_name)} INCLUDING ALL
37
- )
67
+ LIKE #{quote_table_name(table_name)} #{like_option}
68
+ ) #{partition_type ? partition_by_clause(partition_type, partition_key) : nil}
38
69
  SQL
39
70
 
71
+ return if partition_type
40
72
  return if !primary_key
41
73
  return if has_primary_key?(new_table_name)
42
74
 
@@ -54,6 +86,20 @@ module PgParty
54
86
  attach_partition(parent_table_name, child_table_name, list_constraint_clause(values))
55
87
  end
56
88
 
89
+ def attach_hash_partition(parent_table_name, child_table_name, modulus:, remainder:)
90
+ attach_partition(parent_table_name, child_table_name, hash_constraint_clause(modulus, remainder))
91
+ end
92
+
93
+ def attach_default_partition(parent_table_name, child_table_name)
94
+ execute(<<-SQL)
95
+ ALTER TABLE #{quote_table_name(parent_table_name)}
96
+ ATTACH PARTITION #{quote_table_name(child_table_name)}
97
+ DEFAULT
98
+ SQL
99
+
100
+ PgParty.cache.clear!
101
+ end
102
+
57
103
  def detach_partition(parent_table_name, child_table_name)
58
104
  execute(<<-SQL)
59
105
  ALTER TABLE #{quote_table_name(parent_table_name)}
@@ -63,23 +109,101 @@ module PgParty
63
109
  PgParty.cache.clear!
64
110
  end
65
111
 
66
- private
112
+ def partitions_for_table_name(table_name, include_subpartitions:, _accumulator: [])
113
+ select_values(%[
114
+ SELECT pg_inherits.inhrelid::regclass::text
115
+ FROM pg_tables
116
+ INNER JOIN pg_inherits
117
+ ON pg_tables.tablename::regclass = pg_inherits.inhparent::regclass
118
+ WHERE pg_tables.schemaname = current_schema() AND
119
+ pg_tables.tablename = #{quote(table_name)}
120
+ ]).each_with_object(_accumulator) do |partition, acc|
121
+ acc << partition
122
+ next unless include_subpartitions
123
+
124
+ partitions_for_table_name(partition, include_subpartitions: true, _accumulator: acc)
125
+ end
126
+ end
67
127
 
68
- def create_partition(table_name, type, partition_key, **options)
69
- modified_options = options.except(:id, :primary_key, :template)
70
- template = options.fetch(:template, true)
71
- id = options.fetch(:id, :bigserial)
72
- primary_key = options.fetch(:primary_key) { calculate_primary_key(table_name) }
128
+ def parent_for_table_name(table_name, traverse: false)
129
+ parent = select_values(%[
130
+ SELECT pg_inherits.inhparent::regclass::text
131
+ FROM pg_tables
132
+ INNER JOIN pg_inherits
133
+ ON pg_tables.tablename::regclass = pg_inherits.inhrelid::regclass
134
+ WHERE pg_tables.schemaname = current_schema() AND
135
+ pg_tables.tablename = #{quote(table_name)}
136
+ ]).first
137
+ return parent if parent.nil? || !traverse
138
+
139
+ while (parents_parent = parent_for_table_name(parent)) do
140
+ parent = parents_parent
141
+ end
142
+
143
+ parent
144
+ end
73
145
 
74
- validate_primary_key(primary_key)
146
+ def add_index_on_all_partitions(table_name, column_name, in_threads: nil, **options)
147
+ if in_threads && open_transactions > 0
148
+ raise ArgumentError, '`in_threads:` cannot be used within a transaction. If running in a migration, use '\
149
+ '`disable_ddl_transaction!` and break out this operation into its own migration.'
150
+ end
75
151
 
76
- modified_options[:id] = false
77
- modified_options[:options] = "PARTITION BY #{type.to_s.upcase} (#{quote_partition_key(partition_key)})"
152
+ index_name, index_type, index_columns, index_options, algorithm, using = extract_index_options(
153
+ add_index_options(table_name, column_name, **options)
154
+ )
155
+
156
+ # Postgres limits index name to 63 bytes (characters). We will use 8 characters for a `_random_suffix`
157
+ # on partitions to ensure no conflicts, leaving 55 chars for the specified index name
158
+ raise ArgumentError 'index name is too long - must be 55 characters or fewer' if index_name.length > 55
159
+
160
+ recursive_add_index(
161
+ table_name: table_name,
162
+ index_name: index_name,
163
+ index_type: index_type,
164
+ index_columns: index_columns,
165
+ index_options: index_options,
166
+ algorithm: algorithm,
167
+ using: using,
168
+ in_threads: in_threads
169
+ )
170
+ end
78
171
 
79
- create_table(table_name, modified_options) do |td|
80
- if id == :uuid
172
+ def table_partitioned?(table_name)
173
+ select_values(%[
174
+ SELECT relkind FROM pg_catalog.pg_class AS c
175
+ JOIN pg_catalog.pg_namespace AS ns ON c.relnamespace = ns.oid
176
+ WHERE relname = #{quote(table_name)} AND nspname = current_schema()
177
+ ]).first == 'p'
178
+ end
179
+
180
+ private
181
+
182
+ def create_partition(table_name, type, partition_key, **options)
183
+ modified_options = options.except(:id, :primary_key, :template, :create_with_primary_key)
184
+ template = options.fetch(:template, PgParty.config.create_template_tables)
185
+ id = options.fetch(:id, :bigserial)
186
+ primary_key = options.fetch(:primary_key) { calculate_primary_key(table_name) }
187
+ create_with_pks = options.fetch(
188
+ :create_with_primary_key,
189
+ PgParty.config.create_with_primary_key
190
+ )
191
+
192
+ validate_supported_partition_type!(type)
193
+
194
+ if create_with_pks
195
+ modified_options[:primary_key] = primary_key
196
+ modified_options[:id] = id
197
+ else
198
+ validate_primary_key(primary_key)
199
+ modified_options[:id] = false
200
+ end
201
+ modified_options[:options] = partition_by_clause(type, partition_key)
202
+
203
+ create_table(table_name, **modified_options) do |td|
204
+ if !modified_options[:id] && id == :uuid
81
205
  td.column(primary_key, id, null: false, default: uuid_function)
82
- elsif id
206
+ elsif !modified_options[:id] && id
83
207
  td.column(primary_key, id, null: false)
84
208
  end
85
209
 
@@ -87,11 +211,16 @@ module PgParty
87
211
  end
88
212
 
89
213
  # Rails 4 has a bug where uuid columns are always nullable
90
- change_column_null(table_name, primary_key, false) if id == :uuid
214
+ change_column_null(table_name, primary_key, false) if !modified_options[:id] && id == :uuid
91
215
 
92
216
  return unless template
93
217
 
94
- create_table_like(table_name, template_table_name(table_name), primary_key: id && primary_key)
218
+ create_table_like(
219
+ table_name,
220
+ template_table_name(table_name),
221
+ primary_key: id && primary_key,
222
+ create_with_primary_key: create_with_pks
223
+ )
95
224
  end
96
225
 
97
226
  def create_partition_of(table_name, constraint_clause, **options)
@@ -99,13 +228,21 @@ module PgParty
99
228
  primary_key = options.fetch(:primary_key) { calculate_primary_key(table_name) }
100
229
  template_table_name = template_table_name(table_name)
101
230
 
231
+ validate_default_partition_support! if options[:default_partition]
232
+
102
233
  if schema_cache.data_source_exists?(template_table_name)
103
- create_table_like(template_table_name, child_table_name, primary_key: false)
234
+ create_table_like(template_table_name, child_table_name, primary_key: false,
235
+ partition_type: options[:partition_type], partition_key: options[:partition_key])
104
236
  else
105
- create_table_like(table_name, child_table_name, primary_key: primary_key)
237
+ create_table_like(table_name, child_table_name, primary_key: primary_key,
238
+ partition_type: options[:partition_type], partition_key: options[:partition_key])
106
239
  end
107
240
 
108
- attach_partition(table_name, child_table_name, constraint_clause)
241
+ if options[:default_partition]
242
+ attach_default_partition(table_name, child_table_name)
243
+ else
244
+ attach_partition(table_name, child_table_name, constraint_clause)
245
+ end
109
246
 
110
247
  child_table_name
111
248
  end
@@ -120,6 +257,103 @@ module PgParty
120
257
  PgParty.cache.clear!
121
258
  end
122
259
 
260
+ def recursive_add_index(table_name:, index_name:, index_type:, index_columns:, index_options:, using:, algorithm:,
261
+ in_threads: nil, _parent_index_name: nil, _created_index_names: [])
262
+ partitions = partitions_for_table_name(table_name, include_subpartitions: false)
263
+ updated_name = _created_index_names.empty? ? index_name : generate_index_name(index_name, table_name)
264
+
265
+ # If this is a partitioned table, add index ONLY on this table.
266
+ if table_partitioned?(table_name)
267
+ add_index_only(table_name, type: index_type, name: updated_name, using: using, columns: index_columns,
268
+ options: index_options)
269
+ _created_index_names << updated_name
270
+
271
+ parallel_map(partitions, in_threads: in_threads) do |partition_name|
272
+ recursive_add_index(
273
+ table_name: partition_name,
274
+ index_name: index_name,
275
+ index_type: index_type,
276
+ index_columns: index_columns,
277
+ index_options: index_options,
278
+ using: using,
279
+ algorithm: algorithm,
280
+ _parent_index_name: updated_name,
281
+ _created_index_names: _created_index_names
282
+ )
283
+ end
284
+ else
285
+ _created_index_names << updated_name # Track as created before execution of concurrent index command
286
+ add_index_from_options(table_name, name: updated_name, type: index_type, algorithm: algorithm, using: using,
287
+ columns: index_columns, options: index_options)
288
+ end
289
+
290
+ attach_child_index(updated_name, _parent_index_name) if _parent_index_name
291
+
292
+ return true if index_valid?(updated_name)
293
+
294
+ raise 'index creation failed - an index was marked invalid'
295
+ rescue => e
296
+ # Clean up any indexes created so this command can be retried later
297
+ drop_indices_if_exist(_created_index_names)
298
+ raise e
299
+ end
300
+
301
+ def attach_child_index(child, parent)
302
+ return unless postgres_major_version >= 11
303
+
304
+ execute "ALTER INDEX #{quote_column_name(parent)} ATTACH PARTITION #{quote_column_name(child)}"
305
+ end
306
+
307
+ def add_index_only(table_name, type:, name:, using:, columns:, options:)
308
+ return unless postgres_major_version >= 11
309
+
310
+ execute "CREATE #{type} INDEX #{quote_column_name(name)} ON ONLY "\
311
+ "#{quote_table_name(table_name)} #{using} (#{columns})#{options}"
312
+ end
313
+
314
+ def add_index_from_options(table_name, name:, type:, algorithm:, using:, columns:, options:)
315
+ execute "CREATE #{type} INDEX #{algorithm} #{quote_column_name(name)} ON "\
316
+ "#{quote_table_name(table_name)} #{using} (#{columns})#{options}"
317
+ end
318
+
319
+ def extract_index_options(add_index_options_result)
320
+ # Rails 6.1 changes the result of #add_index_options
321
+ index_definition = add_index_options_result.first
322
+ return add_index_options_result unless index_definition.is_a?(ActiveRecord::ConnectionAdapters::IndexDefinition)
323
+
324
+ index_columns = if index_definition.columns.is_a?(String)
325
+ index_definition.columns
326
+ else
327
+ quoted_columns_for_index(index_definition.columns, index_definition.column_options)
328
+ end
329
+
330
+ [
331
+ index_definition.name,
332
+ index_definition.unique ? 'UNIQUE' : index_definition.type,
333
+ index_columns,
334
+ index_definition.where ? " WHERE #{index_definition.where}" : nil,
335
+ add_index_options_result.second, # algorithm option
336
+ index_definition.using ? "USING #{index_definition.using}" : nil
337
+ ]
338
+ end
339
+
340
+ def drop_indices_if_exist(index_names)
341
+ index_names.uniq.each { |name| execute "DROP INDEX IF EXISTS #{quote_column_name(name)}" }
342
+ end
343
+
344
+ def parallel_map(arr, in_threads:)
345
+ return [] if arr.empty?
346
+ return arr.map { |item| yield(item) } unless in_threads && in_threads > 1
347
+
348
+ if ActiveRecord::Base.connection_pool.size <= in_threads
349
+ raise ArgumentError, 'in_threads: must be lower than your database connection pool size'
350
+ end
351
+
352
+ Parallel.map(arr, in_threads: in_threads) do |item|
353
+ ActiveRecord::Base.connection_pool.with_connection { yield(item) }
354
+ end
355
+ end
356
+
123
357
  # Rails 5.2 now returns boolean literals
124
358
  # This causes partition creation to fail when the constraint clause includes a boolean
125
359
  # Might be a PostgreSQL bug, but for now let's revert to the old quoting behavior
@@ -157,27 +391,69 @@ module PgParty
157
391
  end
158
392
 
159
393
  def template_table_name(table_name)
160
- "#{table_name}_template"
394
+ "#{parent_for_table_name(table_name, traverse: true) || table_name}_template"
161
395
  end
162
396
 
163
397
  def range_constraint_clause(start_range, end_range)
164
398
  "FROM (#{quote_collection(start_range)}) TO (#{quote_collection(end_range)})"
165
399
  end
166
400
 
401
+ def hash_constraint_clause(modulus, remainder)
402
+ "WITH (MODULUS #{modulus.to_i}, REMAINDER #{remainder.to_i})"
403
+ end
404
+
167
405
  def list_constraint_clause(values)
168
406
  "IN (#{quote_collection(values.try(:to_a) || values)})"
169
407
  end
170
408
 
409
+ def partition_by_clause(type, partition_key)
410
+ "PARTITION BY #{type.to_s.upcase} (#{quote_partition_key(partition_key)})"
411
+ end
412
+
171
413
  def uuid_function
172
414
  try(:supports_pgcrypto_uuid?) ? "gen_random_uuid()" : "uuid_generate_v4()"
173
415
  end
174
416
 
175
417
  def hashed_table_name(table_name, key)
176
- "#{table_name}_#{Digest::MD5.hexdigest(key)[0..6]}"
418
+ return "#{table_name}_#{Digest::MD5.hexdigest(key)[0..6]}" if key
419
+
420
+ # use _default suffix for default partitions (without a constraint clause)
421
+ "#{table_name}_default"
422
+ end
423
+
424
+ def index_valid?(index_name)
425
+ select_values(
426
+ "SELECT relname FROM pg_class, pg_index WHERE pg_index.indisvalid = false AND "\
427
+ "pg_index.indexrelid = pg_class.oid AND relname = #{quote(index_name)}"
428
+ ).empty?
429
+ end
430
+
431
+ def generate_index_name(index_name, table_name)
432
+ "#{index_name}_#{Digest::MD5.hexdigest(table_name)[0..6]}"
433
+ end
434
+
435
+ def validate_supported_partition_type!(partition_type)
436
+ if (sym = partition_type.to_s.downcase.to_sym) && sym.in?(SUPPORTED_PARTITION_TYPES)
437
+ return if sym != :hash || postgres_major_version >= 11
438
+
439
+ raise NotImplementedError, 'Hash partitions are only available in Postgres 11 or higher'
440
+ end
441
+
442
+ raise ArgumentError, "Supported partition types are #{SUPPORTED_PARTITION_TYPES.join(', ')}"
443
+ end
444
+
445
+ def validate_default_partition_support!
446
+ return if postgres_major_version >= 11
447
+
448
+ raise NotImplementedError, 'Default partitions are only available in Postgres 11 or higher'
177
449
  end
178
450
 
179
451
  def supports_partitions?
180
- __getobj__.send(:postgresql_version) >= 100000
452
+ postgres_major_version >= 10
453
+ end
454
+
455
+ def postgres_major_version
456
+ __getobj__.send(:postgresql_version)/10000
181
457
  end
182
458
  end
183
459
  end
@@ -9,7 +9,7 @@ module PgParty
9
9
  def initialize
10
10
  # automatically initialize a new hash when
11
11
  # accessing an object id that doesn't exist
12
- @store = Hash.new { |h, k| h[k] = { models: {}, partitions: nil } }
12
+ @store = Hash.new { |h, k| h[k] = { models: {}, partitions: nil, partitions_with_subpartitions: nil } }
13
13
  end
14
14
 
15
15
  def clear!
@@ -24,10 +24,11 @@ module PgParty
24
24
  LOCK.synchronize { fetch_value(@store[key][:models], child_table.to_sym, block) }
25
25
  end
26
26
 
27
- def fetch_partitions(key, &block)
27
+ def fetch_partitions(key, include_subpartitions, &block)
28
28
  return block.call unless caching_enabled?
29
+ sub_key = include_subpartitions ? :partitions_with_subpartitions : :partitions
29
30
 
30
- LOCK.synchronize { fetch_value(@store[key], :partitions, block) }
31
+ LOCK.synchronize { fetch_value(@store[key], sub_key, block) }
31
32
  end
32
33
 
33
34
  private
@@ -5,12 +5,18 @@ module PgParty
5
5
  attr_accessor \
6
6
  :caching,
7
7
  :caching_ttl,
8
- :schema_exclude_partitions
8
+ :schema_exclude_partitions,
9
+ :create_template_tables,
10
+ :create_with_primary_key,
11
+ :include_subpartitions_in_partition_list
9
12
 
10
13
  def initialize
11
14
  @caching = true
12
15
  @caching_ttl = -1
13
16
  @schema_exclude_partitions = true
17
+ @create_template_tables = true
18
+ @create_with_primary_key = false
19
+ @include_subpartitions_in_partition_list = false
14
20
  end
15
21
  end
16
22
  end
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pg_party/model_decorator"
4
+ require "ruby2_keywords"
5
+
6
+ module PgParty
7
+ module Model
8
+ module HashMethods
9
+ ruby2_keywords def create_partition(*args)
10
+ PgParty::ModelDecorator.new(self).create_hash_partition(*args)
11
+ end
12
+
13
+ ruby2_keywords def partition_key_in(*args)
14
+ PgParty::ModelDecorator.new(self).hash_partition_key_in(*args)
15
+ end
16
+ end
17
+ end
18
+ end
@@ -10,6 +10,10 @@ module PgParty
10
10
  PgParty::ModelDecorator.new(self).create_list_partition(*args)
11
11
  end
12
12
 
13
+ ruby2_keywords def create_default_partition(*args)
14
+ PgParty::ModelDecorator.new(self).create_default_partition(*args)
15
+ end
16
+
13
17
  ruby2_keywords def partition_key_in(*args)
14
18
  PgParty::ModelDecorator.new(self).list_partition_key_in(*args)
15
19
  end
@@ -13,6 +13,10 @@ module PgParty
13
13
  PgParty::ModelInjector.new(self, *key, &blk).inject_list_methods
14
14
  end
15
15
 
16
+ def hash_partition_by(*key, &blk)
17
+ PgParty::ModelInjector.new(self, *key, &blk).inject_hash_methods
18
+ end
19
+
16
20
  def partitioned?
17
21
  try(:partition_key).present?
18
22
  end
@@ -10,6 +10,10 @@ module PgParty
10
10
  PgParty::ModelDecorator.new(self).create_range_partition(*args)
11
11
  end
12
12
 
13
+ ruby2_keywords def create_default_partition(*args)
14
+ PgParty::ModelDecorator.new(self).create_default_partition(*args)
15
+ end
16
+
13
17
  ruby2_keywords def partition_key_in(*args)
14
18
  PgParty::ModelDecorator.new(self).range_partition_key_in(*args)
15
19
  end
@@ -6,13 +6,15 @@ module PgParty
6
6
  module Model
7
7
  module SharedMethods
8
8
  def reset_primary_key
9
- if self != base_class
10
- base_class.primary_key
11
- elsif partition_name = partitions.first
12
- in_partition(partition_name).get_primary_key(base_class.name)
13
- else
14
- get_primary_key(base_class.name)
15
- end
9
+ return base_class.primary_key if self != base_class
10
+
11
+ partitions = partitions(include_subpartitions: PgParty.config.include_subpartitions_in_partition_list)
12
+ return get_primary_key(base_class.name) if partitions.empty?
13
+
14
+ first_partition = partitions.detect { |p| !connection.table_partitioned?(p) }
15
+ raise 'No leaf partitions exist for this model. Create a partition to contain your data' unless first_partition
16
+
17
+ in_partition(first_partition).get_primary_key(base_class.name)
16
18
  end
17
19
 
18
20
  def table_exists?
@@ -21,8 +23,8 @@ module PgParty
21
23
  connection.schema_cache.data_source_exists?(target_table)
22
24
  end
23
25
 
24
- def partitions
25
- PgParty::ModelDecorator.new(self).partitions
26
+ def partitions(**args)
27
+ PgParty::ModelDecorator.new(self).partitions(**args)
26
28
  end
27
29
 
28
30
  def in_partition(*args)
@@ -36,7 +36,7 @@ module PgParty
36
36
  if complex_partition_key
37
37
  complex_partition_key_query("(#{partition_key}) = (?)", value)
38
38
  else
39
- where_partition_key(:eq, value)
39
+ where(partition_key_arel(:eq, value))
40
40
  end
41
41
  end
42
42
 
@@ -48,9 +48,7 @@ module PgParty
48
48
  end_range
49
49
  )
50
50
  else
51
- where_partition_key(:gteq, start_range).merge(
52
- where_partition_key(:lt, end_range)
53
- )
51
+ where(partition_key_arel(:gteq, start_range).and(partition_key_arel(:lt, end_range)))
54
52
  end
55
53
  end
56
54
 
@@ -62,15 +60,11 @@ module PgParty
62
60
  end
63
61
  end
64
62
 
65
- def partitions
66
- PgParty.cache.fetch_partitions(cache_key) do
67
- connection.select_values(<<-SQL)
68
- SELECT pg_inherits.inhrelid::regclass::text
69
- FROM pg_tables
70
- INNER JOIN pg_inherits
71
- ON pg_tables.tablename::regclass = pg_inherits.inhparent::regclass
72
- WHERE pg_tables.tablename = #{connection.quote(table_name)}
73
- SQL
63
+ alias_method :hash_partition_key_in, :list_partition_key_in
64
+
65
+ def partitions(include_subpartitions: PgParty.config.include_subpartitions_in_partition_list)
66
+ PgParty.cache.fetch_partitions(cache_key, include_subpartitions) do
67
+ connection.partitions_for_table_name(table_name, include_subpartitions: include_subpartitions)
74
68
  end
75
69
  rescue
76
70
  []
@@ -95,6 +89,23 @@ module PgParty
95
89
  create_partition(:create_list_partition_of, table_name, **modified_options)
96
90
  end
97
91
 
92
+ def create_hash_partition(modulus:, remainder:, **options)
93
+ modified_options = options.merge(
94
+ modulus: modulus,
95
+ remainder: remainder,
96
+ primary_key: primary_key,
97
+ )
98
+
99
+ create_partition(:create_hash_partition_of, table_name, **modified_options)
100
+ end
101
+
102
+ def create_default_partition(**options)
103
+ modified_options = options.merge(
104
+ primary_key: primary_key,
105
+ )
106
+ create_partition(:create_default_partition_of, table_name, **modified_options)
107
+ end
108
+
98
109
  private
99
110
 
100
111
  def create_partition(migration_method, table_name, **options)
@@ -133,7 +144,7 @@ module PgParty
133
144
  from(subquery, current_alias)
134
145
  end
135
146
 
136
- def where_partition_key(meth, values)
147
+ def partition_key_arel(meth, values)
137
148
  partition_key_array = Array.wrap(partition_key)
138
149
  values = Array.wrap(values)
139
150
 
@@ -141,7 +152,7 @@ module PgParty
141
152
  raise "number of provided values does not match the number of partition key columns"
142
153
  end
143
154
 
144
- arel_query = partition_key_array.zip(values).inject(nil) do |obj, (column, value)|
155
+ partition_key_array.zip(values).inject(nil) do |obj, (column, value)|
145
156
  node = current_arel_table[column].send(meth, value)
146
157
 
147
158
  if obj.nil?
@@ -150,8 +161,6 @@ module PgParty
150
161
  obj.and(node)
151
162
  end
152
163
  end
153
-
154
- where(arel_query)
155
164
  end
156
165
  end
157
166
  end
@@ -20,6 +20,12 @@ module PgParty
20
20
  inject_methods_for(PgParty::Model::ListMethods)
21
21
  end
22
22
 
23
+ def inject_hash_methods
24
+ require "pg_party/model/hash_methods"
25
+
26
+ inject_methods_for(PgParty::Model::HashMethods)
27
+ end
28
+
23
29
  private
24
30
 
25
31
  def inject_methods_for(mod)
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module PgParty
4
- VERSION = "1.3.0"
4
+ VERSION = "1.6.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pg_party
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.3.0
4
+ version: 1.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ryan Krage
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-06-12 00:00:00.000000000 Z
11
+ date: 2022-04-22 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activerecord
@@ -16,20 +16,20 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: '5.0'
19
+ version: '5.2'
20
20
  - - "<"
21
21
  - !ruby/object:Gem::Version
22
- version: '6.1'
22
+ version: '7.1'
23
23
  type: :runtime
24
24
  prerelease: false
25
25
  version_requirements: !ruby/object:Gem::Requirement
26
26
  requirements:
27
27
  - - ">="
28
28
  - !ruby/object:Gem::Version
29
- version: '5.0'
29
+ version: '5.2'
30
30
  - - "<"
31
31
  - !ruby/object:Gem::Version
32
- version: '6.1'
32
+ version: '7.1'
33
33
  - !ruby/object:Gem::Dependency
34
34
  name: ruby2_keywords
35
35
  requirement: !ruby/object:Gem::Requirement
@@ -44,6 +44,20 @@ dependencies:
44
44
  - - "~>"
45
45
  - !ruby/object:Gem::Version
46
46
  version: 0.0.2
47
+ - !ruby/object:Gem::Dependency
48
+ name: parallel
49
+ requirement: !ruby/object:Gem::Requirement
50
+ requirements:
51
+ - - "~>"
52
+ - !ruby/object:Gem::Version
53
+ version: '1.0'
54
+ type: :runtime
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ requirements:
58
+ - - "~>"
59
+ - !ruby/object:Gem::Version
60
+ version: '1.0'
47
61
  - !ruby/object:Gem::Dependency
48
62
  name: appraisal
49
63
  requirement: !ruby/object:Gem::Requirement
@@ -78,14 +92,14 @@ dependencies:
78
92
  requirements:
79
93
  - - "~>"
80
94
  - !ruby/object:Gem::Version
81
- version: '1.1'
95
+ version: '1.3'
82
96
  type: :development
83
97
  prerelease: false
84
98
  version_requirements: !ruby/object:Gem::Requirement
85
99
  requirements:
86
100
  - - "~>"
87
101
  - !ruby/object:Gem::Version
88
- version: '1.1'
102
+ version: '1.3'
89
103
  - !ruby/object:Gem::Dependency
90
104
  name: database_cleaner
91
105
  requirement: !ruby/object:Gem::Requirement
@@ -196,14 +210,14 @@ dependencies:
196
210
  requirements:
197
211
  - - "~>"
198
212
  - !ruby/object:Gem::Version
199
- version: 0.17.0
213
+ version: '0.21'
200
214
  type: :development
201
215
  prerelease: false
202
216
  version_requirements: !ruby/object:Gem::Requirement
203
217
  requirements:
204
218
  - - "~>"
205
219
  - !ruby/object:Gem::Version
206
- version: 0.17.0
220
+ version: '0.21'
207
221
  - !ruby/object:Gem::Dependency
208
222
  name: timecop
209
223
  requirement: !ruby/object:Gem::Requirement
@@ -218,6 +232,20 @@ dependencies:
218
232
  - - "~>"
219
233
  - !ruby/object:Gem::Version
220
234
  version: '0.9'
235
+ - !ruby/object:Gem::Dependency
236
+ name: psych
237
+ requirement: !ruby/object:Gem::Requirement
238
+ requirements:
239
+ - - "~>"
240
+ - !ruby/object:Gem::Version
241
+ version: '3.3'
242
+ type: :development
243
+ prerelease: false
244
+ version_requirements: !ruby/object:Gem::Requirement
245
+ requirements:
246
+ - - "~>"
247
+ - !ruby/object:Gem::Version
248
+ version: '3.3'
221
249
  description: Migrations and model helpers for creating and managing PostgreSQL 10
222
250
  partitions
223
251
  email:
@@ -235,6 +263,7 @@ files:
235
263
  - lib/pg_party/cache.rb
236
264
  - lib/pg_party/config.rb
237
265
  - lib/pg_party/hacks/postgresql_database_tasks.rb
266
+ - lib/pg_party/model/hash_methods.rb
238
267
  - lib/pg_party/model/list_methods.rb
239
268
  - lib/pg_party/model/methods.rb
240
269
  - lib/pg_party/model/range_methods.rb
@@ -254,7 +283,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
254
283
  requirements:
255
284
  - - ">="
256
285
  - !ruby/object:Gem::Version
257
- version: 2.5.0
286
+ version: 2.7.0
258
287
  required_rubygems_version: !ruby/object:Gem::Requirement
259
288
  requirements:
260
289
  - - ">="