RubyGems - pgslice - Versions diffs - 0.4.5 → 0.4.6 - Mend

pgslice 0.4.5 → 0.4.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (10) hide show

checksums.yaml CHANGED

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 21d2ab39e5d79cc2477aedc2f458f5e42341c76387fcdd97dd7865cfaec777f8
-  data.tar.gz: 9df6923ae6aba4054c6631a3d0c5c34c8de82214db0d0d6644e73002b9224702
+  metadata.gz: 30d1c5d3ed842d1250761b64f3da34a6e25e0ea17d7689d1eca9f1f0d4db1b8f
+  data.tar.gz: 82cbeccebcbdc6edeff9808d9013635d880aa8af20cb462722bdfe0fca5732a1
 SHA512:
-  metadata.gz: eb51a60758f6179337c74052bac2ba433efc65ec211ccbcbf9175f9986df9e45f3f16560669a29fd6e9b3684377164dea622d5e39e26ac0f442b441e92953f72
-  data.tar.gz: fc16788a99b3f307d99dd1d5230cb53e5746241e68cf3028454301f48cd711332b71d4716ab1210c07ca9a86daa07df342d009b306f24cc6245a9efbcb6fb32b
+  metadata.gz: 1db66bb9409d26d39952230c127d5edc900cf6070915d8a58ef4bec94a460d2db4588991fc2ed46e3c9205e904c424c6a379711c0ffc985e1723623f2eed3835
+  data.tar.gz: 60612ea69df38bcc43f27e492f93ca96f17dc9528236f3e107ea1e7990b5df372dd7d799ea1bdebd6ae3064903a63e9d086668208ba6d3e5958b6247d237df43

data/CHANGELOG.md CHANGED

@@ -1,122 +1,127 @@
-## 0.4.5
+## 0.4.6 (2020-05-29)
+- Ensure correct order with multi-column primary keys
+- Ensure fill always uses correct date range (bug introduced in 0.4.5)
+## 0.4.5 (2018-10-18)
 - Added support for Postgres 11 foreign key improvements
 - Improved versioning
-## 0.4.4
+## 0.4.4 (2018-08-18)
 - Added partitioning by `year`
 - Fixed `--source-table` and `--dest-table` options
 - Added descriptions to options
-## 0.4.3
+## 0.4.3 (2018-08-16)
 - Fixed sequence ownership
 - Improved help
-## 0.4.2
+## 0.4.2 (2018-07-23)
 - Added support for Postgres 11 index improvements
 - Added support for all connection options
-## 0.4.1
+## 0.4.1 (2018-04-30)
 - Better support for schemas
 - Use latest partition for schema
 - Added support for composite primary keys
-## 0.4.0
+## 0.4.0 (2017-10-07)
 - Added support for declarative partitioning
 - Added support for foreign keys
-## 0.3.6
+## 0.3.6 (2017-07-10)
 - Fixed drop trigger on `unprep` for non-lowercase tables
 - Fixed index creation for non-lowercase tables
-## 0.3.5
+## 0.3.5 (2017-07-06)
 - Added support for non-lowercase tables and columns
-## 0.3.4
+## 0.3.4 (2017-07-06)
 - Added `analyze` method
 - Fixed `fill` with `--dry-run` option
 - Better error message for tables without primary key
-## 0.3.3
+## 0.3.3 (2017-03-22)
 - Fixed error when creating partitions
-## 0.3.2
+## 0.3.2 (2016-12-15)
 - Exit with error code on interrupt
 - Fixed `--start` option with `--swapped`
-## 0.3.1
+## 0.3.1 (2016-12-13)
 - Fixed exception with `--no-partition` option
 - Use proper cast type in `fill` method for legacy `timestamptz` columns
-## 0.3.0
+## 0.3.0 (2016-12-12)
 - Better query performance for `timestamptz` columns
 - Added support for schemas other than `public`
-## 0.2.3
+## 0.2.3 (2016-10-10)
 - Added `--dest-table` option to `fill`
 - Fixed errors with `fill` when no partitions created
-## 0.2.2
+## 0.2.2 (2016-10-06)
 - Set `lock_timeout` on `swap` to prevent bad things from happening
 - Friendlier error messages
-## 0.2.1
+## 0.2.1 (2016-09-28)
 - Added `--where` option to `fill`
 - Fixed partition detection with `fill`
 - Fixed error for columns named `user` with `fill`
-## 0.2.0
+## 0.2.0 (2016-09-22)
 - Switched to new trigger, which is about 20% faster
-## 0.1.7
+## 0.1.7 (2016-09-14)
 - Added `--source-table` option to `fill`
-## 0.1.6
+## 0.1.6 (2016-08-04)
 - Added `--no-partition` option to `prep`
 - Added `--url` option
-## 0.1.5
+## 0.1.5 (2016-04-26)
 - Removed `activesupport` dependency for speed
 - Fixed `fill` for months
-## 0.1.4
+## 0.1.4 (2016-04-24)
 - Added sequence ownership
 - Default to 0 for `--past` and `--future` options
 - Better `fill` with `--swapped`
-## 0.1.3
+## 0.1.3 (2016-04-24)
 - Fixed table inheritance
-## 0.1.2
+## 0.1.2 (2016-04-24)
 - Added `--dry-run` option
 - Print sql to stdout instead of stderr
-## 0.1.1
+## 0.1.1 (2016-04-24)
 - Added sql commands to output
-## 0.1.0
+## 0.1.0 (2016-04-24)
 - First release

data/LICENSE.txt CHANGED

@@ -1,6 +1,6 @@
 The MIT License (MIT)
-Copyright (c) 2016-2018 Andrew Kane
+Copyright (c) 2016-2020 Andrew Kane
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md CHANGED

@@ -14,7 +14,7 @@ pgslice is a command line tool. To install, run:
 gem install pgslice
 ```
-This will give you the `pgslice` command.
+This will give you the `pgslice` command. If installation fails, you may need to install [dependencies](#dependencies).
 ## Steps
@@ -34,16 +34,14 @@ This will give you the `pgslice` command.
   Period can be `day`, `month`, or `year`.
-  This creates a table named `<table>_intermediate` with the appropriate trigger for partitioning.
+  This creates a partitioned table named `<table>_intermediate`.
-4. Add partitions
+4. Add partitions to the intermediate table
   ```sh
   pgslice add_partitions <table> --intermediate --past 3 --future 3
   ```
-  This creates child tables that inherit from the intermediate table.
   Use the `--past` and `--future` options to control the number of partitions.
 5. *Optional, for tables with data* - Fill the partitions in batches with data from the original table
@@ -80,7 +78,7 @@ This will give you the `pgslice` command.
   ```sql
   pg_dump -c -Fc -t <table>_retired $PGSLICE_URL > <table>_retired.dump
-  psql -c "DROP <table>_retired" $PGSLICE_URL
+  psql -c "DROP TABLE <table>_retired" $PGSLICE_URL
   ```
 ## Sample Output
@@ -220,7 +218,7 @@ Back up and drop older partitions each day, month, or year.
 ```sh
 pg_dump -c -Fc -t <table>_201809 $PGSLICE_URL > <table>_201809.dump
-psql -c "DROP <table>_201809" $PGSLICE_URL
+psql -c "DROP TABLE <table>_201809" $PGSLICE_URL
 ```
 If you use [Amazon S3](https://aws.amazon.com/s3/) for backups, [s3cmd](https://github.com/s3tools/s3cmd) is a nice tool.
@@ -273,6 +271,26 @@ Before Postgres 10, if you use `INSERT` statements with a `RETURNING` clause (as
 1. Insert directly into the partition
 2. Get value before the insert with `SELECT nextval('sequence_name')` (for multiple rows, append `FROM generate_series(1, n)`)
+## Rails
+For Postgres 10+, specify the primary key for partitioned models to ensure it’s returned.
+```ruby
+class Visit < ApplicationRecord
+  self.primary_key = "id"
+end
+```
+Before Postgres 10, preload the value.
+```ruby
+class Visit < ApplicationRecord
+  before_create do
+    self.id ||= self.class.connection.select_all("SELECT nextval('#{self.class.sequence_name}')").first["nextval"]
+  end
+end
+```
 ## One Off Tasks
 You can also use pgslice to reduce the size of a table without partitioning by creating a new table, filling it with a subset of records, and swapping it in.
 Once a table is partitioned, here’s how to change the schema:
-- To add, remove, or modify a column, make the update on the master table only.
-- To add or remove an index or foreign key, make the update on the master table. For Postgres < 11, make the update on all partitions as well.
+To add, remove, or modify a column, make the update on the master table only.
+To add or remove an index or foreign key:
+- For Postgres 11+, make the update on the master table only.
+- For Postgres 10, make the update on partitions only.
+- For Postgres < 10, make the update on the master table and all partitions.
 ## Declarative Partitioning
@@ -296,7 +319,23 @@ Postgres 10 introduces [declarative partitioning](https://www.postgresql.org/doc
 ## Data Protection
-Always make sure your connection is secure when connecting to your database over a network you don’t fully trust. Your best option is to connect over SSH or a VPN. Another option is to use `sslmode=verify-full`. If you don’t do this, your database credentials can be compromised.
+Always make sure your [connection is secure](https://ankane.org/postgres-sslmode-explained) when connecting to a database over a network you don’t fully trust. Your best option is to connect over SSH or a VPN. Another option is to use `sslmode=verify-full`. If you don’t do this, your database credentials can be compromised.
+## Dependencies
+If installation fails, your system may be missing Ruby or libpq.
+On Mac, run:
+```sh
+brew install postgresql
+```
+On Ubuntu, run:
+```sh
+sudo apt-get install ruby-dev libpq-dev build-essential
+```
 ## Upgrading
@@ -331,6 +370,7 @@ This will give you the `pgslice` command.
 Also check out:
+- [Dexter](https://github.com/ankane/dexter) - The automatic indexer for Postgres
 - [PgHero](https://github.com/ankane/pghero) - A performance dashboard for Postgres
 - [pgsync](https://github.com/ankane/pgsync) - Sync Postgres data to your local machine
@@ -343,7 +383,7 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
 - Write, clarify, or fix documentation
 - Suggest or add new features
-To run tests, do:
+To get started with development and testing:
 ```sh
 git clone https://github.com/ankane/pgslice.git
@@ -1,8 +1,7 @@
 #!/usr/bin/env ruby
+# handle interrupts
+trap("SIGINT") { abort }
 require "pgslice"
-begin
-  PgSlice::CLI.start
-rescue Interrupt
-  abort
-end
+PgSlice::CLI.start
@@ -1,7 +1,9 @@
 # dependencies
-require "cgi"
 require "pg"
 require "thor"
+# stdlib
+require "cgi"
 require "time"
 # modules
@@ -14,7 +14,7 @@ module PgSlice
     map %w[--version -v] => :version
     def self.exit_on_failure?
-      true
+      ENV["PGSLICE_ENV"] != "test"
     end
     def initialize(*args)
@@ -33,6 +33,7 @@ module PgSlice
         WHERE s.relkind = 'S'
           AND n.nspname = $1
           AND t.relname = $2
+        ORDER BY s.relname ASC
       SQL
       execute(query, [schema, name])
     end
@@ -41,12 +42,15 @@ module PgSlice
       execute("SELECT pg_get_constraintdef(oid) FROM pg_constraint WHERE conrelid = #{regclass} AND contype ='f'").map { |r| r["pg_get_constraintdef"] }
     end
-    # http://stackoverflow.com/a/20537829
+    # https://stackoverflow.com/a/20537829
+    # TODO can simplify with array_position in Postgres 9.5+
     def primary_key
       query = <<-SQL
         SELECT
           pg_attribute.attname,
-          format_type(pg_attribute.atttypid, pg_attribute.atttypmod)
+          format_type(pg_attribute.atttypid, pg_attribute.atttypmod),
+          pg_attribute.attnum,
+          pg_index.indkey
         FROM
           pg_index, pg_class, pg_attribute, pg_namespace
         WHERE
@@ -58,7 +62,8 @@ module PgSlice
           pg_attribute.attnum = any(pg_index.indkey) AND
           indisprimary
       SQL
-      execute(query, [schema, name]).map { |r| r["attname"] }
+      rows = execute(query, [schema, name])
+      rows.sort_by { |r| r["indkey"].split(" ").index(r["attnum"]) }.map { |r| r["attname"] }
     end
     def index_defs
@@ -117,6 +122,7 @@ module PgSlice
         WHERE
           nmsp_parent.nspname = $1 AND
           parent.relname = $2
+        ORDER BY child.relname ASC
       SQL
       execute(query, [schema, name]).map { |r| Table.new(r["schema"], r["name"]) }
     end
@@ -144,7 +150,7 @@ module PgSlice
         function_def = execute("SELECT pg_get_functiondef(oid) FROM pg_proc WHERE proname = $1", [trigger_name])[0]
         return [] unless function_def
         function_def = function_def["pg_get_functiondef"]
-        sql_format = SQL_FORMAT.find { |_, f| function_def.include?("'#{f}'") }
+        sql_format = Helpers::SQL_FORMAT.find { |_, f| function_def.include?("'#{f}'") }
         return [] unless sql_format
         period = sql_format[0]
         field = /to_char\(NEW\.(\w+),/.match(function_def)[1]
@@ -1,3 +1,3 @@
 module PgSlice
-  VERSION = "0.4.5"
+  VERSION = "0.4.6"
 end
@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pgslice
 version: !ruby/object:Gem::Version
-  version: 0.4.5
+  version: 0.4.6
 platform: ruby
 authors:
 - Andrew Kane
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2018-10-18 00:00:00.000000000 Z
+date: 2020-05-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
@@ -122,8 +122,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
-rubygems_version: 2.7.7
+rubygems_version: 3.1.2
 signing_key:
 specification_version: 4
 summary: Postgres partitioning as easy as pie