RubyGems - pgslice - Versions diffs - 0.4.5 → 0.4.6 - Mend

pgslice 0.4.5 → 0.4.6

Files changed (10) hide show

checksums.yaml CHANGED

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 21d2ab39e5d79cc2477aedc2f458f5e42341c76387fcdd97dd7865cfaec777f8
-  data.tar.gz: 9df6923ae6aba4054c6631a3d0c5c34c8de82214db0d0d6644e73002b9224702
+  metadata.gz: 30d1c5d3ed842d1250761b64f3da34a6e25e0ea17d7689d1eca9f1f0d4db1b8f
+  data.tar.gz: 82cbeccebcbdc6edeff9808d9013635d880aa8af20cb462722bdfe0fca5732a1
 SHA512:
-  metadata.gz: eb51a60758f6179337c74052bac2ba433efc65ec211ccbcbf9175f9986df9e45f3f16560669a29fd6e9b3684377164dea622d5e39e26ac0f442b441e92953f72
-  data.tar.gz: fc16788a99b3f307d99dd1d5230cb53e5746241e68cf3028454301f48cd711332b71d4716ab1210c07ca9a86daa07df342d009b306f24cc6245a9efbcb6fb32b
+  metadata.gz: 1db66bb9409d26d39952230c127d5edc900cf6070915d8a58ef4bec94a460d2db4588991fc2ed46e3c9205e904c424c6a379711c0ffc985e1723623f2eed3835
+  data.tar.gz: 60612ea69df38bcc43f27e492f93ca96f17dc9528236f3e107ea1e7990b5df372dd7d799ea1bdebd6ae3064903a63e9d086668208ba6d3e5958b6247d237df43

data/CHANGELOG.md CHANGED

@@ -1,122 +1,127 @@
-## 0.4.5
+## 0.4.6 (2020-05-29)
+- Ensure correct order with multi-column primary keys
+- Ensure fill always uses correct date range (bug introduced in 0.4.5)
+## 0.4.5 (2018-10-18)
 - Added support for Postgres 11 foreign key improvements
 - Improved versioning
-## 0.4.4
+## 0.4.4 (2018-08-18)
 - Added partitioning by `year`
 - Fixed `--source-table` and `--dest-table` options
 - Added descriptions to options
-## 0.4.3
+## 0.4.3 (2018-08-16)
 - Fixed sequence ownership
 - Improved help
-## 0.4.2
+## 0.4.2 (2018-07-23)
 - Added support for Postgres 11 index improvements
 - Added support for all connection options
-## 0.4.1
+## 0.4.1 (2018-04-30)
 - Better support for schemas
 - Use latest partition for schema
 - Added support for composite primary keys
-## 0.4.0
+## 0.4.0 (2017-10-07)
 - Added support for declarative partitioning
 - Added support for foreign keys
-## 0.3.6
+## 0.3.6 (2017-07-10)
 - Fixed drop trigger on `unprep` for non-lowercase tables
 - Fixed index creation for non-lowercase tables
-## 0.3.5
+## 0.3.5 (2017-07-06)
 - Added support for non-lowercase tables and columns
-## 0.3.4
+## 0.3.4 (2017-07-06)
 - Added `analyze` method
 - Fixed `fill` with `--dry-run` option
 - Better error message for tables without primary key
-## 0.3.3
+## 0.3.3 (2017-03-22)
 - Fixed error when creating partitions
-## 0.3.2
+## 0.3.2 (2016-12-15)
 - Exit with error code on interrupt
 - Fixed `--start` option with `--swapped`
-## 0.3.1
+## 0.3.1 (2016-12-13)
 - Fixed exception with `--no-partition` option
 - Use proper cast type in `fill` method for legacy `timestamptz` columns
-## 0.3.0
+## 0.3.0 (2016-12-12)
 - Better query performance for `timestamptz` columns
 - Added support for schemas other than `public`
-## 0.2.3
+## 0.2.3 (2016-10-10)
 - Added `--dest-table` option to `fill`
 - Fixed errors with `fill` when no partitions created
-## 0.2.2
+## 0.2.2 (2016-10-06)
 - Set `lock_timeout` on `swap` to prevent bad things from happening
 - Friendlier error messages
-## 0.2.1
+## 0.2.1 (2016-09-28)
 - Added `--where` option to `fill`
 - Fixed partition detection with `fill`
 - Fixed error for columns named `user` with `fill`
-## 0.2.0
+## 0.2.0 (2016-09-22)
 - Switched to new trigger, which is about 20% faster
-## 0.1.7
+## 0.1.7 (2016-09-14)
 - Added `--source-table` option to `fill`
-## 0.1.6
+## 0.1.6 (2016-08-04)
 - Added `--no-partition` option to `prep`
 - Added `--url` option
-## 0.1.5
+## 0.1.5 (2016-04-26)
 - Removed `activesupport` dependency for speed
 - Fixed `fill` for months
-## 0.1.4
+## 0.1.4 (2016-04-24)
 - Added sequence ownership
 - Default to 0 for `--past` and `--future` options
 - Better `fill` with `--swapped`
-## 0.1.3
+## 0.1.3 (2016-04-24)
 - Fixed table inheritance
-## 0.1.2
+## 0.1.2 (2016-04-24)
 - Added `--dry-run` option
 - Print sql to stdout instead of stderr
-## 0.1.1
+## 0.1.1 (2016-04-24)
 - Added sql commands to output
-## 0.1.0
+## 0.1.0 (2016-04-24)
 - First release

data/LICENSE.txt CHANGED

@@ -1,6 +1,6 @@
 The MIT License (MIT)
-Copyright (c) 2016-2018 Andrew Kane
+Copyright (c) 2016-2020 Andrew Kane
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md CHANGED

@@ -14,7 +14,7 @@ pgslice is a command line tool. To install, run:
 gem install pgslice
 ```
-This will give you the `pgslice` command.
+This will give you the `pgslice` command. If installation fails, you may need to install [dependencies](#dependencies).
 ## Steps
@@ -34,16 +34,14 @@ This will give you the `pgslice` command.
   Period can be `day`, `month`, or `year`.
-  This creates a table named `<table>_intermediate` with the appropriate trigger for partitioning.
+  This creates a partitioned table named `<table>_intermediate`.
-4. Add partitions
+4. Add partitions to the intermediate table
   ```sh
   pgslice add_partitions <table> --intermediate --past 3 --future 3
   ```
-  This creates child tables that inherit from the intermediate table.
   Use the `--past` and `--future` options to control the number of partitions.
 5. *Optional, for tables with data* - Fill the partitions in batches with data from the original table
@@ -80,7 +78,7 @@ This will give you the `pgslice` command.
   ```sql
   pg_dump -c -Fc -t <table>_retired $PGSLICE_URL > <table>_retired.dump
-  psql -c "DROP <table>_retired" $PGSLICE_URL
+  psql -c "DROP TABLE <table>_retired" $PGSLICE_URL
   ```
 ## Sample Output
@@ -220,7 +218,7 @@ Back up and drop older partitions each day, month, or year.
 ```sh
 pg_dump -c -Fc -t <table>_201809 $PGSLICE_URL > <table>_201809.dump
-psql -c "DROP <table>_201809" $PGSLICE_URL
+psql -c "DROP TABLE <table>_201809" $PGSLICE_URL
 ```
 If you use [Amazon S3](https://aws.amazon.com/s3/) for backups, [s3cmd](https://github.com/s3tools/s3cmd) is a nice tool.
@@ -273,6 +271,26 @@ Before Postgres 10, if you use `INSERT` statements with a `RETURNING` clause (as
 1. Insert directly into the partition
 2. Get value before the insert with `SELECT nextval('sequence_name')` (for multiple rows, append `FROM generate_series(1, n)`)
+## Rails
+For Postgres 10+, specify the primary key for partitioned models to ensure it’s returned.
+```ruby
+class Visit < ApplicationRecord
+  self.primary_key = "id"
+end
+```
+Before Postgres 10, preload the value.
+```ruby
+class Visit < ApplicationRecord
+  before_create do
+    self.id ||= self.class.connection.select_all("SELECT nextval('#{self.class.sequence_name}')").first["nextval"]
+  end
+end
+```
 ## One Off Tasks
 You can also use pgslice to reduce the size of a table without partitioning by creating a new table, filling it with a subset of records, and swapping it in.
 Once a table is partitioned, here’s how to change the schema:
-- To add, remove, or modify a column, make the update on the master table only.
-- To add or remove an index or foreign key, make the update on the master table. For Postgres < 11, make the update on all partitions as well.
+To add, remove, or modify a column, make the update on the master table only.
+To add or remove an index or foreign key:
+- For Postgres 11+, make the update on the master table only.
+- For Postgres 10, make the update on partitions only.
+- For Postgres < 10, make the update on the master table and all partitions.
 ## Declarative Partitioning
@@ -296,7 +319,23 @@ Postgres 10 introduces [declarative partitioning](https://www.postgresql.org/doc
 ## Data Protection
-Always make sure your connection is secure when connecting to your database over a network you don’t fully trust. Your best option is to connect over SSH or a VPN. Another option is to use `sslmode=verify-full`. If you don’t do this, your database credentials can be compromised.
+Always make sure your [connection is secure](https://ankane.org/postgres-sslmode-explained) when connecting to a database over a network you don’t fully trust. Your best option is to connect over SSH or a VPN. Another option is to use `sslmode=verify-full`. If you don’t do this, your database credentials can be compromised.
+## Dependencies
+If installation fails, your system may be missing Ruby or libpq.
+On Mac, run:
+```sh
+brew install postgresql
+```
+On Ubuntu, run:
+```sh
+sudo apt-get install ruby-dev libpq-dev build-essential
+```
 ## Upgrading
@@ -331,6 +370,7 @@ This will give you the `pgslice` command.
 Also check out:
+- [Dexter](https://github.com/ankane/dexter) - The automatic indexer for Postgres
 - [PgHero](https://github.com/ankane/pghero) - A performance dashboard for Postgres
 - [pgsync](https://github.com/ankane/pgsync) - Sync Postgres data to your local machine
@@ -343,7 +383,7 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
 - Write, clarify, or fix documentation
 - Suggest or add new features
-To run tests, do:
+To get started with development and testing:
 ```sh
 git clone https://github.com/ankane/pgslice.git
@@ -1,8 +1,7 @@
 #!/usr/bin/env ruby
+# handle interrupts
+trap("SIGINT") { abort }
 require "pgslice"
-begin
-  PgSlice::CLI.start
-rescue Interrupt
-  abort
-end
+PgSlice::CLI.start
@@ -1,7 +1,9 @@
 # dependencies
-require "cgi"
 require "pg"
 require "thor"
+# stdlib
+require "cgi"
 require "time"
 # modules
@@ -14,7 +14,7 @@ module PgSlice
     map %w[--version -v] => :version
     def self.exit_on_failure?
-      true
+      ENV["PGSLICE_ENV"] != "test"
     end
     def initialize(*args)
@@ -33,6 +33,7 @@ module PgSlice
         WHERE s.relkind = 'S'
           AND n.nspname = $1
           AND t.relname = $2
+        ORDER BY s.relname ASC
       SQL
       execute(query, [schema, name])
     end
@@ -41,12 +42,15 @@ module PgSlice
       execute("SELECT pg_get_constraintdef(oid) FROM pg_constraint WHERE conrelid = #{regclass} AND contype ='f'").map { |r| r["pg_get_constraintdef"] }
     end
-    # http://stackoverflow.com/a/20537829
+    # https://stackoverflow.com/a/20537829
+    # TODO can simplify with array_position in Postgres 9.5+
     def primary_key
       query = <<-SQL
         SELECT
           pg_attribute.attname,
-          format_type(pg_attribute.atttypid, pg_attribute.atttypmod)
+          format_type(pg_attribute.atttypid, pg_attribute.atttypmod),
+          pg_attribute.attnum,
+          pg_index.indkey
         FROM
           pg_index, pg_class, pg_attribute, pg_namespace
         WHERE
@@ -58,7 +62,8 @@ module PgSlice
           pg_attribute.attnum = any(pg_index.indkey) AND
           indisprimary
       SQL
-      execute(query, [schema, name]).map { |r| r["attname"] }
+      rows = execute(query, [schema, name])
+      rows.sort_by { |r| r["indkey"].split(" ").index(r["attnum"]) }.map { |r| r["attname"] }
     end
     def index_defs
@@ -117,6 +122,7 @@ module PgSlice
         WHERE
           nmsp_parent.nspname = $1 AND
           parent.relname = $2
+        ORDER BY child.relname ASC
       SQL
       execute(query, [schema, name]).map { |r| Table.new(r["schema"], r["name"]) }
     end
@@ -144,7 +150,7 @@ module PgSlice
         function_def = execute("SELECT pg_get_functiondef(oid) FROM pg_proc WHERE proname = $1", [trigger_name])[0]
         return [] unless function_def
         function_def = function_def["pg_get_functiondef"]
-        sql_format = SQL_FORMAT.find { |_, f| function_def.include?("'#{f}'") }
+        sql_format = Helpers::SQL_FORMAT.find { |_, f| function_def.include?("'#{f}'") }
         return [] unless sql_format
         period = sql_format[0]
         field = /to_char\(NEW\.(\w+),/.match(function_def)[1]
@@ -1,3 +1,3 @@
 module PgSlice
-  VERSION = "0.4.5"
+  VERSION = "0.4.6"
 end
@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pgslice
 version: !ruby/object:Gem::Version
-  version: 0.4.5
+  version: 0.4.6
 platform: ruby
 authors:
 - Andrew Kane
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2018-10-18 00:00:00.000000000 Z
+date: 2020-05-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
@@ -122,8 +122,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
-rubygems_version: 2.7.7
+rubygems_version: 3.1.2
 signing_key:
 specification_version: 4
 summary: Postgres partitioning as easy as pie