pgbackups-archive 0.3.0 → 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 0b1dfe7b8843a859e5eff6c7bac691d919a647d0
4
- data.tar.gz: 506782bd0780f16a5a2ea1c9729aa74d8065b486
3
+ metadata.gz: 8c51d872d12dc01426e3a9185e63a118b204ecdd
4
+ data.tar.gz: 92b49a4dcbb9ef8eae5d2ed1f6a9892bfa01f52b
5
5
  SHA512:
6
- metadata.gz: 0edd75f6f73cbd37b5fd5be1e6c78a2c517c7c28841cead96a6c4489951cdd806c5d04792ec5bd4f05a97018c2aa685bef884b6617bad32a808d5e0a806f3c13
7
- data.tar.gz: a7e4855f4e3f3541457d7cc0356b9972bdd4f710d013caf9aa35e460f46964d5b99eb4e5b24b0fee3fd8232486abd6b20777c52748e7cc1928b7fe79f969fc34
6
+ metadata.gz: 409279267d694930ee6e35452bdfdbb23cf5955769be967ea9fef96fe133683306bf541623a969e65823245559c84ae448b52466f8493371017228ffc5f817a7
7
+ data.tar.gz: 021689847fb1cbade0f0c45829d1a351ed2d89dd6e10df572ec7e5fd09dc10ab6418bca37fd092c591fbbe9e3ac5286bd870b1b198406228f9d357967d291584
@@ -1,4 +1,4 @@
1
- Copyright 2014 Kenny Johnston
1
+ Copyright 2015 Kenny Johnston
2
2
 
3
3
  Permission is hereby granted, free of charge, to any person obtaining
4
4
  a copy of this software and associated documentation files (the
data/README.md CHANGED
@@ -3,66 +3,64 @@
3
3
  [![Gem Version](https://badge.fury.io/rb/pgbackups-archive.svg)](http://badge.fury.io/rb/pgbackups-archive)
4
4
  [![Code Climate](https://codeclimate.com/github/kjohnston/pgbackups-archive/badges/gpa.svg)](https://codeclimate.com/github/kjohnston/pgbackups-archive)
5
5
 
6
- A means of automating Heroku's pgbackups and archiving them to Amazon S3.
6
+ A means of automating Heroku PGBackups and archiving them to Amazon S3.
7
+
8
+ ## Upgrade Alert
9
+
10
+ As of v1.0.0, `pgbackups-archive` works with the new [Heroku PGBackups](https://devcenter.heroku.com/articles/heroku-postgres-backups) service, which replaced the older [PG Backups](https://devcenter.heroku.com/articles/pgbackups) add-on.
11
+
12
+ If you're still using an older version of `pgbackups-archive`, it's time to upgrade!
13
+
14
+ Read more about this transition in Heroku's offerings on the Heroku Blog: [PG Backups Levels Up](https://blog.heroku.com/archives/2015/3/11/pgbackups-levels-up)
15
+
16
+ Please note that the environment variables that need to be defined have changed with
17
+ v1.0.0.
7
18
 
8
19
  ## Overview
9
20
 
10
- The `pgbackups:archive` rake task this gem provides will capture a pgbackup, wait for it to complete, then store it within the Amazon S3 bucket you specify. This rake task can be scheduled via the Heroku Scheduler, thus producing automated, offsite, backups.
21
+ The `pgbackups:archive` rake task this gem provides will capture a Heroku PGBackup, wait for it to complete, then store it within the Amazon S3 bucket you specify. This rake task can be scheduled via the Heroku Scheduler, thus producing automated, offsite, backups.
22
+
23
+ This gem doesn't interfere with or utilze automated backups, so feel free to schedule those with the `pg:backups schedule` command as you desire.
11
24
 
12
- The rake task will use pgbackups' `--expire` flag to remove the oldest pgbackup Heroku is storing when there are no free slots remaining.
25
+ You can configure how many manual backups (created by you or this gem) you'd like to keep at the Heroku PGBackups level to ensure there is always space to capture a new backup.
13
26
 
14
27
  You can configure retention settings at the Amazon S3 bucket level from within the AWS Console if you like.
15
28
 
16
29
  ## Use
17
30
 
18
- ### Determine which Heroku app to run the task under
19
-
20
- #### Option 1 - Add `pgbackups-archive` to your existing application
21
-
22
- Recommended for apps with light memory usage.
31
+ ### Install the gem
23
32
 
24
33
  Add the gem to your Gemfile and bundle:
25
34
 
26
35
  gem "pgbackups-archive"
27
36
  bundle install
28
37
 
29
- #### Option 2 - Add `pgbackups-archive` to a standalone application
30
-
31
- Recommended for apps with heavier memory usage.
32
-
33
- * Create a new repo with `pgackups-archive` added to that app's Gemfile and push
34
- it to a new Heroku app.
35
- * Ensure the `PGBACKUPS_DATABASE_URL` environment variable you set for your
36
- backup app points to your main app's `DATABASE_URL`, or other follower URL, so
37
- that `pgbackups-archive` in your backup app knows to backup your real app's
38
- database.
38
+ ### Install Heroku Scheduler add-on
39
39
 
40
- This option exists because the streaming download & upload of the backup file
41
- will utilize a certain amount of memory beyond what an instance of your
42
- application uses and if you're close to the threshold of your Dyno size as it
43
- is, this increment could put the instance over the limit and cause it to
44
- encounter [memory allocation errors](https://devcenter.heroku.com/articles/error-codes#r14-memory-quota-exceeded).
45
- By running a dedicated Heroku app to run
46
- `pgbackups-archive` the task will have ample room at the 1X Dyno level to stream
47
- the backup files.
40
+ heroku addons:add scheduler:standard
48
41
 
49
- ### Install Heroku addons
42
+ ### Setup an AWS IAM user, S3 bucket and policy
50
43
 
51
- heroku addons:add pgbackups
52
- heroku addons:add scheduler:standard
44
+ A good security measure would be to use a dedicated set of AWS credentials with a security policy only allowing access to the bucket you're specifying. See this Pro Tip on [Assigning an AWS IAM user access to a single S3 bucket](http://coderwall.com/p/dwhlma).
53
45
 
54
- ### Apply environment variables
46
+ ### Apply Environment Variables
55
47
 
48
+ # Required
49
+ heroku config:add HEROKU_API_KEY="collaborator-api-key"
50
+ heroku config:add PGBACKUPS_APP="myapp"
56
51
  heroku config:add PGBACKUPS_AWS_ACCESS_KEY_ID="XXX"
57
52
  heroku config:add PGBACKUPS_AWS_SECRET_ACCESS_KEY="YYY"
58
53
  heroku config:add PGBACKUPS_BUCKET="myapp-backups"
59
54
  heroku config:add PGBACKUPS_REGION="us-west-2"
60
- heroku config:add PGBACKUPS_DATABASE_URL="your main app's DATABASE_URL or other follower URL here"
61
55
 
62
- * `PGBACKUPS_DATABASE_URL` can be set either to `DATABASE_URL` or a follower database you setup if you would prefer to not backup from your primary databse for performance reasons.
63
- * If `PGBACKUPS_DATABASE_URL` is omitted, `pgbackups-archive` will default to the `DATABASE_URL` of the Heroku app it runs under. This setting will be required going forward, so you'll want to have it set.
64
- * As mentioned above, the `PGBACKUPS_DATABASE_URL` is mandatory if you are the using Option 2 above.
65
- * A good security measure would be to use a dedicated set of AWS credentials with a security policy only allowing access to the bucket you're specifying. See this Pro Tip on [Assigning an AWS IAM user access to a single S3 bucket](http://coderwall.com/p/dwhlma).
56
+ # Optional: If you wish to backup a database other than the one that
57
+ # DATABASE_URL points to, set this to the name of the variable for that
58
+ # database (useful for follower databases).
59
+ heroku config:add PGBACKUPS_DATABASE="HEROKU_POSTGRESQL_BLACK_URL"
60
+
61
+ # Optional: If you wish to customize the number of manual backups kept at
62
+ # the Heroku PGBackups level, set this.
63
+ heroku config:add PGBACKUPS_KEEP="30"
66
64
 
67
65
  ### Add the rake task to scheduler
68
66
 
@@ -92,11 +90,11 @@ I shouldn't have to say this, but I will. Your backups are your responsibility.
92
90
 
93
91
  ## Contributing
94
92
 
95
- 1. [Fork it](https://github.com/kjohnston/pgbackups-archive/fork_select)
93
+ 1. Fork it
96
94
  2. Create your feature branch (`git checkout -b my-new-feature`)
97
95
  3. Commit your changes (`git commit -am 'Added some feature'`)
98
96
  4. Push to the branch (`git push origin my-new-feature`)
99
- 5. [Create a Pull Request](https://github.com/kjohnston/pgbackups-archive/pull/new)
97
+ 5. Create a Pull Request
100
98
 
101
99
  ## Contributors
102
100
 
@@ -1,5 +1,5 @@
1
1
  module PgbackupsArchive
2
- require "pgbackups-archive/heroku/client/pgbackups_archive"
2
+ require "pgbackups-archive/job"
3
3
  require "pgbackups-archive/railtie"
4
4
  require "pgbackups-archive/storage"
5
5
  end
@@ -0,0 +1,109 @@
1
+ require "heroku/command/pg"
2
+ require "heroku/command/pg_backups"
3
+ require "heroku/api"
4
+ require "tmpdir"
5
+
6
+ class PgbackupsArchive::Job
7
+
8
+ attr_reader :client
9
+ attr_accessor :backup_url, :created_at
10
+
11
+ def self.call
12
+ new.call
13
+ end
14
+
15
+ def initialize(attrs={})
16
+ Heroku::Command.load
17
+ @client = Heroku::Command::Pg.new([], :app => ENV["PGBACKUPS_APP"])
18
+ end
19
+
20
+ def call
21
+ expire
22
+ capture
23
+ download
24
+ archive
25
+ delete
26
+ end
27
+
28
+ def archive
29
+ if PgbackupsArchive::Storage.new(key, file).store
30
+ client.display "Backup archived"
31
+ end
32
+ end
33
+
34
+ def capture
35
+ attachment = client.send(:generate_resolver).resolve(database)
36
+ backup = client.send(:hpg_client, attachment).backups_capture
37
+ client.send(:poll_transfer, "backup", backup[:uuid])
38
+
39
+ self.created_at = backup[:created_at]
40
+
41
+ self.backup_url = Heroku::Client::HerokuPostgresqlApp
42
+ .new(ENV["PGBACKUPS_APP"]).transfers_public_url(backup[:num])[:url]
43
+ end
44
+
45
+ def delete
46
+ File.delete(temp_file)
47
+ end
48
+
49
+ def download
50
+ File.open(temp_file, "wb") do |output|
51
+ streamer = lambda do |chunk, remaining_bytes, total_bytes|
52
+ output.write chunk
53
+ end
54
+
55
+ # https://github.com/excon/excon/issues/475
56
+ Excon.get backup_url,
57
+ :response_block => streamer,
58
+ :omit_default_port => true
59
+ end
60
+ end
61
+
62
+ def expire
63
+ transfers = client.send(:hpg_app_client, ENV["PGBACKUPS_APP"]).transfers
64
+ .select { |b| b[:from_type] == "pg_dump" && b[:to_type] == "gof3r" }
65
+ .sort_by { |b| b[:created_at] }
66
+
67
+ if transfers.size > pgbackups_to_keep
68
+ backup_id = "b%03d" % transfers.first[:num]
69
+ backup_num = client.send(:backup_num, backup_id)
70
+
71
+ expire_backup(backup_num)
72
+
73
+ client.display "Backup #{backup_id} expired"
74
+ end
75
+ end
76
+
77
+ private
78
+
79
+ def expire_backup(backup_num)
80
+ client.send(:hpg_app_client, ENV["PGBACKUPS_APP"])
81
+ .transfers_delete(backup_num)
82
+ end
83
+
84
+ def database
85
+ ENV["PGBACKUPS_DATABASE"] || "DATABASE_URL"
86
+ end
87
+
88
+ def environment
89
+ defined?(Rails) ? Rails.env : nil
90
+ end
91
+
92
+ def file
93
+ File.open(temp_file, "r")
94
+ end
95
+
96
+ def key
97
+ timestamp = created_at.gsub(/\/|\:|\.|\s/, "-").concat(".dump")
98
+ ["pgbackups", environment, timestamp].compact.join("/")
99
+ end
100
+
101
+ def pgbackups_to_keep
102
+ var = ENV["PGBACKUPS_KEEP"] ? var.to_i : 30
103
+ end
104
+
105
+ def temp_file
106
+ "#{Dir.tmpdir}/pgbackup"
107
+ end
108
+
109
+ end
@@ -1,3 +1,3 @@
1
1
  module PgbackupsArchive
2
- VERSION = "0.3.0"
2
+ VERSION = "1.0.0"
3
3
  end
@@ -1,8 +1,8 @@
1
1
  namespace :pgbackups do
2
2
 
3
- desc "Perform a pgbackups backup then archive to S3."
3
+ desc "Capture a Heroku PGBackups backup and archive it to Amazon S3."
4
4
  task :archive do
5
- Heroku::Client::PgbackupsArchive.perform
5
+ PgbackupsArchive::Job.call
6
6
  end
7
7
 
8
8
  end
@@ -0,0 +1,131 @@
1
+ require "test_helper"
2
+
3
+ describe PgbackupsArchive::Job do
4
+
5
+ describe ".call" do
6
+ it { PgbackupsArchive::Job.must_respond_to(:call) }
7
+ end
8
+
9
+ describe "instance methods" do
10
+ before do
11
+ @app = "foobar"
12
+ ENV["PGBACKUPS_APP"] = @app
13
+ @job = PgbackupsArchive::Job.new
14
+ @client = @job.client
15
+ @backup_url = "https://raw.githubusercontent.com/kjohnston/pgbackups-archive/master/pgbackups-archive.gemspec"
16
+ @created_at = Time.now
17
+ end
18
+
19
+ describe "#initialize" do
20
+ it do
21
+ @client.must_be_kind_of(Heroku::Command::Pg)
22
+ @client.options[:app].must_equal @app
23
+ end
24
+ end
25
+
26
+ describe "#call" do
27
+ before do
28
+ @job.expects(:expire)
29
+ @job.expects(:capture)
30
+ @job.expects(:download)
31
+ @job.expects(:archive)
32
+ @job.expects(:delete)
33
+ end
34
+
35
+ it { @job.call }
36
+ end
37
+
38
+ describe "#archive" do
39
+ before do
40
+ @key = "some-key"
41
+ @file = "some-file"
42
+ @job.stubs(:key).returns(@key)
43
+ @job.stubs(:file).returns(@file)
44
+
45
+ PgbackupsArchive::Storage.expects(:new).with(@key, @file)
46
+ .returns(mock(store: true))
47
+
48
+ @client.expects(:display)
49
+ end
50
+
51
+ it { @job.archive }
52
+ end
53
+
54
+ describe "#capture" do
55
+ before do
56
+ @client.stubs(:generate_resolver).returns(mock(:resolve))
57
+ @client.stubs(:hpg_client)
58
+ .returns(
59
+ mock(backups_capture: {
60
+ uuid: "baz",
61
+ num: "10",
62
+ created_at: @created_at }
63
+ )
64
+ )
65
+ @client.stubs(:poll_transfer)
66
+
67
+ Heroku::Client::HerokuPostgresqlApp.expects(:new).with(@app)
68
+ .returns(mock(transfers_public_url: { url: @backup_url }))
69
+ end
70
+
71
+ it do
72
+ @job.capture
73
+ @job.created_at.must_equal @created_at
74
+ @job.backup_url.must_equal @backup_url
75
+ end
76
+ end
77
+
78
+ describe "#delete" do
79
+ it do
80
+ @temp_file = @job.send(:temp_file)
81
+ File.write(@temp_file, "content")
82
+ File.exist?(@temp_file).must_equal true
83
+ @job.delete
84
+ File.exist?(@temp_file).must_equal false
85
+ end
86
+ end
87
+
88
+ describe "#download" do
89
+ before { @job.backup_url = @backup_url }
90
+
91
+ it do
92
+ @job.download
93
+ @job.send(:file).read.must_match /Gem::Specification/
94
+ end
95
+ end
96
+
97
+ describe "#expire" do
98
+ before do
99
+ @transfers = [
100
+ { from_type: "pg_dump", to_type: "gof3r", created_at: Date.today, num: 20 },
101
+ { from_type: "pg_dump", to_type: "gof3r", created_at: Date.today-2, num: 17 },
102
+ { from_type: "pg_dump", to_type: "gof3r", created_at: Date.today-1, num: 18 },
103
+ { from_type: "pg_dump", to_type: "foo", created_at: Date.today-1, num: 19 }
104
+ ]
105
+
106
+ @client.expects(:hpg_app_client).with(@app).returns(mock(transfers: @transfers))
107
+ end
108
+
109
+ describe "when slots are available" do
110
+ it "does not expire a backup" do
111
+ @job.expire
112
+ end
113
+ end
114
+
115
+ describe "when a slot needs to be freed" do
116
+ before do
117
+ ENV["PGBACKUPS_KEEP"] = "2"
118
+ @client.expects(:backup_num).with("b017").returns("017")
119
+ @job.expects(:expire_backup).with("017")
120
+ @client.expects(:display)
121
+ end
122
+
123
+ it "expires a backup" do
124
+ @job.expire
125
+ end
126
+ end
127
+ end
128
+
129
+ end
130
+
131
+ end
@@ -1,33 +1,33 @@
1
1
  require "test_helper"
2
2
 
3
3
  describe PgbackupsArchive::Storage do
4
- let(:connection) {
5
- Fog::Storage.new(
6
- :provider => "AWS",
7
- :aws_access_key_id => "XXX",
8
- :aws_secret_access_key => "YYY")
9
- }
10
- let(:bucket) { connection.directories.create(:key => "someapp-backups") }
11
- let(:key) { "pgbackups/test/2012-08-02-12-00-00.dump" }
12
- let(:file) { "test" }
13
- let(:storage) { PgbackupsArchive::Storage.new(key, file) }
14
-
15
4
  before do
16
5
  Fog.mock!
17
- storage.stubs(:connection).returns(connection)
18
- storage.stubs(:bucket).returns(bucket)
6
+
7
+ @connection = Fog::Storage.new(
8
+ provider: "AWS",
9
+ aws_access_key_id: "XXX",
10
+ aws_secret_access_key: "YYY"
11
+ )
12
+ @bucket = @connection.directories.create(key: "someapp-backups")
13
+ @key = "pgbackups/test/2012-08-02-12-00-00.dump"
14
+ @file = "test"
15
+ @storage = PgbackupsArchive::Storage.new(@key, @file)
16
+
17
+ @storage.stubs(:connection).returns(@connection)
18
+ @storage.stubs(:bucket).returns(@bucket)
19
19
  end
20
20
 
21
21
  it "should create a fog connection" do
22
- storage.connection.class.must_equal Fog::Storage::AWS::Mock
22
+ @storage.connection.class.must_equal Fog::Storage::AWS::Mock
23
23
  end
24
24
 
25
25
  it "should create a fog directory" do
26
- storage.bucket.class.must_equal Fog::Storage::AWS::Directory
26
+ @storage.bucket.class.must_equal Fog::Storage::AWS::Directory
27
27
  end
28
28
 
29
29
  it "should create a fog file" do
30
- storage.store.class.must_equal Fog::Storage::AWS::File
30
+ @storage.store.class.must_equal Fog::Storage::AWS::File
31
31
  end
32
32
 
33
33
  end
metadata CHANGED
@@ -1,43 +1,49 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgbackups-archive
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.0
4
+ version: 1.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kenny Johnston
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-01-20 00:00:00.000000000 Z
11
+ date: 2015-03-16 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
- name: fog-aws
14
+ name: heroku
15
15
  requirement: !ruby/object:Gem::Requirement
16
16
  requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '3.28'
17
20
  - - ">="
18
21
  - !ruby/object:Gem::Version
19
- version: '0'
22
+ version: 3.28.6
20
23
  type: :runtime
21
24
  prerelease: false
22
25
  version_requirements: !ruby/object:Gem::Requirement
23
26
  requirements:
27
+ - - "~>"
28
+ - !ruby/object:Gem::Version
29
+ version: '3.28'
24
30
  - - ">="
25
31
  - !ruby/object:Gem::Version
26
- version: '0'
32
+ version: 3.28.6
27
33
  - !ruby/object:Gem::Dependency
28
- name: heroku
34
+ name: fog-aws
29
35
  requirement: !ruby/object:Gem::Requirement
30
36
  requirements:
31
37
  - - ">="
32
38
  - !ruby/object:Gem::Version
33
- version: 2.34.0
39
+ version: '0'
34
40
  type: :runtime
35
41
  prerelease: false
36
42
  version_requirements: !ruby/object:Gem::Requirement
37
43
  requirements:
38
44
  - - ">="
39
45
  - !ruby/object:Gem::Version
40
- version: 2.34.0
46
+ version: '0'
41
47
  - !ruby/object:Gem::Dependency
42
48
  name: rake
43
49
  requirement: !ruby/object:Gem::Requirement
@@ -66,6 +72,26 @@ dependencies:
66
72
  - - ">="
67
73
  - !ruby/object:Gem::Version
68
74
  version: '0'
75
+ - !ruby/object:Gem::Dependency
76
+ name: guard
77
+ requirement: !ruby/object:Gem::Requirement
78
+ requirements:
79
+ - - "~>"
80
+ - !ruby/object:Gem::Version
81
+ version: '2.12'
82
+ - - "~>"
83
+ - !ruby/object:Gem::Version
84
+ version: 2.12.5
85
+ type: :development
86
+ prerelease: false
87
+ version_requirements: !ruby/object:Gem::Requirement
88
+ requirements:
89
+ - - "~>"
90
+ - !ruby/object:Gem::Version
91
+ version: '2.12'
92
+ - - "~>"
93
+ - !ruby/object:Gem::Version
94
+ version: 2.12.5
69
95
  - !ruby/object:Gem::Dependency
70
96
  name: guard-minitest
71
97
  requirement: !ruby/object:Gem::Requirement
@@ -146,8 +172,7 @@ dependencies:
146
172
  - - ">="
147
173
  - !ruby/object:Gem::Version
148
174
  version: 0.9.1
149
- description: A means of automating Heroku's pgbackups and archiving them to Amazon
150
- S3 via the fog gem.
175
+ description: A means of automating Heroku PGBackups and archiving them to Amazon S3.
151
176
  email:
152
177
  - kjohnston.ca@gmail.com
153
178
  executables: []
@@ -158,7 +183,7 @@ files:
158
183
  - README.md
159
184
  - Rakefile
160
185
  - lib/pgbackups-archive.rb
161
- - lib/pgbackups-archive/heroku/client/pgbackups_archive.rb
186
+ - lib/pgbackups-archive/job.rb
162
187
  - lib/pgbackups-archive/railtie.rb
163
188
  - lib/pgbackups-archive/storage.rb
164
189
  - lib/pgbackups-archive/version.rb
@@ -166,7 +191,7 @@ files:
166
191
  - lib/tasks/test.rake
167
192
  - test/ci/before_script.sh
168
193
  - test/ci/ci_runner.sh
169
- - test/lib/pgbackups-archive/heroku/client/pgbackups_archive_test.rb
194
+ - test/lib/pgbackups-archive/job_test.rb
170
195
  - test/lib/pgbackups-archive/storage_test.rb
171
196
  - test/support/mocha.rb
172
197
  - test/test_helper.rb
@@ -193,11 +218,11 @@ rubyforge_project:
193
218
  rubygems_version: 2.4.5
194
219
  signing_key:
195
220
  specification_version: 4
196
- summary: Automates archival of Heroku's pgbackups to S3
221
+ summary: Automates archival of Heroku PGBackups to S3
197
222
  test_files:
198
223
  - test/ci/before_script.sh
199
224
  - test/ci/ci_runner.sh
200
- - test/lib/pgbackups-archive/heroku/client/pgbackups_archive_test.rb
225
+ - test/lib/pgbackups-archive/job_test.rb
201
226
  - test/lib/pgbackups-archive/storage_test.rb
202
227
  - test/support/mocha.rb
203
228
  - test/test_helper.rb
@@ -1,77 +0,0 @@
1
- require "heroku/client"
2
- require "tmpdir"
3
-
4
- class Heroku::Client::PgbackupsArchive
5
-
6
- attr_reader :client, :pgbackup
7
-
8
- def self.perform
9
- backup = new
10
- backup.capture
11
- backup.download
12
- backup.archive
13
- backup.delete
14
- end
15
-
16
- def initialize(attrs={})
17
- Heroku::Command.load
18
- @client = Heroku::Client::Pgbackups.new pgbackups_url
19
- @pgbackup = nil
20
- end
21
-
22
- def archive
23
- PgbackupsArchive::Storage.new(key, file).store
24
- end
25
-
26
- def capture
27
- @pgbackup = @client.create_transfer(database_url, database_url, nil,
28
- "BACKUP", :expire => true)
29
-
30
- until @pgbackup["finished_at"]
31
- print "."
32
- sleep 1
33
- @pgbackup = @client.get_transfer @pgbackup["id"]
34
- end
35
- end
36
-
37
- def delete
38
- File.delete temp_file
39
- end
40
-
41
- def download
42
- File.open(temp_file, "wb") do |output|
43
- streamer = lambda do |chunk, remaining_bytes, total_bytes|
44
- output.write chunk
45
- end
46
- Excon.get(@pgbackup["public_url"], :response_block => streamer)
47
- end
48
- end
49
-
50
- private
51
-
52
- def database_url
53
- ENV["PGBACKUPS_DATABASE_URL"] || ENV["DATABASE_URL"]
54
- end
55
-
56
- def environment
57
- defined?(Rails) ? Rails.env : nil
58
- end
59
-
60
- def file
61
- File.open temp_file, "r"
62
- end
63
-
64
- def key
65
- ["pgbackups", environment, @pgbackup["finished_at"]
66
- .gsub(/\/|\:|\.|\s/, "-").concat(".dump")].compact.join("/")
67
- end
68
-
69
- def pgbackups_url
70
- ENV["PGBACKUPS_URL"]
71
- end
72
-
73
- def temp_file
74
- "#{Dir.tmpdir}/#{URI(@pgbackup['public_url']).path.split('/').last}"
75
- end
76
-
77
- end
@@ -1,181 +0,0 @@
1
- require "test_helper"
2
- require "heroku/client"
3
-
4
- describe Heroku::Client::PgbackupsArchive do
5
-
6
- describe "#self.perform" do
7
- before do
8
- Heroku::Client::PgbackupsArchive.expects(:new).returns(
9
- mock(
10
- :capture => stub,
11
- :download => stub,
12
- :archive => stub,
13
- :delete => stub
14
- )
15
- )
16
- end
17
-
18
- it { Heroku::Client::PgbackupsArchive.perform }
19
- end
20
-
21
- describe "An instance" do
22
- let(:database_url) { "db_url" }
23
- let(:pgbackups_url) { "https://ip:password@pgbackups.heroku.com/client" }
24
- let(:backup) { Heroku::Client::PgbackupsArchive.new }
25
-
26
- before do
27
- ENV["PGBACKUPS_URL"] = pgbackups_url
28
- end
29
-
30
- describe "#initialize" do
31
- it "should set client to a Heroku::Client::Pgbackups instance" do
32
- backup.client.class.must_equal Heroku::Client::Pgbackups
33
- end
34
- end
35
-
36
- describe "#archive" do
37
- let(:key) { "some-key" }
38
- let(:file) { "some-file" }
39
-
40
- before do
41
- backup.stubs(:key).returns(key)
42
- backup.stubs(:file).returns(file)
43
-
44
- PgbackupsArchive::Storage.expects(:new).with(key, file)
45
- .returns(mock(:store => stub))
46
- end
47
-
48
- it "should use a storage instance to store the archive" do
49
- backup.archive
50
- end
51
- end
52
-
53
- describe "#capture" do
54
- let(:pgbackup) { { "finished_at" => "some-timestamp" } }
55
-
56
- before do
57
- backup.stubs(:database_url).returns(database_url)
58
-
59
- backup.client.expects(:create_transfer)
60
- .with(database_url, database_url, nil, "BACKUP", :expire => true)
61
- .returns(pgbackup)
62
- end
63
-
64
- it "uses the client to create a pgbackup" do
65
- backup.capture
66
- end
67
- end
68
-
69
- describe "#delete" do
70
- let(:temp_file) { "temp-file" }
71
-
72
- before do
73
- backup.stubs(:temp_file).returns(temp_file)
74
- File.expects(:delete).with(temp_file).returns(true)
75
- end
76
-
77
- it "should delete the temp file" do
78
- backup.delete
79
- end
80
- end
81
-
82
- describe "#download" do
83
- let(:pgbackup) do
84
- {
85
- "public_url" => "https://raw.github.com/kjohnston/" +
86
- "pgbackups-archive/master/pgbackups-archive.gemspec"
87
- }
88
- end
89
-
90
- before do
91
- backup.instance_eval do
92
- @pgbackup = {
93
- "public_url" => "https://raw.githubusercontent.com/kjohnston/pgbackups-archive/master/pgbackups-archive.gemspec"
94
- }
95
- end
96
- backup.download
97
- end
98
-
99
- it "downloads the backup file" do
100
- backup.send(:file).read.must_match /Gem::Specification/
101
- end
102
-
103
- after do
104
- backup.delete
105
- end
106
- end
107
-
108
- describe "#database_url" do
109
- describe "when an alternate database to backup is not set" do
110
- before do
111
- ENV["PGBACKUPS_DATABASE_URL"] = nil
112
- ENV["DATABASE_URL"] = "default_url"
113
- end
114
-
115
- it "defaults to using the DATABASE_URL" do
116
- backup.send(:database_url).must_equal "default_url"
117
- end
118
- end
119
-
120
- describe "an alternate database to backup is set" do
121
- before do
122
- ENV["PGBACKUPS_DATABASE_URL"] = "alternate_url"
123
- ENV["DATABASE_URL"] = "default_url"
124
- end
125
-
126
- it "uses the PGBACKUPS_DATABASE_URL" do
127
- backup.send(:database_url).must_equal "alternate_url"
128
- end
129
- end
130
- end
131
-
132
- describe "#file" do
133
- let(:temp_file) { "temp-file" }
134
-
135
- before do
136
- backup.stubs(:temp_file).returns(temp_file)
137
- File.expects(:open).with(temp_file, "r").returns("")
138
- end
139
-
140
- it { backup.send(:file) }
141
- end
142
-
143
- describe "#key" do
144
- before do
145
- backup.instance_eval do
146
- @pgbackup = {
147
- "finished_at" => "timestamp"
148
- }
149
- end
150
- end
151
-
152
- it "should be composed properly" do
153
- path = ["pgbackups", backup.send(:environment), "timestamp.dump"]
154
- .compact.join("/")
155
- backup.send(:key).must_equal path
156
- end
157
- end
158
-
159
- describe "#pgbackups_url" do
160
- it { backup.send(:pgbackups_url).must_equal pgbackups_url }
161
- end
162
-
163
- describe "#temp_file" do
164
- before do
165
- backup.instance_eval do
166
- @pgbackup = {
167
- "public_url" => "https://raw.github.com/kjohnston/pgbackups-archive/master/pgbackups-archive.gemspec"
168
- }
169
- end
170
- end
171
-
172
- it "should be composed properly" do
173
- temp_file = backup.send(:temp_file)
174
- temp_file.must_match /^\/var\//
175
- temp_file.must_match /\/pgbackups-archive.gemspec$/
176
- end
177
- end
178
-
179
- end
180
-
181
- end