intermix-client 0.0.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 87731151937f4cd67118f193d514e5f91dff0f1d4e2c10b6420fa65a35534393
4
+ data.tar.gz: 2057a99edf427f0266194a57f5b53cc2207173830eddbe5aff9914056bb43e4b
5
+ SHA512:
6
+ metadata.gz: db718e660550a3f38b5dee9ffd990a5255112f78e1bc5c03850e12738c8946b59177c5d2d03e1b751880e2d3a8ba1fd1148244355a1823f6c2b2cd98367325a0
7
+ data.tar.gz: f1fe04612f3adf8d9f6351f3dee6fce280a70586b4976936c8e282fa19cbdacb9b05b6cddc920bedbb5efb9b097659ccdc52e788d998135ce190af6a20702596
data/.coveralls.yml ADDED
@@ -0,0 +1 @@
1
+ service_name: travis-ci
data/.gitignore ADDED
@@ -0,0 +1,7 @@
1
+ *.gem
2
+ Gemfile.lock
3
+ /.approvals
4
+ /.bundle
5
+ /coverage
6
+ .DS_Store
7
+ .idea
data/.rspec ADDED
@@ -0,0 +1,3 @@
1
+ --require spec_helper
2
+ --format documentation
3
+ --color
data/.rubocop.yml ADDED
@@ -0,0 +1,40 @@
1
+ AllCops:
2
+ TargetRubyVersion: 2.3
3
+
4
+ Metrics/BlockLength:
5
+ Enabled: false
6
+ Metrics/LineLength:
7
+ Enabled: false
8
+ Metrics/MethodLength:
9
+ Enabled: false
10
+ Metrics/ParameterLists:
11
+ Enabled: false
12
+ Metrics/AbcSize:
13
+ Enabled: false
14
+ Metrics/CyclomaticComplexity:
15
+ Enabled: false
16
+ Metrics/PerceivedComplexity:
17
+ Enabled: false
18
+
19
+ Layout/ElseAlignment:
20
+ Enabled: false
21
+ Layout/EndAlignment:
22
+ Enabled: false
23
+ Layout/IndentationWidth:
24
+ Enabled: false
25
+
26
+ Style/Documentation:
27
+ Enabled: false
28
+ Style/FrozenStringLiteralComment:
29
+ Enabled: false
30
+ Style/MutableConstant:
31
+ Enabled: false
32
+ Style/RedundantConditional:
33
+ Enabled: false
34
+
35
+ # https://github.com/lovewithfood/rubocop-rspec-focused
36
+ require:
37
+ - rubocop/rspec/focused
38
+
39
+ RSpec/Focused:
40
+ Enabled: true
data/.travis.yml ADDED
@@ -0,0 +1,8 @@
1
+ sudo: false
2
+ language: ruby
3
+ rvm:
4
+ - 2.3.8
5
+ script:
6
+ - bundle exec rubocop
7
+ - bundle exec rake
8
+ cache: bundler
data/Gemfile ADDED
@@ -0,0 +1,20 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec
4
+
5
+ group :development do
6
+ gem 'rubocop-rspec-focused', '~> 0.1', require: false # Find focused specs.
7
+ end
8
+
9
+ group :development, :test do
10
+ gem 'awesome_print'
11
+ gem 'pry'
12
+ gem 'rake'
13
+ gem 'rubocop'
14
+ end
15
+
16
+ group :test do
17
+ gem 'approvals'
18
+ gem 'coveralls', require: false
19
+ gem 'rspec'
20
+ end
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2019 Joe Manley
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,61 @@
1
+ [![Build Status](https://travis-ci.org/tophatter/intermix-api-ruby.svg?branch=master)](https://travis-ci.org/tophatter/intermix-api-ruby)
2
+ [![Coverage Status](https://coveralls.io/repos/github/tophatter/intermix-api-ruby/badge.svg?branch=master&t=nC87CM)](https://coveralls.io/github/tophatter/intermix-api-ruby?branch=master)
3
+
4
+ # Intermix API client for Ruby
5
+
6
+ Intermix is an analytics platform that instruments Amazon Redshift to improve performance, reduce costs, and eliminate downtime. Our SaaS product intelligently tunes databases in the cloud, provides deep analytics, recommendations, and predictions, so companies don't have to hire DBA experts, throw money at performance problems, or deal with slow queries.
7
+ This gem helps to interact with the Intermix API.
8
+
9
+ To experiment with that code, run `bin/console` for an interactive prompt.
10
+
11
+ ## Installation
12
+
13
+ Add this line to your application's Gemfile:
14
+
15
+ ```ruby
16
+ gem 'intermix-client'
17
+ ```
18
+
19
+ And then execute:
20
+
21
+ $ bundle
22
+
23
+ Or install it yourself as:
24
+
25
+ $ gem install intermix-client
26
+
27
+ ## Usage
28
+
29
+ #### Creating a client
30
+ Before performing any operation through the intermix api, we need an instance of `Intermix::Client`.
31
+ ```
32
+ require 'intermix'
33
+
34
+ configuration = Intermix::Configuration.new(api_token: api_token, cluster_id: cluster_id)
35
+ client = Intermix::Client.new(configuration)
36
+ ```
37
+
38
+ #### Get list of tables
39
+ ```
40
+ tables = client.tables # These are instances of `Intermix::Table`.
41
+ ```
42
+
43
+ #### Generate vacuum script
44
+ ```
45
+ vacuum = Intermix::Vacuum.new(client: client, delete_only: true, unsorted_threshold: 0.50)
46
+
47
+ vacuum.generate_script # This returns the script in string format.
48
+ vacuum.save_script # This saves the genered vacuum script in `output/vacuum_scripts/vacuum_databases.sh`.
49
+ ```
50
+
51
+ ## Development
52
+
53
+ After checking out the repo, run `bundle install` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
54
+
55
+ ## Contributing
56
+
57
+ Bug reports and pull requests are welcome on GitHub at https://github.com/tophatter/intermix-client.
58
+
59
+ ## License
60
+
61
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,5 @@
1
+ require 'rspec/core/rake_task'
2
+
3
+ RSpec::Core::RakeTask.new(:spec)
4
+
5
+ task default: :spec
data/bin/console ADDED
@@ -0,0 +1,9 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'bundler/setup'
4
+ require 'intermix/client'
5
+ require 'awesome_print'
6
+ require 'pry'
7
+
8
+ AwesomePrint.pry!
9
+ Pry.start
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
@@ -0,0 +1,24 @@
1
+ # To publish the next version:
2
+ # gem build intermix-client.gemspec
3
+ # gem push intermix-client-0.0.1.gem
4
+ Gem::Specification.new do |spec|
5
+ spec.name = 'intermix-client'
6
+ spec.version = '0.0.2'
7
+ spec.authors = ['Joe Manley']
8
+ spec.email = ['joemanley201@gmail.com']
9
+
10
+ spec.summary = 'Intermix API Client in Ruby'
11
+ spec.description = 'Intermix API Client in Ruby'
12
+ spec.homepage = 'https://github.com/tophatter/intermix-api-ruby'
13
+ spec.license = 'MIT'
14
+
15
+ spec.required_ruby_version = '~> 2.3'
16
+
17
+ spec.add_dependency 'activesupport', '~> 4.2'
18
+ spec.add_dependency 'httparty', '~> 0.14.0'
19
+
20
+ spec.files = `git ls-files`.split("\n")
21
+ spec.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
22
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
23
+ spec.require_paths = ['lib']
24
+ end
data/lib/intermix.rb ADDED
@@ -0,0 +1,8 @@
1
+ require 'httparty'
2
+ require 'active_support'
3
+ require 'active_support/core_ext'
4
+
5
+ require File.dirname(__FILE__) + '/intermix/client'
6
+ require File.dirname(__FILE__) + '/intermix/configuration'
7
+ require File.dirname(__FILE__) + '/intermix/table'
8
+ require File.dirname(__FILE__) + '/intermix/vacuum'
@@ -0,0 +1,40 @@
1
+ module Intermix
2
+ class Client
3
+ # paths
4
+ TABLES_PATH = '/tables'
5
+
6
+ attr_reader :configuration
7
+
8
+ def initialize(configuration)
9
+ raise ArgumentError, 'configuration cannot be nil.' unless configuration.present?
10
+
11
+ @configuration = configuration
12
+ end
13
+
14
+ def tables(fields = Table::FIELDS)
15
+ fields &&= Table::FIELDS
16
+ query = "fields=#{fields.join(',')}" if fields.any?
17
+
18
+ response = post(TABLES_PATH, query: query)
19
+
20
+ if response.success?
21
+ parsed_response = response.parsed_response
22
+ parsed_response['data'].map { |entry| Intermix::Table.new(entry) }
23
+ else
24
+ []
25
+ end
26
+ end
27
+
28
+ private
29
+
30
+ def post(path, query: nil)
31
+ uri = "#{@configuration.base_uri}#{path}?" + query
32
+
33
+ HTTParty.post(uri, headers: headers)
34
+ end
35
+
36
+ def headers
37
+ { 'Authorization' => "Token #{@configuration.api_token}" }
38
+ end
39
+ end
40
+ end
@@ -0,0 +1,23 @@
1
+ module Intermix
2
+ class Configuration
3
+ CLUSTER_TYPE = 'RedshiftCluster'
4
+ API_URL = 'https://dashboard.intermix.io/api'
5
+
6
+ attr_reader :api_token, :cluster_id, :cluster_type, :api_url
7
+
8
+ def initialize(api_token:, cluster_id:)
9
+ raise ArgumentError, 'api_token cannot be nil.' unless api_token.present?
10
+ raise ArgumentError, 'cluster_id cannot be nil.' unless cluster_id.present?
11
+
12
+ @api_token = api_token
13
+ @cluster_id = cluster_id
14
+
15
+ @cluster_type = CLUSTER_TYPE
16
+ @api_url = API_URL
17
+ end
18
+
19
+ def base_uri
20
+ @base_uri = "#{@api_url}/#{@cluster_type}/#{@cluster_id}" unless defined?(@base_uri)
21
+ end
22
+ end
23
+ end
@@ -0,0 +1,28 @@
1
+ module Intermix
2
+ class Table
3
+ FIELDS = %w[
4
+ db_id
5
+ db_name
6
+ schema_id
7
+ schema_name
8
+ table_id
9
+ table_name
10
+ stats_pct_off
11
+ size_pct_unsorted
12
+ row_count
13
+ sort_key
14
+ ]
15
+
16
+ attr_accessor(*FIELDS)
17
+
18
+ def initialize(data)
19
+ raise ArgumentError, 'data cannot be nil.' unless data.present?
20
+
21
+ self.class::FIELDS.each { |field| send("#{field}=", data[field]) }
22
+ end
23
+
24
+ def full_name
25
+ "\"#{schema_name}\".\"#{table_name}\""
26
+ end
27
+ end
28
+ end
@@ -0,0 +1,121 @@
1
+ module Intermix
2
+ class Vacuum
3
+ DEFAULT_STATS_OFF_THRESHOLD = 0.10
4
+ DEFAULT_UNSORTED_THRESHOLD = 0.10
5
+ DEFAULT_VACUUM_THRESHOLD = 0.95
6
+
7
+ REDSHIFT_USERNAME = ''
8
+ REDSHIFT_HOST = ''
9
+ REDSHIFT_PORT = 5439
10
+
11
+ IGNORED_SCHEMAS = %w[pg_internal]
12
+
13
+ attr_reader :client,
14
+ :full, :delete_only, :sort, :analyze,
15
+ :stats_off_threshold, :unsorted_threshold,
16
+ :admin_user, :host, :port
17
+
18
+ def initialize(client:,
19
+ delete_only: false, full: false, sort: false,
20
+ stats_off_threshold: DEFAULT_STATS_OFF_THRESHOLD, unsorted_threshold: DEFAULT_UNSORTED_THRESHOLD,
21
+ vacuum_threshold: DEFAULT_VACUUM_THRESHOLD,
22
+ admin_user: REDSHIFT_USERNAME, host: REDSHIFT_HOST, port: REDSHIFT_PORT)
23
+ raise ArgumentError, 'client cannot be nil.' unless client.present?
24
+ raise ArgumentError, 'invalid vacuum mode.' if full && [delete_only, sort].any?
25
+
26
+ @client = client
27
+
28
+ @full = full
29
+ @delete_only = delete_only
30
+ @sort = sort
31
+ @analyze = true if full || delete_only
32
+
33
+ @stats_off_threshold = stats_off_threshold
34
+ @unsorted_threshold = unsorted_threshold
35
+ @vacuum_threshold_pct = [vacuum_threshold * 100, 100].min
36
+
37
+ @admin_user = admin_user
38
+ @host = host
39
+ @port = port
40
+ end
41
+
42
+ def eligible_tables
43
+ client.tables.select do |table|
44
+ if IGNORED_SCHEMAS.include?(table.schema_name)
45
+ false
46
+ elsif table.stats_pct_off.nil?
47
+ false
48
+ elsif table.size_pct_unsorted.nil?
49
+ false
50
+ elsif table.stats_pct_off <= @stats_off_threshold
51
+ false
52
+ elsif table.size_pct_unsorted <= @unsorted_threshold
53
+ false
54
+ else
55
+ true
56
+ end
57
+ end
58
+ end
59
+
60
+ def generate_script
61
+ output = script_header
62
+
63
+ eligible_tables.group_by(&:db_name).each do |db_name, tables|
64
+ output += "\n\\c #{db_name}\n\n"
65
+ tables.sort_by { |table| -table.size_pct_unsorted }.each do |table|
66
+ output += vacuum_command(table: table)
67
+ output += analyze_command(table: table) if @analyze
68
+ output += "\n"
69
+ end
70
+ end
71
+
72
+ output += "\n"
73
+
74
+ output
75
+ end
76
+
77
+ def save_script(location = 'output/vacuum_scripts/vacuum_databases.sh')
78
+ File.open(location, 'w') { |file| file.write(generate_script) }
79
+ end
80
+
81
+ private
82
+
83
+ def script_header
84
+ <<~SCRIPT
85
+ #!/bin/bash
86
+ # Intermix.io Vacuum Script
87
+ #
88
+ # This script requires the psql postgres command line client be installed prior
89
+ # to running. The psql client is avaliable for download here,
90
+ # https://www.postgresql.org/download/.
91
+ #
92
+ #
93
+ # You will be prompted for the administrator password
94
+ # Override the administrator username here
95
+ adminuser="#{@admin_user}"
96
+ psqlcommand="psql -e"
97
+ # Don't edit anything beyond this point
98
+ logindb="dev"
99
+ redshiftport="#{@host}"
100
+ redshifthost="#{@port}"
101
+ ${{psqlcommand}} -d ${{logindb}} -U ${{adminuser}} -p ${{redshiftport}} -h ${{redshifthost}} <<EOF
102
+ SCRIPT
103
+ end
104
+
105
+ def vacuum_command(table:)
106
+ command = if @full
107
+ 'FULL'
108
+ elsif @delete_only
109
+ 'DELETE ONLY'
110
+ elsif @sort
111
+ table.sort_key == 'INTERLEAVED' ? 'REINDEX' : 'SORT ONLY'
112
+ end
113
+
114
+ "VACUUM #{command} #{table.full_name} to #{@vacuum_threshold_pct.to_i} percent;\n"
115
+ end
116
+
117
+ def analyze_command(table:)
118
+ "ANALYZE #{table.full_name};\n"
119
+ end
120
+ end
121
+ end
@@ -0,0 +1,24 @@
1
+ #!/bin/bash
2
+ # Intermix.io Vacuum Script
3
+ #
4
+ # This script requires the psql postgres command line client be installed prior
5
+ # to running. The psql client is avaliable for download here,
6
+ # https://www.postgresql.org/download/.
7
+ #
8
+ #
9
+ # You will be prompted for the administrator password
10
+ # Override the administrator username here
11
+ adminuser=""
12
+ psqlcommand="psql -e"
13
+ # Don't edit anything beyond this point
14
+ logindb="dev"
15
+ redshiftport=""
16
+ redshifthost="5439"
17
+ ${{psqlcommand}} -d ${{logindb}} -U ${{adminuser}} -p ${{redshiftport}} -h ${{redshifthost}} <<EOF
18
+
19
+ \c db1
20
+
21
+ VACUUM DELETE ONLY "public"."hello"
22
+ ANALYZE "public"."hello";
23
+
24
+
@@ -0,0 +1,46 @@
1
+ # rspec spec/client_spec.rb
2
+ RSpec.describe Intermix::Client do
3
+ describe '#initialize' do
4
+ subject { Intermix::Client.new(configuration) }
5
+
6
+ context 'when configuration is nil' do
7
+ let(:configuration) { nil }
8
+
9
+ it 'throws an error' do
10
+ expect { subject }.to raise_error(ArgumentError, 'configuration cannot be nil.')
11
+ end
12
+ end
13
+
14
+ context 'when configuration is present', :stubbed_client do
15
+ let(:configuration) { stubbed_configuration }
16
+
17
+ it 'returns a client object' do
18
+ expect(subject).to be_present
19
+ end
20
+ end
21
+ end
22
+
23
+ describe '#tables', :stubbed_client do
24
+ let(:client) { stubbed_client }
25
+
26
+ subject { client.tables(Intermix::Table::FIELDS) }
27
+
28
+ before { allow(HTTParty).to receive(:post).with(any_args).and_return(response) }
29
+
30
+ context 'when the response is successful' do
31
+ let(:response) { successful_response }
32
+
33
+ it 'returns instances of Intermix::Table' do
34
+ expect(subject.first.is_a?(Intermix::Table)).to be_truthy
35
+ end
36
+ end
37
+
38
+ context 'when the response is not successful' do
39
+ let(:response) { failed_response }
40
+
41
+ it 'returns an empty array' do
42
+ expect(subject).to be_empty
43
+ end
44
+ end
45
+ end
46
+ end
@@ -0,0 +1,44 @@
1
+ # rspec spec/configuration_spec.rb
2
+ RSpec.describe Intermix::Configuration do
3
+ describe '#initialize' do
4
+ let(:api_token) { 'token' }
5
+ let(:cluster_id) { 1 }
6
+
7
+ subject { Intermix::Configuration.new(api_token: api_token, cluster_id: cluster_id) }
8
+
9
+ context 'when api_token is nil' do
10
+ let(:api_token) { nil }
11
+
12
+ it 'throws an error' do
13
+ expect { subject }.to raise_error(ArgumentError, 'api_token cannot be nil.')
14
+ end
15
+ end
16
+
17
+ context 'when cluster_id is nil' do
18
+ let(:cluster_id) { nil }
19
+
20
+ it 'throws an error' do
21
+ expect { subject }.to raise_error(ArgumentError, 'cluster_id cannot be nil.')
22
+ end
23
+ end
24
+
25
+ it 'assigns attributes correctly' do
26
+ expect(subject.api_token).to eq(api_token)
27
+ expect(subject.cluster_id).to eq(cluster_id)
28
+
29
+ expect(subject.cluster_type).to eq(Intermix::Configuration::CLUSTER_TYPE)
30
+ expect(subject.api_url).to eq(Intermix::Configuration::API_URL)
31
+ end
32
+ end
33
+
34
+ describe '#base_uri' do
35
+ let(:api_token) { 'token' }
36
+ let(:cluster_id) { 1 }
37
+
38
+ subject { Intermix::Configuration.new(api_token: api_token, cluster_id: cluster_id).base_uri }
39
+
40
+ it 'returns the correct uri based on api_url, cluster_type and cluster_type' do
41
+ expect(subject).to eq('https://dashboard.intermix.io/api/RedshiftCluster/1')
42
+ end
43
+ end
44
+ end
@@ -0,0 +1,27 @@
1
+ #!/bin/bash
2
+ # Intermix.io Vacuum Script
3
+ #
4
+ # This script requires the psql postgres command line client be installed prior
5
+ # to running. The psql client is avaliable for download here,
6
+ # https://www.postgresql.org/download/.
7
+ #
8
+ #
9
+ # You will be prompted for the administrator password
10
+ # Override the administrator username here
11
+ adminuser=""
12
+ psqlcommand="psql -e"
13
+ # Don't edit anything beyond this point
14
+ logindb="dev"
15
+ redshiftport=""
16
+ redshifthost="5439"
17
+ ${{psqlcommand}} -d ${{logindb}} -U ${{adminuser}} -p ${{redshiftport}} -h ${{redshifthost}} <<EOF
18
+
19
+ \c db1
20
+
21
+ VACUUM DELETE ONLY "public"."table2" to 99 percent;
22
+ ANALYZE "public"."table2";
23
+
24
+ VACUUM DELETE ONLY "public"."table1" to 99 percent;
25
+ ANALYZE "public"."table1";
26
+
27
+
@@ -0,0 +1,27 @@
1
+ #!/bin/bash
2
+ # Intermix.io Vacuum Script
3
+ #
4
+ # This script requires the psql postgres command line client be installed prior
5
+ # to running. The psql client is avaliable for download here,
6
+ # https://www.postgresql.org/download/.
7
+ #
8
+ #
9
+ # You will be prompted for the administrator password
10
+ # Override the administrator username here
11
+ adminuser=""
12
+ psqlcommand="psql -e"
13
+ # Don't edit anything beyond this point
14
+ logindb="dev"
15
+ redshiftport=""
16
+ redshifthost="5439"
17
+ ${{psqlcommand}} -d ${{logindb}} -U ${{adminuser}} -p ${{redshiftport}} -h ${{redshifthost}} <<EOF
18
+
19
+ \c db1
20
+
21
+ VACUUM FULL "public"."table2" to 99 percent;
22
+ ANALYZE "public"."table2";
23
+
24
+ VACUUM FULL "public"."table1" to 99 percent;
25
+ ANALYZE "public"."table1";
26
+
27
+
@@ -0,0 +1,25 @@
1
+ #!/bin/bash
2
+ # Intermix.io Vacuum Script
3
+ #
4
+ # This script requires the psql postgres command line client be installed prior
5
+ # to running. The psql client is avaliable for download here,
6
+ # https://www.postgresql.org/download/.
7
+ #
8
+ #
9
+ # You will be prompted for the administrator password
10
+ # Override the administrator username here
11
+ adminuser=""
12
+ psqlcommand="psql -e"
13
+ # Don't edit anything beyond this point
14
+ logindb="dev"
15
+ redshiftport=""
16
+ redshifthost="5439"
17
+ ${{psqlcommand}} -d ${{logindb}} -U ${{adminuser}} -p ${{redshiftport}} -h ${{redshifthost}} <<EOF
18
+
19
+ \c db1
20
+
21
+ VACUUM SORT ONLY "public"."table2" to 99 percent;
22
+
23
+ VACUUM SORT ONLY "public"."table1" to 99 percent;
24
+
25
+
@@ -0,0 +1,23 @@
1
+ require 'intermix'
2
+ require 'awesome_print'
3
+ require 'approvals/rspec'
4
+ require 'coveralls'
5
+
6
+ Coveralls.wear!
7
+
8
+ RSpec.configure do |rspec|
9
+ # This config option will be enabled by default on RSpec 4,
10
+ # but for reasons of backwards compatibility, you have to
11
+ # set it on RSpec 3.
12
+ #
13
+ # It causes the host group and examples to inherit metadata
14
+ # from the shared context.
15
+ rspec.shared_context_metadata_behavior = :apply_to_host_groups
16
+ end
17
+
18
+ Approvals.configure do |config|
19
+ config.approvals_path = 'spec/fixtures/approvals/'
20
+ end
21
+
22
+ gemspec = Gem::Specification.find_by_name('intermix-client')
23
+ Dir["#{gemspec.gem_dir}/spec/support/**/*.rb"].each { |f| require f }
@@ -0,0 +1,30 @@
1
+ RSpec.shared_context 'stubbed_client' do
2
+ let(:api_token) { 'token' }
3
+ let(:cluster_id) { 1 }
4
+
5
+ let(:stubbed_configuration) { Intermix::Configuration.new(api_token: api_token, cluster_id: cluster_id) }
6
+ let(:stubbed_client) { Intermix::Client.new(stubbed_configuration) }
7
+
8
+ # tables
9
+ let(:stubbed_table) do
10
+ {
11
+ db_id: '1',
12
+ db_name: 'db1',
13
+ schema_id: 1,
14
+ schema_name: 'public',
15
+ table_id: 1,
16
+ table_name: 'events',
17
+ stats_pct_off: 0.12,
18
+ size_pct_unsorted: 0.13,
19
+ row_count: 123_456,
20
+ sort_key: 'id'
21
+ }
22
+ end
23
+
24
+ let(:successful_response) { OpenStruct.new('code': 200, success?: true, 'data': [stubbed_table].to_json, parsed_response: { 'data' => [stubbed_table] }) }
25
+ let(:failed_response) { OpenStruct.new('code': 403, 'data': nil) }
26
+ end
27
+
28
+ RSpec.configure do |rspec|
29
+ rspec.include_context 'stubbed_client', stubbed_client: true
30
+ end
@@ -0,0 +1,39 @@
1
+ # rspec spec/table_spec.rb
2
+ RSpec.describe Intermix::Table do
3
+ describe '#initialize', :stubbed_client do
4
+ subject { Intermix::Table.new(data) }
5
+
6
+ context 'when data is nil' do
7
+ let(:data) { nil }
8
+
9
+ it 'throws an error' do
10
+ expect { subject }.to raise_error(ArgumentError, 'data cannot be nil.')
11
+ end
12
+ end
13
+
14
+ context 'when data is present' do
15
+ let(:data) { stubbed_table.with_indifferent_access }
16
+
17
+ it 'assigns the attributes correctly' do
18
+ expect(subject.db_id).to eq('1')
19
+ expect(subject.db_name).to eq('db1')
20
+ expect(subject.schema_id).to eq(1)
21
+ expect(subject.schema_name).to eq('public')
22
+ expect(subject.table_id).to eq(1)
23
+ expect(subject.table_name).to eq('events')
24
+ expect(subject.stats_pct_off).to eq(0.12)
25
+ expect(subject.size_pct_unsorted).to eq(0.13)
26
+ expect(subject.row_count).to eq(123_456)
27
+ expect(subject.sort_key).to eq('id')
28
+ end
29
+ end
30
+ end
31
+
32
+ describe '#full_name', :stubbed_client do
33
+ subject { Intermix::Table.new(stubbed_table.with_indifferent_access) }
34
+
35
+ it 'returns the schema name along with the table name' do
36
+ expect(subject.full_name).to eq('"public"."events"')
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,136 @@
1
+ # rspec spec/vacuum_spec.rb
2
+ RSpec.describe Intermix::Vacuum do
3
+ describe '#initialize', :stubbed_client do
4
+ let(:delete_only) { false }
5
+ let(:full) { true }
6
+ let(:sort) { false }
7
+
8
+ subject { Intermix::Vacuum.new(client: client, full: full, delete_only: delete_only, sort: sort) }
9
+
10
+ context 'when client is nil' do
11
+ let(:client) { nil }
12
+
13
+ it 'throws an error' do
14
+ expect { subject }.to raise_error(ArgumentError, 'client cannot be nil.')
15
+ end
16
+ end
17
+
18
+ context 'invalid vacuum mode' do
19
+ context 'full and delete_only' do
20
+ let(:full) { true }
21
+ let(:delete_only) { true }
22
+ let(:sort) { false }
23
+
24
+ let(:client) { stubbed_client }
25
+
26
+ it 'throws an error' do
27
+ expect { subject }.to raise_error(ArgumentError, 'invalid vacuum mode.')
28
+ end
29
+ end
30
+
31
+ context 'full and sort' do
32
+ let(:full) { true }
33
+ let(:delete_only) { false }
34
+ let(:sort) { true }
35
+
36
+ let(:client) { stubbed_client }
37
+
38
+ it 'throws an error' do
39
+ expect { subject }.to raise_error(ArgumentError, 'invalid vacuum mode.')
40
+ end
41
+ end
42
+ end
43
+
44
+ context 'valid vacuum mode' do
45
+ context 'when it is only full' do
46
+ let(:full) { true }
47
+ let(:delete_only) { false }
48
+ let(:sort) { false }
49
+
50
+ let(:client) { stubbed_client }
51
+
52
+ it 'assigns the correct attributes' do
53
+ expect { subject }.not_to raise_error
54
+
55
+ expect(subject.client).to be_present
56
+
57
+ expect(subject.full).to be_truthy
58
+ expect(subject.delete_only).to be_falsey
59
+ expect(subject.sort).to be_falsey
60
+ expect(subject.analyze).to be_truthy
61
+
62
+ expect(subject.stats_off_threshold).to eq(0.10)
63
+ expect(subject.stats_off_threshold).to eq(0.10)
64
+
65
+ expect(subject.admin_user).to eq('')
66
+ expect(subject.host).to eq('')
67
+ expect(subject.port).to eq(5439)
68
+ end
69
+ end
70
+ end
71
+ end
72
+
73
+ describe '#generate_script', :stubbed_client do
74
+ let(:stats_off_threshold) { 0.21 }
75
+ let(:unsorted_threshold) { 0.22 }
76
+ let(:threshold_met) do
77
+ [
78
+ stubbed_table.merge(table_name: 'table1', stats_pct_off: 0.45, size_pct_unsorted: 0.41), # included
79
+ stubbed_table.merge(table_name: 'table2', stats_pct_off: 0.56, size_pct_unsorted: 0.78) # included
80
+ ]
81
+ end
82
+ let(:threshold_unmet) do
83
+ [
84
+ stubbed_table.merge(table_name: 'table3', stats_pct_off: 0.45, size_pct_unsorted: nil), # excluded
85
+ stubbed_table.merge(table_name: 'table4', stats_pct_off: 0.45, size_pct_unsorted: 0.20), # excluded
86
+ stubbed_table.merge(table_name: 'table5', stats_pct_off: 0.20, size_pct_unsorted: 0.41) # excluded
87
+ ]
88
+ end
89
+ let(:excluded_schema) do
90
+ [
91
+ threshold_met.first.merge(schema_name: Intermix::Vacuum::IGNORED_SCHEMAS.first) # excluded
92
+ ]
93
+ end
94
+
95
+ let(:delete_only) { false }
96
+ let(:full) { false }
97
+ let(:sort) { false }
98
+
99
+ subject do
100
+ vacuum = Intermix::Vacuum.new(client: stubbed_client,
101
+ delete_only: delete_only, full: full, sort: sort,
102
+ stats_off_threshold: stats_off_threshold, unsorted_threshold: unsorted_threshold,
103
+ vacuum_threshold: 0.99)
104
+ vacuum.generate_script
105
+ end
106
+
107
+ before do
108
+ tables = [threshold_met + threshold_unmet + excluded_schema].flatten.map { |t| Intermix::Table.new(t.with_indifferent_access) }
109
+ allow(stubbed_client).to receive(:tables).and_return(tables)
110
+ end
111
+
112
+ context 'when it is in delete_only mode' do
113
+ let(:delete_only) { true }
114
+
115
+ it 'generates the expected script' do
116
+ verify(format: :txt) { subject }
117
+ end
118
+ end
119
+
120
+ context 'when it is in sort mode' do
121
+ let(:sort) { true }
122
+
123
+ it 'generates the expected script' do
124
+ verify(format: :txt) { subject }
125
+ end
126
+ end
127
+
128
+ context 'when it is in full mode' do
129
+ let(:full) { true }
130
+
131
+ it 'generates the expected script' do
132
+ verify(format: :txt) { subject }
133
+ end
134
+ end
135
+ end
136
+ end
metadata ADDED
@@ -0,0 +1,108 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: intermix-client
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.2
5
+ platform: ruby
6
+ authors:
7
+ - Joe Manley
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2019-07-08 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: activesupport
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '4.2'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '4.2'
27
+ - !ruby/object:Gem::Dependency
28
+ name: httparty
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: 0.14.0
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: 0.14.0
41
+ description: Intermix API Client in Ruby
42
+ email:
43
+ - joemanley201@gmail.com
44
+ executables: []
45
+ extensions: []
46
+ extra_rdoc_files: []
47
+ files:
48
+ - ".coveralls.yml"
49
+ - ".gitignore"
50
+ - ".rspec"
51
+ - ".rubocop.yml"
52
+ - ".travis.yml"
53
+ - Gemfile
54
+ - LICENSE.txt
55
+ - README.md
56
+ - Rakefile
57
+ - bin/console
58
+ - bin/setup
59
+ - intermix-client.gemspec
60
+ - lib/intermix.rb
61
+ - lib/intermix/client.rb
62
+ - lib/intermix/configuration.rb
63
+ - lib/intermix/table.rb
64
+ - lib/intermix/vacuum.rb
65
+ - output/vacuum_scripts/vacuum_databases.sh
66
+ - spec/client_spec.rb
67
+ - spec/configuration_spec.rb
68
+ - spec/fixtures/approvals/intermix_vacuum/generate_script/when_it_is_in_delete_only_mode/generates_the_expected_script.approved.txt
69
+ - spec/fixtures/approvals/intermix_vacuum/generate_script/when_it_is_in_full_mode/generates_the_expected_script.approved.txt
70
+ - spec/fixtures/approvals/intermix_vacuum/generate_script/when_it_is_in_sort_mode/generates_the_expected_script.approved.txt
71
+ - spec/spec_helper.rb
72
+ - spec/support/shared_contexts/stubbed_client.rb
73
+ - spec/table_spec.rb
74
+ - spec/vacuum_spec.rb
75
+ homepage: https://github.com/tophatter/intermix-api-ruby
76
+ licenses:
77
+ - MIT
78
+ metadata: {}
79
+ post_install_message:
80
+ rdoc_options: []
81
+ require_paths:
82
+ - lib
83
+ required_ruby_version: !ruby/object:Gem::Requirement
84
+ requirements:
85
+ - - "~>"
86
+ - !ruby/object:Gem::Version
87
+ version: '2.3'
88
+ required_rubygems_version: !ruby/object:Gem::Requirement
89
+ requirements:
90
+ - - ">="
91
+ - !ruby/object:Gem::Version
92
+ version: '0'
93
+ requirements: []
94
+ rubyforge_project:
95
+ rubygems_version: 2.7.9
96
+ signing_key:
97
+ specification_version: 4
98
+ summary: Intermix API Client in Ruby
99
+ test_files:
100
+ - spec/client_spec.rb
101
+ - spec/configuration_spec.rb
102
+ - spec/fixtures/approvals/intermix_vacuum/generate_script/when_it_is_in_delete_only_mode/generates_the_expected_script.approved.txt
103
+ - spec/fixtures/approvals/intermix_vacuum/generate_script/when_it_is_in_full_mode/generates_the_expected_script.approved.txt
104
+ - spec/fixtures/approvals/intermix_vacuum/generate_script/when_it_is_in_sort_mode/generates_the_expected_script.approved.txt
105
+ - spec/spec_helper.rb
106
+ - spec/support/shared_contexts/stubbed_client.rb
107
+ - spec/table_spec.rb
108
+ - spec/vacuum_spec.rb