RubyGems - prodder - Versions diffs - 1.7 - Mend

prodder 1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (33) hide show

checksums.yaml +7 -0
data/.gitignore +22 -0
data/.travis.yml +52 -0
data/Gemfile +16 -0
data/Gemfile.lock +64 -0
data/LICENSE.txt +21 -0
data/README.md +272 -0
data/Rakefile +20 -0
data/bin/prodder +4 -0
data/features/commit.feature +69 -0
data/features/dump.feature +187 -0
data/features/init.feature +24 -0
data/features/lint.feature +46 -0
data/features/prodder.feature +76 -0
data/features/push.feature +25 -0
data/features/step_definitions/git_steps.rb +65 -0
data/features/step_definitions/prodder_steps.rb +150 -0
data/features/support/blog.git.tgz +0 -0
data/features/support/env.rb +116 -0
data/features/support/prodder__blog_prod.sql +153 -0
data/lib/prodder.rb +5 -0
data/lib/prodder/cli.rb +135 -0
data/lib/prodder/config.rb +95 -0
data/lib/prodder/git.rb +97 -0
data/lib/prodder/pg.rb +486 -0
data/lib/prodder/prodder.rake +390 -0
data/lib/prodder/project.rb +150 -0
data/lib/prodder/railtie.rb +7 -0
data/lib/prodder/version.rb +3 -0
data/prodder.gemspec +25 -0
data/spec/config_spec.rb +64 -0
data/spec/spec_helper.rb +3 -0
metadata +91 -0

data/Rakefile ADDED

@@ -0,0 +1,20 @@
+# Will this ever be useful? ...
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec)
+task :extract_fixtures do
+  unless File.directory? 'features/support/blog.git'
+    Dir.chdir('features/support') { sh "tar xzf blog.git.tgz" }
+  end
+end
+require 'cucumber/rake/task'
+Cucumber::Rake::Task.new :cucumber => :extract_fixtures
+namespace :cuke do
+  Cucumber::Rake::Task.new(:wip => :extract_fixtures) do |t|
+    t.cucumber_opts = '--tags @wip'
+  end
+end
+task :default => [:spec, :cucumber]

data/bin/prodder ADDED

@@ -0,0 +1,4 @@
+#!/usr/bin/env ruby
+$: << File.expand_path('../lib', File.dirname(__FILE__))
+require 'prodder/cli'
+Prodder::CLI.start

data/features/commit.feature ADDED

@@ -0,0 +1,69 @@
+Feature: Commiting updated dumps to a project's repository
+  Background:
+    Given a prodder config in "prodder.yml" with project: blog
+    And a "blog" git repository
+    And I successfully run `prodder init -c prodder.yml`
+    And I successfully run `prodder dump -c prodder.yml`
+  Scenario: Structure, seed, quality_checks, permissions and settings files not yet tracked
+    When I run `prodder commit -c prodder.yml`
+    Then 1 commit by "prodder auto-commit" should be in the "blog" repository
+    And the file "db/structure.sql" should now be tracked
+    And the file "db/seeds.sql" should now be tracked
+    And the file "db/quality_checks.sql" should now be tracked
+    And the file "db/permissions.sql" should now be tracked
+    And the file "db/settings.sql" should now be tracked
+  Scenario: No changes to any file
+    When I run `prodder commit -c prodder.yml`
+    Then 1 commit by "prodder auto-commit" should be in the "blog" repository
+    When I run `prodder dump -c prodder.yml`
+    And  I run `prodder commit -c prodder.yml`
+    Then 1 commit by "prodder auto-commit" should be in the "blog" repository
+  Scenario: Changes only to the structure
+    When I run `prodder commit -c prodder.yml`
+    Then 1 commit by "prodder auto-commit" should be in the "blog" repository
+    When I create a new table "linkbacks" in the "blog" database
+    And  I run `prodder dump -c prodder.yml`
+    And  I run `prodder commit -c prodder.yml`
+    And  2 commits by "prodder auto-commit" should be in the "blog" repository
+    And  the latest commit should have changed "db/structure.sql" to contain "CREATE TABLE linkbacks"
+    And  the latest commit should not have changed "db/seeds.sql"
+    And  the latest commit should not have changed "db/quality_checks.sql"
+  Scenario: Changes only to the seed file
+    When I run `prodder commit -c prodder.yml`
+    Then 1 commit by "prodder auto-commit" should be in the "blog" repository
+    When I add a new author "Marley" to the "blog" database
+    And  I run `prodder dump -c prodder.yml`
+    And  I run `prodder commit -c prodder.yml`
+    Then 2 commits by "prodder auto-commit" should be in the "blog" repository
+    And  the latest commit should have changed "db/seeds.sql" to contain "Marley"
+    And  the latest commit should not have changed "db/structure.sql"
+    And  the latest commit should not have changed "db/quality_checks.sql"
+  Scenario: Changes to both
+    When I run `prodder commit -c prodder.yml`
+    Then 1 commit by "prodder auto-commit" should be in the "blog" repository
+    When I create a new table "captchas" in the "blog" database
+    And  I add a new author "Bob McBobbington" to the "blog" database
+    And  I run `prodder dump -c prodder.yml`
+    And  I run `prodder commit -c prodder.yml`
+    Then 2 commits by "prodder auto-commit" should be in the "blog" repository
+    And  the latest commit should have changed "db/structure.sql" to contain "CREATE TABLE captchas"
+    And  the latest commit should have changed "db/seeds.sql" to contain "Bob McBobbington"
+    And  the latest commit should not have changed "db/quality_checks.sql"
+  Scenario: Changes only to permissions
+    When I run `prodder commit -c prodder.yml`
+    Then 1 commit by "prodder auto-commit" should be in the "blog" repository
+    When I create a new table "gotchas" in the "blog" database
+    When I grant all permissions on table "gotchas" in the "blog" database to "prodder"
+    And  I run `prodder dump -c prodder.yml`
+    And  I run `prodder commit -c prodder.yml`
+    And  2 commits by "prodder auto-commit" should be in the "blog" repository
+    And  the latest commit should have changed "db/permissions.sql" to contain "GRANT ALL ON TABLE gotchas TO prodder"
+    And  the latest commit should not have changed "db/seeds.sql"
+    And  the latest commit should not have changed "db/quality_checks.sql"

data/features/dump.feature ADDED

@@ -0,0 +1,187 @@
+Feature: prodder dump
+  Background:
+    Given a prodder config in "prodder.yml" with project: blog
+  Scenario: Happy path: dump structure.sql, listed seed tables, quality_checks.sql, permissions.sql and settings.sql
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/structure.sql" should match /CREATE TABLE posts/
+    And  the workspace file "blog/db/structure.sql" should match /CREATE TABLE authors/
+    And  the workspace file "blog/db/seeds.sql" should match /COPY posts/
+    And  the workspace file "blog/db/seeds.sql" should match /COPY authors/
+    And  the workspace file "blog/db/quality_checks.sql" should match /SET search_path/
+    And  the workspace file "blog/db/quality_checks.sql" should match /CREATE TRIGGER /
+    And  the workspace file "blog/db/permissions.sql" should match /GRANT /
+    And  the workspace file "blog/db/settings.sql" should match /ALTER DATABASE /
+  Scenario: Include specified users, exclude other login roles from permissions dump
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should not match /exclude_this/
+    And  the workspace file "blog/db/permissions.sql" should match /include_this/
+  Scenario: Roles missing ACL must be created, modified and granted if they're granted to non-login/included users
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should match /create_role_if_not_exists\('prodder__blog_prod:read_only'\);/
+    And  the workspace file "blog/db/permissions.sql" should match /ALTER ROLE "prodder__blog_prod:read_only" WITH NOSUPERUSER/
+    And  the workspace file "blog/db/permissions.sql" should match /GRANT "prodder__blog_prod:read_only" TO "prodder"/
+  Scenario: Roles are created smartly based on connected components in structure
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should not match /create_role_if_not_exists\('_92b'\);/
+    And  the workspace file "blog/db/permissions.sql" should not match /create_role_if_not_exists\('_93b'\);/
+    And  the workspace file "blog/db/permissions.sql" should not match /ALTER ROLE "_92b" WITH/
+    And  the workspace file "blog/db/permissions.sql" should not match /ALTER ROLE "_93b" WITH/
+    And  the workspace file "blog/db/permissions.sql" should not match /GRANT "_92b" TO "_93b"/
+  #TODO: Not sure how to test this
+  Scenario: All loginable (are there other kinds?) superusers are dumped
+  #TODO: Not sure how to test this
+  Scenario: Exhaustively test ACL
+  Scenario: Valid until option is quoted
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should match /VALID UNTIL '.*'/
+  Scenario: Exclude passwords from permissions dump
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should not match /PASSWORD '.*'/
+  Scenario: Roles are created if not existing instead of always being created
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should not match /CREATE ROLE \S*;/
+    And  the workspace file "blog/db/permissions.sql" should match /create_role_if_not_exists\(.*\)/
+  Scenario: Exclude permissions dump if file missing
+    Given the prodder config in "prodder.yml" does not include a permissions file for the "blog" project
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    Then  the workspace file "blog/db/permissions.sql" should not exist
+  Scenario: Exclude permissions dump if permissions object is missing
+    Given the prodder config in "prodder.yml" does not include permissions for the "blog" project
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    Then  the workspace file "blog/db/permissions.sql" should not exist
+  Scenario Outline: Exhaustively test role creation and altering
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should match /<role_creation>/
+    And  the workspace file "blog/db/permissions.sql" should match /<role_altering>/
+    Examples:
+      |                role_creation                |                                role_altering                              |
+      |                   prodder                   |                     ALTER ROLE "prodder" WITH NOSUPERUSER                 |
+      |         prodder__blog_prod:read_only        |           ALTER ROLE "prodder__blog_prod:read_only" WITH NOSUPERUSER      |
+      |                 include_this                |                  ALTER ROLE "include_this" WITH NOSUPERUSER               |
+      |                  _90enva                    |                       ALTER ROLE "_90enva" WITH NOSUPERUSER               |
+      |                   _91b                      |                       ALTER ROLE "_91b" WITH NOSUPERUSER                  |
+      |                   _91qa                     |                       ALTER ROLE "_91qa" WITH NOSUPERUSER                 |
+      |                   _91se                     |                       ALTER ROLE "_91se" WITH NOSUPERUSER                 |
+      |                   _92qa                     |                       ALTER ROLE "_92qa" WITH NOSUPERUSER                 |
+      |                   _92se                     |                       ALTER ROLE "_92se" WITH NOSUPERUSER                 |
+      |                   _93se                     |                       ALTER ROLE "_93se" WITH NOSUPERUSER                 |
+      |                   _94se                     |                       ALTER ROLE "_94se" WITH NOSUPERUSER                 |
+      |prodder__blog_prod:permissions_test:read_only|ALTER ROLE "prodder__blog_prod:permissions_test:read_only" WITH NOSUPERUSER|
+     |prodder__blog_prod:permissions_test:read_only|ALTER ROLE "prodder__blog_prod:permissions_test:read_write" WITH NOSUPERUSER|
+  Scenario Outline: Exhaustively test memeberships
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/permissions.sql" should match /<membership>/
+    Examples:
+      |                                         membership                                        |
+      |         GRANT "prodder__blog_prod:permissions_test:read_write" TO "include_this"          |
+      |                     GRANT "prodder__blog_prod:read_only" TO "prodder"                     |
+      |  GRANT "prodder__blog_prod:permissions_test:read_only" TO "prodder__blog_prod:read_only"  |
+      |                                GRANT "_90enva" TO "_91se"                                 |
+      |                                GRANT "_90enva" TO "_91qa"                                 |
+      |                                GRANT "_90enva" TO "_91b"                                  |
+      |                                GRANT "_91se" TO "_92se"                                   |
+      |                                GRANT "_91qa" TO "_92se"                                   |
+      |                                GRANT "_91qa" TO "_92qa"                                   |
+      |                                GRANT "_91b" TO "_92qa"                                    |
+      |                                GRANT "_91qa" TO "_94se"                                   |
+      |                                GRANT "_92se" TO "_93se"                                   |
+      |                                GRANT "_93se" TO "_94se"                                   |
+  Scenario: Exclude specified tables from structure dump
+    Given the prodder config in "prodder.yml" excludes the table "authors" from the dump of "blog"
+    When  I run `prodder dump -c prodder.yml`
+    Then  the exit status should be 0
+    And   the workspace file "blog/db/structure.sql" should not match /CREATE TABLE authors/
+  Scenario: Verify settings file contents
+    Given I add a customer parameter "c.p" with value "v" in the "blog" project's database
+    When  I run `prodder dump -c prodder.yml`
+    Then the workspace file "blog/db/settings.sql" should match /ALTER DATABASE prodder__blog_prod SET  c.p=v/
+  Scenario: Exclude specified schemas from structure dump
+    Given the prodder config in "prodder.yml" excludes the schema "ads" from the dump of "blog"
+    And   I add a "ads" schema to the "blog" project's database
+    When  I run `prodder dump -c prodder.yml`
+    Then  the exit status should be 0
+    And   the workspace file "blog/db/structure.sql" should not match /CREATE SCHEMA ads/
+  Scenario: Unspecified tables are not dumped to seeds
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/seeds.sql" should not match /COPY comments/
+  Scenario: Grant/revoke/ownership are not in the structure dump
+    When I run `prodder dump -c prodder.yml`
+    Then the workspace file "blog/db/structure.sql" should not match /GRANT/
+    And  the workspace file "blog/db/structure.sql" should not match /ALTER TABLE.*OWNER TO/
+  Scenario: Ownership is not in the seeds dump
+    When I run `prodder dump -c prodder.yml`
+    Then the workspace file "blog/db/seeds.sql" should match /Owner: -/
+    And  the workspace file "blog/db/seeds.sql" should not match /Owner: [^-]/
+  Scenario: Verify quality_checks file contents
+    Given I add a foreign key from table "posts" and column "author_id" to table "authors" and column "author_id" in the "blog" project's database
+    When I run `prodder dump -c prodder.yml`
+    Then the workspace file "blog/db/quality_checks.sql" should match /ADD CONSTRAINT .* FOREIGN KEY/
+    Given I add an index to table "posts" on column "author_id" in the "blog" project's database
+    When I run `prodder dump -c prodder.yml`
+    Then the workspace file "blog/db/quality_checks.sql" should match /CREATE INDEX /
+  Scenario: Maintaining quality checks directly in the structure file
+    Given the prodder config in "prodder.yml" does not include a quality check file for the "blog" project
+    And   I add a foreign key from table "posts" and column "author_id" to table "authors" and column "author_id" in the "blog" project's database
+    When  I run `prodder dump -c prodder.yml`
+    Then  the workspace file "blog/db/quality_checks.sql" should not exist
+    And   the workspace file "blog/db/structure.sql" should match /ADD CONSTRAINT .* FOREIGN KEY/
+  Scenario: Projects can use a YAML file to specify their seed tables
+    Given the prodder config in "prodder.yml" says to read the "blog" seed tables from "db/seeds.yml"
+    And  the "blog" file "db/seeds.yml" contains:
+      """
+      - posts
+      """
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 0
+    And  the workspace file "blog/db/seeds.sql" should match /COPY posts/
+    But  the workspace file "blog/db/seeds.sql" should not match /COPY authors/
+  Scenario: YAML file listing seed tables does not exist
+    Given the prodder config in "prodder.yml" says to read the "blog" seed tables from "db/seeds.yml"
+    But   the "blog" file "db/seeds.yml" does not exist
+    When  I run `prodder dump -c prodder.yml`
+    Then  the exit status should be 1
+    And   the output should contain "No such file: blog/db/seeds.yml"
+  @restore-perms
+  Scenario: pg_dump failed
+    Given the "prodder" role can not read from the "blog" database's tables
+    When I run `prodder dump -c prodder.yml`
+    Then the exit status should be 1
+    And the output should contain "permission denied"

data/features/init.feature ADDED

@@ -0,0 +1,24 @@
+Feature: prodder init
+  Background:
+    Given a prodder config in "prodder.yml" with project: blog
+    And a "blog" git repository
+  Scenario: Clone a project
+    When I run `prodder init -c prodder.yml`
+    Then the exit status should be 0
+    And a directory named "prodder-workspace/blog" should exist
+    And the workspace file "blog/README" should match /Read me/
+  Scenario: Update pre-existing repository on re-initializing
+    Given I run `prodder init -c prodder.yml`
+    And a new commit is already in the "blog" git repository
+    When I run `prodder init -c prodder.yml`
+    Then the new commit should be in the workspace copy of the "blog" repository
+  Scenario: Can't clone repository (no such repo, permissions, whatever)
+    Given I deleted the "blog" git repository
+    When I run `prodder init -c prodder.yml`
+    Then the exit status should be 1
+    And the output should contain "Failed to run 'git clone ./repos/blog"
+    And the output should match /repository.*does not exist/

data/features/lint.feature ADDED

@@ -0,0 +1,46 @@
+Feature: Configuration lint
+  In order to simplify later error handling and hopefully provide
+  rapid feedback on configuration issues, the config file is always
+  run through a linting process to ensure sanity before any other
+  operations are performed.
+  See spec/config_spec for a more detailed test of linting. This
+  test only ensures that the CLI reports lint failures reasonably.
+  Background:
+    Given a prodder config in "prodder.yml" with project: store
+    And no-op versions of these bins are available on my PATH: pg_dump, git
+  Scenario: One of the required project config keys is missing
+    But the "store/db/name" key is missing from "prodder.yml"
+    When I run `prodder ls -c prodder.yml`
+    Then the exit status should be 1
+    And the output should contain:
+      """
+      Missing required configuration key: store/db/name
+      Example configuration:
+      """
+    And the output should contain the example config contents
+  Scenario: pg_dump is not available
+    Given "pg_dump" is not available on my PATH
+    When I run `prodder ls -c prodder.yml`
+    Then the exit status should be 1
+    And the output should contain:
+      """
+      `pg_dump` could not be found on your $PATH.
+      Current PATH:
+      """
+  Scenario: git is not available
+    Given "git" is not available on my PATH
+    When I run `prodder ls -c prodder.yml`
+    Then the exit status should be 1
+    And the output should contain:
+      """
+      `git` could not be found on your $PATH.
+      Current PATH:
+      """

data/features/prodder.feature ADDED

@@ -0,0 +1,76 @@
+Feature: Basic CLI usage
+  Scenario: prodder help
+    When I run `prodder help`
+    Then the output should contain "Tasks:"
+  Scenario: No config file supplied
+    When I run `prodder`
+    Then the output should contain "required option '--config'"
+  Scenario: Config file not found
+    When I run `prodder init -c not-there.yml`
+    Then the output should contain exactly "Config file not found: not-there.yml\n"
+    And the exit status should be 1
+  Scenario: Malformed config file
+    Given a file named "broken.yml" with:
+      """
+      %this IS NOT VALID YAML
+      """
+    When I run `prodder init -c broken.yml`
+    Then the output should contain:
+      """
+      Invalid YAML in config file broken.yml. Current file contents:
+      %this IS NOT VALID YAML
+      """
+  Scenario: No database defined on project
+    Given a file named "prodder.yml" with:
+      """
+      blog:
+        structure_file: db/structure.sql
+        seed_file: db/seeds.sql
+        quality_check_file: db/quality_checks.sql
+        git:
+          origin: git@github.com:pd/blog.git
+          author: prodder auto-commit <pd+prodder@krh.me>
+      """
+    When I run `prodder dump blog -c prodder.yml`
+    Then the output should contain:
+      """
+      Missing required configuration key: blog/db
+      Example configuration:
+      blog:
+        structure_file: db/structure.sql
+        seed_file: db/seeds.sql
+        quality_check_file: db/quality_checks.sql
+        git:
+          origin: git@github.com:your/repo.git
+          author: prodder <prodder@example.com>
+        db:
+          name: database_name
+          host: database.server.example.com
+          user: username
+          password: password
+          tables:
+            - posts
+            - authors
+      """
+  Scenario: Projects named not defined in config file
+    Given a prodder config in "prodder.yml" with project: store
+    When I run `prodder init blog api -c prodder.yml`
+    Then the output should contain "Project not defined: blog, api"
+  Scenario: Listing projects
+    Given a prodder config in "prodder.yml" with projects: store, blog
+    When I run `prodder ls -c prodder.yml`
+    Then the output should contain exactly:
+      """
+      store
+      blog
+      """

data/features/push.feature ADDED

@@ -0,0 +1,25 @@
+Feature: prodder push
+  Background:
+    Given a prodder config in "prodder.yml" with project: blog
+    And a "blog" git repository
+    And I successfully run `prodder init -c prodder.yml`
+    And I successfully run `prodder dump -c prodder.yml`
+    And I successfully run `prodder commit -c prodder.yml`
+  Scenario: Changes are pushed upstream
+    When I successfully run `prodder push -c prodder.yml`
+    Then the exit status should be 0
+    Then the new commit should be in the remote repository
+  Scenario: Push fails due to permissions
+    Given the "blog" git repository does not allow pushing to it
+    When I run `prodder push -c prodder.yml`
+    Then the exit status should be 1
+    And the output should contain "insufficient permission"
+  Scenario: Push fails due to non-fast-forward
+    Given a new commit is already in the "blog" git repository
+    When I run `prodder push -c prodder.yml`
+    Then the exit status should be 1
+    And the output should contain "Refusing to push to remote"