scenic-cascade 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (42) hide show
  1. checksums.yaml +7 -0
  2. data/.dockerignore +6 -0
  3. data/.env +4 -0
  4. data/.rspec +3 -0
  5. data/.rubocop.yml +24 -0
  6. data/.ruby-version +1 -0
  7. data/CHANGELOG.md +5 -0
  8. data/CODE_OF_CONDUCT.md +84 -0
  9. data/Dockerfile +6 -0
  10. data/Gemfile +20 -0
  11. data/Gemfile.lock +231 -0
  12. data/LICENSE +201 -0
  13. data/README.md +95 -0
  14. data/Rakefile +22 -0
  15. data/Steepfile +9 -0
  16. data/docker-compose.yml +16 -0
  17. data/lib/generators/scenic/view/cascade/USAGE +15 -0
  18. data/lib/generators/scenic/view/cascade/cascade_generator.rb +38 -0
  19. data/lib/generators/scenic/view/cascade/templates/db/migrate/create_view.rb.erb +7 -0
  20. data/lib/generators/scenic/view/cascade/templates/db/migrate/update_view.rb.erb +26 -0
  21. data/lib/scenic/cascade/definition_finder.rb +41 -0
  22. data/lib/scenic/cascade/dependency.rb +37 -0
  23. data/lib/scenic/cascade/dependency_finder.rb +51 -0
  24. data/lib/scenic/cascade/dependent_finder.rb +53 -0
  25. data/lib/scenic/cascade/version.rb +7 -0
  26. data/lib/scenic/cascade/view.rb +36 -0
  27. data/lib/scenic/cascade/view_with_version.rb +34 -0
  28. data/lib/scenic/cascade.rb +18 -0
  29. data/manifest.yaml +2 -0
  30. data/rbs_collection.lock.yaml +188 -0
  31. data/rbs_collection.yaml +15 -0
  32. data/sig/generators/scenic/view/cascade/cascade_generator.rbs +13 -0
  33. data/sig/generators/scenic/view/view_generator.rbs +14 -0
  34. data/sig/scenic/cascade/definition_finder.rbs +14 -0
  35. data/sig/scenic/cascade/dependency.rbs +22 -0
  36. data/sig/scenic/cascade/dependency_finder.rbs +11 -0
  37. data/sig/scenic/cascade/dependent_finder.rbs +12 -0
  38. data/sig/scenic/cascade/view.rbs +21 -0
  39. data/sig/scenic/cascade/view_with_version.rbs +17 -0
  40. data/sig/scenic/cascade.rbs +13 -0
  41. data/sig/scenic/definition.rbs +9 -0
  42. metadata +101 -0
data/README.md ADDED
@@ -0,0 +1,95 @@
1
+ # scenic-cascade
2
+
3
+ [![Ruby](https://github.com/akiomik/scenic-cascade/actions/workflows/ci.yml/badge.svg)](https://github.com/akiomik/scenic-cascade/actions/workflows/ci.yml)
4
+
5
+ `scenic-cascade` is a [scenic](https://github.com/scenic-views/scenic) migration file generator that supports cascading view updates.
6
+
7
+ ## Installation
8
+
9
+ Add this line to your application's Gemfile:
10
+
11
+ ```ruby
12
+ gem 'scenic-cascade'
13
+ ```
14
+
15
+ And then execute:
16
+
17
+ ```shell-session
18
+ $ bundle install
19
+ ```
20
+
21
+ Or install it yourself as:
22
+
23
+ ```shell-session
24
+ $ gem install scenic-cascade
25
+ ```
26
+
27
+ ## Usage
28
+
29
+ To generate migration files, use `scenic:view:cascade` generator instead of `scenic:view`.
30
+ The following example generates migration files for `search_results` view.
31
+
32
+ ```shell-session
33
+ $ bin/rails generate scenic:view:cascade search_results
34
+ create db/views/search_results_v01.sql
35
+ create db/migrate/20220714233704_create_search_results.rb
36
+ ```
37
+
38
+ ## How it works
39
+
40
+ Consider a situation where the following three views exist:
41
+
42
+ * `first_results` is a parent view (version 1)
43
+ * `second_results` is a materialized view that depends on `first_results` (version 3)
44
+ * `third_results` is a view that depends on `first_results` and `second_results` (version 2)
45
+
46
+ ```sql
47
+ -- first_results
48
+ SELECT 'foo' AS bar;
49
+
50
+ -- second_results
51
+ SELECT * FROM first_results;
52
+
53
+ -- third_results
54
+ SELECT * FROM first_results UNION SELECT * FROM second_results;
55
+ ```
56
+
57
+ Executing the `scenic:view:cascade` generator for `first_results` in this state will generate the following migration file:
58
+
59
+ ```shell-session
60
+ $ bin/rails generate scenic:view:cascade first_results
61
+ create db/views/first_results_v02.sql
62
+ create db/migrate/20220714233704_update_first_results_to_version_2.rb
63
+ ```
64
+
65
+ Since all dependencies are described, you can execute `bin/rails db:migrate` without changing the migration file.
66
+
67
+ > [!WARNING]
68
+ > Currently, index re-creation is not supported.
69
+ > Please change a migration file to recreate indexes if it contains `drop_view` for materialized views.
70
+
71
+ ```ruby
72
+ class UpdateSearchResultsToVersion2 < ActiveRecord::Migration
73
+ def change
74
+ drop_view :third_results, revert_to_version: 2, materialized: false
75
+ drop_view :second_results, revert_to_version: 3, materialized: true
76
+ replace_view :first_results, version: 2, revert_to_version: 1
77
+ create_view :second_results, version: 3, materialized: true
78
+ create_view :third_results, version: 2, materialized: false
79
+ end
80
+ end
81
+ ```
82
+
83
+ ## Development
84
+
85
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
86
+
87
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
88
+
89
+ ## Contributing
90
+
91
+ Bug reports and pull requests are welcome on GitHub at https://github.com/akiomik/scenic-cascade. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/akiomik/scenic-cascade/blob/main/CODE_OF_CONDUCT.md).
92
+
93
+ ## Code of Conduct
94
+
95
+ Everyone interacting in the `scenic-cascade' project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/akiomik/scenic-cascade/blob/main/CODE_OF_CONDUCT.md).
data/Rakefile ADDED
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'bundler/gem_tasks'
4
+ require 'rspec/core/rake_task'
5
+
6
+ RSpec::Core::RakeTask.new(:spec)
7
+ RSpec::Core::RakeTask.new('spec:small') do |task|
8
+ task.rspec_opts = '--tag size:small'
9
+ end
10
+
11
+ require 'rubocop/rake_task'
12
+
13
+ RuboCop::RakeTask.new
14
+
15
+ desc 'Run `bundle exec steep check`'
16
+ task 'steep:check' do
17
+ sh 'bundle exec steep check'
18
+ end
19
+
20
+ task steep: ['steep:check']
21
+
22
+ task default: %i[spec rubocop steep:check]
data/Steepfile ADDED
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ target :lib
4
+
5
+ # target :test do
6
+ # signature "sig", "sig-private"
7
+ #
8
+ # check "spec"
9
+ # end
@@ -0,0 +1,16 @@
1
+ version: '3.9'
2
+ services:
3
+ lib:
4
+ build: .
5
+ volumes:
6
+ - .:/scenic-cascade
7
+ env_file:
8
+ .env
9
+ links:
10
+ - postgres
11
+ postgres:
12
+ image: postgres:14-alpine
13
+ env_file:
14
+ .env
15
+ ports:
16
+ - 5432:5432
@@ -0,0 +1,15 @@
1
+ Description:
2
+ Create a database view migration with re-creating of dependent views.
3
+
4
+ Example:
5
+ bin/rails generate scenic:view:cascade search_results
6
+
7
+ This will create:
8
+ db/views/search_results_v01.sql
9
+ db/migrate/20220714233704_create_search_results.rb
10
+
11
+ bin/rails generate scenic:view:cascade search_results
12
+
13
+ This will create:
14
+ db/views/search_results_v02.sql
15
+ db/migrate/20220714234219_update_search_results_to_version_2.rb
@@ -0,0 +1,38 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'generators/scenic/view/view_generator'
4
+
5
+ module Scenic
6
+ module Generators
7
+ module View
8
+ # Create a database view migration with re-creating of dependent views
9
+ class CascadeGenerator < Scenic::Generators::ViewGenerator
10
+ source_root File.expand_path('templates', __dir__ || '.')
11
+
12
+ def create_migration_file
13
+ if creating_new_view? || destroying_initial_view?
14
+ migration_template('db/migrate/create_view.rb.erb',
15
+ "db/migrate/create_#{plural_file_name}.rb")
16
+ else
17
+ migration_template('db/migrate/update_view.rb.erb',
18
+ "db/migrate/update_#{plural_file_name}_to_version_#{version}.rb")
19
+ end
20
+ end
21
+
22
+ private
23
+
24
+ def dependents
25
+ return @dependents if @dependents.present?
26
+
27
+ deps = Scenic::Cascade.view_dependents_of(plural_name, recursive: true)
28
+ @dependents ||= deps.map do |dep|
29
+ name = dep.from.name
30
+ version = Scenic::Cascade.find_latest_definition_of(name).version.to_i
31
+ is_materialized = dep.from.materialized?
32
+ Scenic::Cascade::ViewWithVersion.new(name: name, version: version, materialized: is_materialized)
33
+ end.uniq
34
+ end
35
+ end
36
+ end
37
+ end
38
+ end
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+
3
+ class <%= migration_class_name %> < <%= activerecord_migration_class %>
4
+ def change
5
+ create_view <%= formatted_plural_name %><%= create_view_options %>
6
+ end
7
+ end
@@ -0,0 +1,26 @@
1
+ # frozen_string_literal: true
2
+
3
+ class <%= migration_class_name %> < <%= activerecord_migration_class %>
4
+ def change
5
+ <%- dependents.reverse_each do |dep| -%>
6
+ drop_view <%= dep.formatted_name %>,
7
+ revert_to_version: <%= dep.version %>,
8
+ materialized: <%= dep.materialized? %>
9
+ <%- end -%>
10
+
11
+ <%- if materialized? -%>
12
+ update_view <%= formatted_plural_name %>,
13
+ version: <%= version %>,
14
+ revert_to_version: <%= previous_version %>,
15
+ materialized: <%= no_data? ? "{ no_data: true }" : true %>
16
+ <%- else -%>
17
+ update_view <%= formatted_plural_name %>, version: <%= version %>, revert_to_version: <%= previous_version %>
18
+ <%- end -%>
19
+
20
+ <%- dependents.each do |dep| -%>
21
+ create_view <%= dep.formatted_name %>,
22
+ version: <%= dep.version %>,
23
+ materialized: <%= dep.materialized? %>
24
+ <%- end -%>
25
+ end
26
+ end
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'scenic/definition'
4
+
5
+ module Scenic
6
+ # Visualize database view dependencies for Scenic
7
+ module Cascade
8
+ # Finds view definitions
9
+ module DefinitionFinder
10
+ def self.included(klass)
11
+ klass.extend(ClassMethods)
12
+ end
13
+
14
+ # Provides class methods to injected class
15
+ module ClassMethods
16
+ def find_definitions_of(view_name)
17
+ go(view_name, 1)
18
+ end
19
+
20
+ def find_latest_definition_of(view_name)
21
+ latest_definition = find_definitions_of(view_name).last
22
+ unless latest_definition.nil?
23
+ # @type var latest_definition: Scenic::Definition
24
+ return latest_definition
25
+ end
26
+
27
+ raise ArgumentError, "View #{view_name} does not exist"
28
+ end
29
+
30
+ private
31
+
32
+ def go(view_name, version)
33
+ definition = Scenic::Definition.new(view_name, version)
34
+ return [] unless File.exist?(definition.full_path)
35
+
36
+ [definition, *go(view_name, version + 1)]
37
+ end
38
+ end
39
+ end
40
+ end
41
+ end
@@ -0,0 +1,37 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'view'
4
+
5
+ module Scenic
6
+ module Cascade
7
+ # Represents a view dependency
8
+ class Dependency
9
+ attr_reader :from, :to
10
+
11
+ def initialize(from:, to:)
12
+ raise TypeError unless from.is_a?(Scenic::Cascade::View)
13
+ raise TypeError unless to.is_a?(Scenic::Cascade::View)
14
+
15
+ @from = from
16
+ @to = to
17
+ end
18
+
19
+ def ==(other)
20
+ from == other.from && to == other.to
21
+ end
22
+
23
+ # mermaid-like syntax
24
+ def to_s
25
+ "#{from.name}[#{from}] --> #{to.name}[#{to}]"
26
+ end
27
+
28
+ alias inspect to_s
29
+
30
+ def self.from_hash(hash)
31
+ from = Scenic::Cascade::View.new(name: hash['from'], materialized: hash['from_materialized'])
32
+ to = Scenic::Cascade::View.new(name: hash['to'], materialized: hash['to_materialized'])
33
+ new(from: from, to: to)
34
+ end
35
+ end
36
+ end
37
+ end
@@ -0,0 +1,51 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'dependency'
4
+
5
+ module Scenic
6
+ module Cascade
7
+ # Finds database views which the specified view depends
8
+ module DependencyFinder
9
+ def self.included(klass)
10
+ klass.extend(ClassMethods)
11
+ end
12
+
13
+ # Provides class methods to injected class
14
+ module ClassMethods
15
+ DEPENDENCY_SQL = <<-SQL
16
+ SELECT DISTINCT
17
+ dependee.relname AS to
18
+ , dependee.relkind = 'm' AS to_materialized
19
+ , depender.relname AS from
20
+ , depender.relkind = 'm' AS from_materialized
21
+ FROM pg_depend d
22
+ JOIN pg_rewrite r
23
+ ON d.objid = r.oid
24
+ JOIN pg_class AS dependee
25
+ ON d.refobjid = dependee.oid
26
+ JOIN pg_class AS depender
27
+ ON r.ev_class = depender.oid
28
+ WHERE depender.relname = ?
29
+ AND dependee.relname != depender.relname
30
+ AND dependee.relkind in ('m', 'v')
31
+ ORDER BY dependee.relname;
32
+ SQL
33
+
34
+ private_constant :DEPENDENCY_SQL
35
+
36
+ def view_dependencies_of(view_name, recursive: false)
37
+ query = ActiveRecord::Base.sanitize_sql_array([DEPENDENCY_SQL, view_name])
38
+ raw_dependencies = ActiveRecord::Base.connection.select_all(query).to_a
39
+ dependencies = raw_dependencies.map { |dep| Scenic::Cascade::Dependency.from_hash(dep) }
40
+
41
+ return [] if dependencies.empty?
42
+ return dependencies unless recursive
43
+
44
+ dependencies.flat_map do |dependency|
45
+ [dependency, *view_dependencies_of(dependency.to.name)]
46
+ end
47
+ end
48
+ end
49
+ end
50
+ end
51
+ end
@@ -0,0 +1,53 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'dependency'
4
+
5
+ module Scenic
6
+ module Cascade
7
+ # Finds database views that depend on the specified view
8
+ module DependentFinder
9
+ def self.included(klass)
10
+ klass.extend(ClassMethods)
11
+ end
12
+
13
+ # Provides class methods to injected class
14
+ module ClassMethods
15
+ DEPENDANT_SQL = <<-SQL
16
+ SELECT DISTINCT
17
+ dependee.relname AS to
18
+ , dependee.relkind = 'm' AS to_materialized
19
+ , depender.relname AS from
20
+ , depender.relkind = 'm' AS from_materialized
21
+ FROM pg_depend d
22
+ JOIN pg_rewrite r
23
+ ON d.objid = r.oid
24
+ JOIN pg_class AS dependee
25
+ ON d.refobjid = dependee.oid
26
+ JOIN pg_class AS depender
27
+ ON r.ev_class = depender.oid
28
+ WHERE dependee.relname = ?
29
+ AND depender.relname != dependee.relname
30
+ AND depender.relkind in ('m', 'v')
31
+ ORDER BY depender.relname;
32
+ SQL
33
+
34
+ private_constant :DEPENDANT_SQL
35
+
36
+ def view_dependents_of(view_name, recursive: false)
37
+ query = ActiveRecord::Base.sanitize_sql_array([DEPENDANT_SQL, view_name])
38
+ raw_dependencies = ActiveRecord::Base.connection.select_all(query).to_a
39
+ dependencies = raw_dependencies.map { |dep| Scenic::Cascade::Dependency.from_hash(dep) }
40
+
41
+ return [] if dependencies.empty?
42
+ return dependencies unless recursive
43
+
44
+ dependencies.flat_map do |dependency|
45
+ [dependency, *view_dependents_of(dependency.from.name)]
46
+ end
47
+ end
48
+
49
+ alias view_dependants_of view_dependents_of
50
+ end
51
+ end
52
+ end
53
+ end
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Scenic
4
+ module Cascade
5
+ VERSION = '0.1.0'
6
+ end
7
+ end
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Scenic
4
+ module Cascade
5
+ # Represents a view
6
+ class View
7
+ attr_reader :name
8
+
9
+ def initialize(name:, materialized:)
10
+ raise TypeError unless name.is_a?(String)
11
+ raise TypeError unless materialized.is_a?(TrueClass) || materialized.is_a?(FalseClass)
12
+
13
+ @name = name
14
+ @materialized = materialized
15
+ end
16
+
17
+ def materialized?
18
+ @materialized
19
+ end
20
+
21
+ def formatted_name
22
+ name.include?('.') ? %("#{name}") : ":#{name}"
23
+ end
24
+
25
+ def ==(other)
26
+ name == other.name && materialized? == other.materialized?
27
+ end
28
+
29
+ def to_s
30
+ "#{name} (#{materialized? ? 'MV' : 'V'})"
31
+ end
32
+
33
+ alias inspect to_s
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,34 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Scenic
4
+ module Cascade
5
+ # Represents a view with version
6
+ class ViewWithVersion < Scenic::Cascade::View
7
+ attr_reader :version
8
+
9
+ def initialize(name:, version:, materialized:)
10
+ super(name: name, materialized: materialized)
11
+
12
+ raise TypeError unless version.is_a?(Integer)
13
+
14
+ @version = version
15
+ end
16
+
17
+ def ==(other)
18
+ name == other.name && version == other.version && materialized? == other.materialized?
19
+ end
20
+
21
+ def to_s
22
+ "#{name}_v#{formatted_version} (#{materialized? ? 'MV' : 'V'})"
23
+ end
24
+
25
+ alias inspect to_s
26
+
27
+ private
28
+
29
+ def formatted_version
30
+ version.to_s.rjust(2, '0')
31
+ end
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'active_record'
4
+ require 'scenic'
5
+
6
+ require_relative 'cascade/version'
7
+ require_relative 'cascade/definition_finder'
8
+ require_relative 'cascade/dependency_finder'
9
+ require_relative 'cascade/dependent_finder'
10
+
11
+ module Scenic
12
+ # Manage database view dependencies for Scenic
13
+ module Cascade
14
+ include Scenic::Cascade::DefinitionFinder
15
+ include Scenic::Cascade::DependencyFinder
16
+ include Scenic::Cascade::DependentFinder
17
+ end
18
+ end
data/manifest.yaml ADDED
@@ -0,0 +1,2 @@
1
+ dependencies:
2
+ - name: pathname