scenic-cascade 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.dockerignore +6 -0
- data/.env +4 -0
- data/.rspec +3 -0
- data/.rubocop.yml +24 -0
- data/.ruby-version +1 -0
- data/CHANGELOG.md +5 -0
- data/CODE_OF_CONDUCT.md +84 -0
- data/Dockerfile +6 -0
- data/Gemfile +20 -0
- data/Gemfile.lock +231 -0
- data/LICENSE +201 -0
- data/README.md +95 -0
- data/Rakefile +22 -0
- data/Steepfile +9 -0
- data/docker-compose.yml +16 -0
- data/lib/generators/scenic/view/cascade/USAGE +15 -0
- data/lib/generators/scenic/view/cascade/cascade_generator.rb +38 -0
- data/lib/generators/scenic/view/cascade/templates/db/migrate/create_view.rb.erb +7 -0
- data/lib/generators/scenic/view/cascade/templates/db/migrate/update_view.rb.erb +26 -0
- data/lib/scenic/cascade/definition_finder.rb +41 -0
- data/lib/scenic/cascade/dependency.rb +37 -0
- data/lib/scenic/cascade/dependency_finder.rb +51 -0
- data/lib/scenic/cascade/dependent_finder.rb +53 -0
- data/lib/scenic/cascade/version.rb +7 -0
- data/lib/scenic/cascade/view.rb +36 -0
- data/lib/scenic/cascade/view_with_version.rb +34 -0
- data/lib/scenic/cascade.rb +18 -0
- data/manifest.yaml +2 -0
- data/rbs_collection.lock.yaml +188 -0
- data/rbs_collection.yaml +15 -0
- data/sig/generators/scenic/view/cascade/cascade_generator.rbs +13 -0
- data/sig/generators/scenic/view/view_generator.rbs +14 -0
- data/sig/scenic/cascade/definition_finder.rbs +14 -0
- data/sig/scenic/cascade/dependency.rbs +22 -0
- data/sig/scenic/cascade/dependency_finder.rbs +11 -0
- data/sig/scenic/cascade/dependent_finder.rbs +12 -0
- data/sig/scenic/cascade/view.rbs +21 -0
- data/sig/scenic/cascade/view_with_version.rbs +17 -0
- data/sig/scenic/cascade.rbs +13 -0
- data/sig/scenic/definition.rbs +9 -0
- metadata +101 -0
data/README.md
ADDED
@@ -0,0 +1,95 @@
|
|
1
|
+
# scenic-cascade
|
2
|
+
|
3
|
+
[![Ruby](https://github.com/akiomik/scenic-cascade/actions/workflows/ci.yml/badge.svg)](https://github.com/akiomik/scenic-cascade/actions/workflows/ci.yml)
|
4
|
+
|
5
|
+
`scenic-cascade` is a [scenic](https://github.com/scenic-views/scenic) migration file generator that supports cascading view updates.
|
6
|
+
|
7
|
+
## Installation
|
8
|
+
|
9
|
+
Add this line to your application's Gemfile:
|
10
|
+
|
11
|
+
```ruby
|
12
|
+
gem 'scenic-cascade'
|
13
|
+
```
|
14
|
+
|
15
|
+
And then execute:
|
16
|
+
|
17
|
+
```shell-session
|
18
|
+
$ bundle install
|
19
|
+
```
|
20
|
+
|
21
|
+
Or install it yourself as:
|
22
|
+
|
23
|
+
```shell-session
|
24
|
+
$ gem install scenic-cascade
|
25
|
+
```
|
26
|
+
|
27
|
+
## Usage
|
28
|
+
|
29
|
+
To generate migration files, use `scenic:view:cascade` generator instead of `scenic:view`.
|
30
|
+
The following example generates migration files for `search_results` view.
|
31
|
+
|
32
|
+
```shell-session
|
33
|
+
$ bin/rails generate scenic:view:cascade search_results
|
34
|
+
create db/views/search_results_v01.sql
|
35
|
+
create db/migrate/20220714233704_create_search_results.rb
|
36
|
+
```
|
37
|
+
|
38
|
+
## How it works
|
39
|
+
|
40
|
+
Consider a situation where the following three views exist:
|
41
|
+
|
42
|
+
* `first_results` is a parent view (version 1)
|
43
|
+
* `second_results` is a materialized view that depends on `first_results` (version 3)
|
44
|
+
* `third_results` is a view that depends on `first_results` and `second_results` (version 2)
|
45
|
+
|
46
|
+
```sql
|
47
|
+
-- first_results
|
48
|
+
SELECT 'foo' AS bar;
|
49
|
+
|
50
|
+
-- second_results
|
51
|
+
SELECT * FROM first_results;
|
52
|
+
|
53
|
+
-- third_results
|
54
|
+
SELECT * FROM first_results UNION SELECT * FROM second_results;
|
55
|
+
```
|
56
|
+
|
57
|
+
Executing the `scenic:view:cascade` generator for `first_results` in this state will generate the following migration file:
|
58
|
+
|
59
|
+
```shell-session
|
60
|
+
$ bin/rails generate scenic:view:cascade first_results
|
61
|
+
create db/views/first_results_v02.sql
|
62
|
+
create db/migrate/20220714233704_update_first_results_to_version_2.rb
|
63
|
+
```
|
64
|
+
|
65
|
+
Since all dependencies are described, you can execute `bin/rails db:migrate` without changing the migration file.
|
66
|
+
|
67
|
+
> [!WARNING]
|
68
|
+
> Currently, index re-creation is not supported.
|
69
|
+
> Please change a migration file to recreate indexes if it contains `drop_view` for materialized views.
|
70
|
+
|
71
|
+
```ruby
|
72
|
+
class UpdateSearchResultsToVersion2 < ActiveRecord::Migration
|
73
|
+
def change
|
74
|
+
drop_view :third_results, revert_to_version: 2, materialized: false
|
75
|
+
drop_view :second_results, revert_to_version: 3, materialized: true
|
76
|
+
replace_view :first_results, version: 2, revert_to_version: 1
|
77
|
+
create_view :second_results, version: 3, materialized: true
|
78
|
+
create_view :third_results, version: 2, materialized: false
|
79
|
+
end
|
80
|
+
end
|
81
|
+
```
|
82
|
+
|
83
|
+
## Development
|
84
|
+
|
85
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
86
|
+
|
87
|
+
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
88
|
+
|
89
|
+
## Contributing
|
90
|
+
|
91
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/akiomik/scenic-cascade. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/akiomik/scenic-cascade/blob/main/CODE_OF_CONDUCT.md).
|
92
|
+
|
93
|
+
## Code of Conduct
|
94
|
+
|
95
|
+
Everyone interacting in the `scenic-cascade' project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/akiomik/scenic-cascade/blob/main/CODE_OF_CONDUCT.md).
|
data/Rakefile
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'bundler/gem_tasks'
|
4
|
+
require 'rspec/core/rake_task'
|
5
|
+
|
6
|
+
RSpec::Core::RakeTask.new(:spec)
|
7
|
+
RSpec::Core::RakeTask.new('spec:small') do |task|
|
8
|
+
task.rspec_opts = '--tag size:small'
|
9
|
+
end
|
10
|
+
|
11
|
+
require 'rubocop/rake_task'
|
12
|
+
|
13
|
+
RuboCop::RakeTask.new
|
14
|
+
|
15
|
+
desc 'Run `bundle exec steep check`'
|
16
|
+
task 'steep:check' do
|
17
|
+
sh 'bundle exec steep check'
|
18
|
+
end
|
19
|
+
|
20
|
+
task steep: ['steep:check']
|
21
|
+
|
22
|
+
task default: %i[spec rubocop steep:check]
|
data/Steepfile
ADDED
data/docker-compose.yml
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
Description:
|
2
|
+
Create a database view migration with re-creating of dependent views.
|
3
|
+
|
4
|
+
Example:
|
5
|
+
bin/rails generate scenic:view:cascade search_results
|
6
|
+
|
7
|
+
This will create:
|
8
|
+
db/views/search_results_v01.sql
|
9
|
+
db/migrate/20220714233704_create_search_results.rb
|
10
|
+
|
11
|
+
bin/rails generate scenic:view:cascade search_results
|
12
|
+
|
13
|
+
This will create:
|
14
|
+
db/views/search_results_v02.sql
|
15
|
+
db/migrate/20220714234219_update_search_results_to_version_2.rb
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'generators/scenic/view/view_generator'
|
4
|
+
|
5
|
+
module Scenic
|
6
|
+
module Generators
|
7
|
+
module View
|
8
|
+
# Create a database view migration with re-creating of dependent views
|
9
|
+
class CascadeGenerator < Scenic::Generators::ViewGenerator
|
10
|
+
source_root File.expand_path('templates', __dir__ || '.')
|
11
|
+
|
12
|
+
def create_migration_file
|
13
|
+
if creating_new_view? || destroying_initial_view?
|
14
|
+
migration_template('db/migrate/create_view.rb.erb',
|
15
|
+
"db/migrate/create_#{plural_file_name}.rb")
|
16
|
+
else
|
17
|
+
migration_template('db/migrate/update_view.rb.erb',
|
18
|
+
"db/migrate/update_#{plural_file_name}_to_version_#{version}.rb")
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
private
|
23
|
+
|
24
|
+
def dependents
|
25
|
+
return @dependents if @dependents.present?
|
26
|
+
|
27
|
+
deps = Scenic::Cascade.view_dependents_of(plural_name, recursive: true)
|
28
|
+
@dependents ||= deps.map do |dep|
|
29
|
+
name = dep.from.name
|
30
|
+
version = Scenic::Cascade.find_latest_definition_of(name).version.to_i
|
31
|
+
is_materialized = dep.from.materialized?
|
32
|
+
Scenic::Cascade::ViewWithVersion.new(name: name, version: version, materialized: is_materialized)
|
33
|
+
end.uniq
|
34
|
+
end
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -0,0 +1,26 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
class <%= migration_class_name %> < <%= activerecord_migration_class %>
|
4
|
+
def change
|
5
|
+
<%- dependents.reverse_each do |dep| -%>
|
6
|
+
drop_view <%= dep.formatted_name %>,
|
7
|
+
revert_to_version: <%= dep.version %>,
|
8
|
+
materialized: <%= dep.materialized? %>
|
9
|
+
<%- end -%>
|
10
|
+
|
11
|
+
<%- if materialized? -%>
|
12
|
+
update_view <%= formatted_plural_name %>,
|
13
|
+
version: <%= version %>,
|
14
|
+
revert_to_version: <%= previous_version %>,
|
15
|
+
materialized: <%= no_data? ? "{ no_data: true }" : true %>
|
16
|
+
<%- else -%>
|
17
|
+
update_view <%= formatted_plural_name %>, version: <%= version %>, revert_to_version: <%= previous_version %>
|
18
|
+
<%- end -%>
|
19
|
+
|
20
|
+
<%- dependents.each do |dep| -%>
|
21
|
+
create_view <%= dep.formatted_name %>,
|
22
|
+
version: <%= dep.version %>,
|
23
|
+
materialized: <%= dep.materialized? %>
|
24
|
+
<%- end -%>
|
25
|
+
end
|
26
|
+
end
|
@@ -0,0 +1,41 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'scenic/definition'
|
4
|
+
|
5
|
+
module Scenic
|
6
|
+
# Visualize database view dependencies for Scenic
|
7
|
+
module Cascade
|
8
|
+
# Finds view definitions
|
9
|
+
module DefinitionFinder
|
10
|
+
def self.included(klass)
|
11
|
+
klass.extend(ClassMethods)
|
12
|
+
end
|
13
|
+
|
14
|
+
# Provides class methods to injected class
|
15
|
+
module ClassMethods
|
16
|
+
def find_definitions_of(view_name)
|
17
|
+
go(view_name, 1)
|
18
|
+
end
|
19
|
+
|
20
|
+
def find_latest_definition_of(view_name)
|
21
|
+
latest_definition = find_definitions_of(view_name).last
|
22
|
+
unless latest_definition.nil?
|
23
|
+
# @type var latest_definition: Scenic::Definition
|
24
|
+
return latest_definition
|
25
|
+
end
|
26
|
+
|
27
|
+
raise ArgumentError, "View #{view_name} does not exist"
|
28
|
+
end
|
29
|
+
|
30
|
+
private
|
31
|
+
|
32
|
+
def go(view_name, version)
|
33
|
+
definition = Scenic::Definition.new(view_name, version)
|
34
|
+
return [] unless File.exist?(definition.full_path)
|
35
|
+
|
36
|
+
[definition, *go(view_name, version + 1)]
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
@@ -0,0 +1,37 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'view'
|
4
|
+
|
5
|
+
module Scenic
|
6
|
+
module Cascade
|
7
|
+
# Represents a view dependency
|
8
|
+
class Dependency
|
9
|
+
attr_reader :from, :to
|
10
|
+
|
11
|
+
def initialize(from:, to:)
|
12
|
+
raise TypeError unless from.is_a?(Scenic::Cascade::View)
|
13
|
+
raise TypeError unless to.is_a?(Scenic::Cascade::View)
|
14
|
+
|
15
|
+
@from = from
|
16
|
+
@to = to
|
17
|
+
end
|
18
|
+
|
19
|
+
def ==(other)
|
20
|
+
from == other.from && to == other.to
|
21
|
+
end
|
22
|
+
|
23
|
+
# mermaid-like syntax
|
24
|
+
def to_s
|
25
|
+
"#{from.name}[#{from}] --> #{to.name}[#{to}]"
|
26
|
+
end
|
27
|
+
|
28
|
+
alias inspect to_s
|
29
|
+
|
30
|
+
def self.from_hash(hash)
|
31
|
+
from = Scenic::Cascade::View.new(name: hash['from'], materialized: hash['from_materialized'])
|
32
|
+
to = Scenic::Cascade::View.new(name: hash['to'], materialized: hash['to_materialized'])
|
33
|
+
new(from: from, to: to)
|
34
|
+
end
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
@@ -0,0 +1,51 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'dependency'
|
4
|
+
|
5
|
+
module Scenic
|
6
|
+
module Cascade
|
7
|
+
# Finds database views which the specified view depends
|
8
|
+
module DependencyFinder
|
9
|
+
def self.included(klass)
|
10
|
+
klass.extend(ClassMethods)
|
11
|
+
end
|
12
|
+
|
13
|
+
# Provides class methods to injected class
|
14
|
+
module ClassMethods
|
15
|
+
DEPENDENCY_SQL = <<-SQL
|
16
|
+
SELECT DISTINCT
|
17
|
+
dependee.relname AS to
|
18
|
+
, dependee.relkind = 'm' AS to_materialized
|
19
|
+
, depender.relname AS from
|
20
|
+
, depender.relkind = 'm' AS from_materialized
|
21
|
+
FROM pg_depend d
|
22
|
+
JOIN pg_rewrite r
|
23
|
+
ON d.objid = r.oid
|
24
|
+
JOIN pg_class AS dependee
|
25
|
+
ON d.refobjid = dependee.oid
|
26
|
+
JOIN pg_class AS depender
|
27
|
+
ON r.ev_class = depender.oid
|
28
|
+
WHERE depender.relname = ?
|
29
|
+
AND dependee.relname != depender.relname
|
30
|
+
AND dependee.relkind in ('m', 'v')
|
31
|
+
ORDER BY dependee.relname;
|
32
|
+
SQL
|
33
|
+
|
34
|
+
private_constant :DEPENDENCY_SQL
|
35
|
+
|
36
|
+
def view_dependencies_of(view_name, recursive: false)
|
37
|
+
query = ActiveRecord::Base.sanitize_sql_array([DEPENDENCY_SQL, view_name])
|
38
|
+
raw_dependencies = ActiveRecord::Base.connection.select_all(query).to_a
|
39
|
+
dependencies = raw_dependencies.map { |dep| Scenic::Cascade::Dependency.from_hash(dep) }
|
40
|
+
|
41
|
+
return [] if dependencies.empty?
|
42
|
+
return dependencies unless recursive
|
43
|
+
|
44
|
+
dependencies.flat_map do |dependency|
|
45
|
+
[dependency, *view_dependencies_of(dependency.to.name)]
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
@@ -0,0 +1,53 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'dependency'
|
4
|
+
|
5
|
+
module Scenic
|
6
|
+
module Cascade
|
7
|
+
# Finds database views that depend on the specified view
|
8
|
+
module DependentFinder
|
9
|
+
def self.included(klass)
|
10
|
+
klass.extend(ClassMethods)
|
11
|
+
end
|
12
|
+
|
13
|
+
# Provides class methods to injected class
|
14
|
+
module ClassMethods
|
15
|
+
DEPENDANT_SQL = <<-SQL
|
16
|
+
SELECT DISTINCT
|
17
|
+
dependee.relname AS to
|
18
|
+
, dependee.relkind = 'm' AS to_materialized
|
19
|
+
, depender.relname AS from
|
20
|
+
, depender.relkind = 'm' AS from_materialized
|
21
|
+
FROM pg_depend d
|
22
|
+
JOIN pg_rewrite r
|
23
|
+
ON d.objid = r.oid
|
24
|
+
JOIN pg_class AS dependee
|
25
|
+
ON d.refobjid = dependee.oid
|
26
|
+
JOIN pg_class AS depender
|
27
|
+
ON r.ev_class = depender.oid
|
28
|
+
WHERE dependee.relname = ?
|
29
|
+
AND depender.relname != dependee.relname
|
30
|
+
AND depender.relkind in ('m', 'v')
|
31
|
+
ORDER BY depender.relname;
|
32
|
+
SQL
|
33
|
+
|
34
|
+
private_constant :DEPENDANT_SQL
|
35
|
+
|
36
|
+
def view_dependents_of(view_name, recursive: false)
|
37
|
+
query = ActiveRecord::Base.sanitize_sql_array([DEPENDANT_SQL, view_name])
|
38
|
+
raw_dependencies = ActiveRecord::Base.connection.select_all(query).to_a
|
39
|
+
dependencies = raw_dependencies.map { |dep| Scenic::Cascade::Dependency.from_hash(dep) }
|
40
|
+
|
41
|
+
return [] if dependencies.empty?
|
42
|
+
return dependencies unless recursive
|
43
|
+
|
44
|
+
dependencies.flat_map do |dependency|
|
45
|
+
[dependency, *view_dependents_of(dependency.from.name)]
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
alias view_dependants_of view_dependents_of
|
50
|
+
end
|
51
|
+
end
|
52
|
+
end
|
53
|
+
end
|
@@ -0,0 +1,36 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Scenic
|
4
|
+
module Cascade
|
5
|
+
# Represents a view
|
6
|
+
class View
|
7
|
+
attr_reader :name
|
8
|
+
|
9
|
+
def initialize(name:, materialized:)
|
10
|
+
raise TypeError unless name.is_a?(String)
|
11
|
+
raise TypeError unless materialized.is_a?(TrueClass) || materialized.is_a?(FalseClass)
|
12
|
+
|
13
|
+
@name = name
|
14
|
+
@materialized = materialized
|
15
|
+
end
|
16
|
+
|
17
|
+
def materialized?
|
18
|
+
@materialized
|
19
|
+
end
|
20
|
+
|
21
|
+
def formatted_name
|
22
|
+
name.include?('.') ? %("#{name}") : ":#{name}"
|
23
|
+
end
|
24
|
+
|
25
|
+
def ==(other)
|
26
|
+
name == other.name && materialized? == other.materialized?
|
27
|
+
end
|
28
|
+
|
29
|
+
def to_s
|
30
|
+
"#{name} (#{materialized? ? 'MV' : 'V'})"
|
31
|
+
end
|
32
|
+
|
33
|
+
alias inspect to_s
|
34
|
+
end
|
35
|
+
end
|
36
|
+
end
|
@@ -0,0 +1,34 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Scenic
|
4
|
+
module Cascade
|
5
|
+
# Represents a view with version
|
6
|
+
class ViewWithVersion < Scenic::Cascade::View
|
7
|
+
attr_reader :version
|
8
|
+
|
9
|
+
def initialize(name:, version:, materialized:)
|
10
|
+
super(name: name, materialized: materialized)
|
11
|
+
|
12
|
+
raise TypeError unless version.is_a?(Integer)
|
13
|
+
|
14
|
+
@version = version
|
15
|
+
end
|
16
|
+
|
17
|
+
def ==(other)
|
18
|
+
name == other.name && version == other.version && materialized? == other.materialized?
|
19
|
+
end
|
20
|
+
|
21
|
+
def to_s
|
22
|
+
"#{name}_v#{formatted_version} (#{materialized? ? 'MV' : 'V'})"
|
23
|
+
end
|
24
|
+
|
25
|
+
alias inspect to_s
|
26
|
+
|
27
|
+
private
|
28
|
+
|
29
|
+
def formatted_version
|
30
|
+
version.to_s.rjust(2, '0')
|
31
|
+
end
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'active_record'
|
4
|
+
require 'scenic'
|
5
|
+
|
6
|
+
require_relative 'cascade/version'
|
7
|
+
require_relative 'cascade/definition_finder'
|
8
|
+
require_relative 'cascade/dependency_finder'
|
9
|
+
require_relative 'cascade/dependent_finder'
|
10
|
+
|
11
|
+
module Scenic
|
12
|
+
# Manage database view dependencies for Scenic
|
13
|
+
module Cascade
|
14
|
+
include Scenic::Cascade::DefinitionFinder
|
15
|
+
include Scenic::Cascade::DependencyFinder
|
16
|
+
include Scenic::Cascade::DependentFinder
|
17
|
+
end
|
18
|
+
end
|
data/manifest.yaml
ADDED