columns_trace 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: cd2b5b47e4fa2ca409bf7dd531c8766a6ff6487b50e996f04425577c07a65667
4
+ data.tar.gz: 28fa32928a82d9c1df876bad8002d677bd7c7e5f0a001f56ad6170b9b87e8f2c
5
+ SHA512:
6
+ metadata.gz: df2740562388acd113084e28f50940d2bbf0d0d934abe0b8b4def515158f2bb36ba29179ae317e7cf77665039e5d77d2c15d76ef044b3e4f56f314f046882ba0
7
+ data.tar.gz: 77fcfa0230679ba9efc329225c6eda32f6ad915bc0bee79df227b724bc1c75ba3cb5e7b51943b1142ba36586f74d94cdeec9bc9873daa807f1e009c4c53f8e4f
data/CHANGELOG.md ADDED
@@ -0,0 +1,3 @@
1
+ ## master (unreleased)
2
+
3
+ - First release
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2023 fatkodima
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,119 @@
1
+ # ColumnsTrace
2
+
3
+ [![Build Status](https://github.com/fatkodima/columns_trace/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/fatkodima/columns_trace/actions/workflows/test.yml)
4
+
5
+ Detects unnecessary selected database columns in Rails controllers, `ActiveJob` and `Sidekiq` jobs.
6
+
7
+ ## Requirements
8
+
9
+ - ruby 2.7+
10
+ - rails 6.0+
11
+
12
+ ## Installation
13
+
14
+ Add this line to your application's Gemfile:
15
+
16
+ ```ruby
17
+ gem 'columns_trace'
18
+ ```
19
+
20
+ And then run:
21
+
22
+ ```sh
23
+ $ bundle install
24
+ ```
25
+
26
+ ## Usage
27
+
28
+ Hit a controller action or run `ActiveJob` (or `Sidekiq`) job, open `log/columns_trace.log`,
29
+ and see the output:
30
+
31
+ ```
32
+ ImportsController#create
33
+ 1 User record: unused columns - "bio", "settings"; used columns - "id", "email", "name",
34
+ "account_id", "created_at", "updated_at"
35
+ ↳ app/controllers/application_controller.rb:32:in `block in <class:ApplicationController>'
36
+
37
+ 1 Account record: unused columns - "settings", "logo", "updated_at";
38
+ used columns - "id", "plan_id"
39
+ ↳ app/controllers/application_controller.rb:33:in `block in <class:ApplicationController>'
40
+
41
+ 10 Project records: unused columns - "description", "avatar", "url", "created_at", "updated_at";
42
+ used columns - "id", "user_id"
43
+ ↳ app/models/user.rb:46: in `projects'
44
+ app/services/imports_service.rb:129: in `import_projects'
45
+ app/controllers/imports_controller.rb:49:in `index'
46
+
47
+ ImportProjectJob
48
+ 1 User record: unused columns - "email", "name", "bio", "created_at", "updated_at";
49
+ used columns - "id", "settings"
50
+ ↳ app/jobs/import_project_job.rb:23:in `perform'
51
+
52
+ 1 Project record: unused columns - "description", "avatar", "settings", "created_at",
53
+ "updated_at"; used columns - "id", "user_id", "url"
54
+ ↳ app/jobs/import_project_job.rb:24:in `perform'
55
+ ```
56
+
57
+ ## Configuration
58
+
59
+ You can override the following default options:
60
+
61
+ ```ruby
62
+ # config/initializers/columns_trace.rb
63
+
64
+ ColumnsTrace.configure do |config|
65
+ # Configures models that will be ignored.
66
+ # Always adds Rails' internal `ActiveRecord::SchemaMigration`
67
+ # and `ActiveRecord::InternalMetadata` models by default.
68
+ config.ignored_models = []
69
+
70
+ # Configures columns that will be ignored.
71
+ #
72
+ # Global setting
73
+ # config.ignored_columns = [:updated_at]
74
+ # Per-model setting
75
+ # config.ignored_columns = [:updated_at, { User => :admin }]
76
+ config.ignored_columns = []
77
+
78
+ # The logger to log to.
79
+ # Defaults to logger, that outputs to `log/columns_trace.log` file
80
+ # when the gem is used in the Rails app.
81
+ config.logger = nil
82
+
83
+ # Controls the contents of the printed backtrace.
84
+ # Is set to the default Rails.backtrace_cleaner when the gem is used in the Rails app.
85
+ config.backtrace_cleaner = ->(backtrace) { backtrace }
86
+ end
87
+ ```
88
+
89
+ `Sidekiq` integration is disabled by default. You need to explicitly enable it:
90
+
91
+ ```ruby
92
+ # config/initializers/columns_trace.rb
93
+
94
+ ColumnsTrace.enable_sidekiq_tracing!
95
+ ```
96
+
97
+ ## Development
98
+
99
+ After checking out the repo, run `bundle install` to install dependencies. Then, run `rake` to run the linter and tests.
100
+
101
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
102
+
103
+ ## Additional resources
104
+
105
+ Alternatives:
106
+
107
+ - [snip_snip](https://github.com/kddnewton/snip_snip) - archived, supports only controllers
108
+
109
+ Interesting reads:
110
+
111
+ - [Reasons why SELECT * is bad for SQL performance](https://tanelpoder.com/posts/reasons-why-select-star-is-bad-for-sql-performance/)
112
+
113
+ ## Contributing
114
+
115
+ Bug reports and pull requests are welcome on GitHub at https://github.com/fatkodima/columns_trace.
116
+
117
+ ## License
118
+
119
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
@@ -0,0 +1,25 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ColumnsTrace
4
+ ActiveSupport.on_load(:active_record) do
5
+ after_find do
6
+ backtrace = Kernel.caller
7
+
8
+ if ColumnsTrace.backtrace_cleaner
9
+ backtrace = ColumnsTrace.backtrace_cleaner.call(backtrace)
10
+ end
11
+
12
+ Registry.register(self, backtrace) unless ColumnsTrace.ignored_model?(self.class)
13
+ end
14
+ end
15
+
16
+ ActiveSupport.on_load(:action_controller) do
17
+ before_action { Registry.clear }
18
+ after_action { Reporter.report("#{self.class.name}##{action_name}") }
19
+ end
20
+
21
+ ActiveSupport.on_load(:active_job) do
22
+ before_perform { Registry.clear }
23
+ after_perform { Reporter.report(self.class.name) }
24
+ end
25
+ end
@@ -0,0 +1,11 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ColumnsTrace
4
+ # @private
5
+ class Railtie < Rails::Railtie
6
+ initializer "columns_trace.set_configs" do
7
+ ColumnsTrace.backtrace_cleaner = Rails.backtrace_cleaner
8
+ ColumnsTrace.logger = ActiveSupport::Logger.new(Rails.root.join("log", "columns_trace.log"))
9
+ end
10
+ end
11
+ end
@@ -0,0 +1,28 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ColumnsTrace
4
+ # @private
5
+ # Note: can use ActiveSupport::IsolatedExecutionState instead of this module for rails 7.0+.
6
+ module Registry
7
+ Entry = Struct.new(:model, :record, :backtrace)
8
+
9
+ class << self
10
+ def register(record, backtrace)
11
+ state << Entry.new(record.class, record, backtrace)
12
+ end
13
+
14
+ def clear
15
+ state.clear
16
+ end
17
+
18
+ def entries
19
+ state
20
+ end
21
+
22
+ private
23
+ def state
24
+ Thread.current[:columns_trace] ||= []
25
+ end
26
+ end
27
+ end
28
+ end
@@ -0,0 +1,53 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ColumnsTrace
4
+ # @private
5
+ class Reporter
6
+ def self.report(title)
7
+ new.report(title)
8
+ end
9
+
10
+ def report(title)
11
+ lines = []
12
+
13
+ Registry.entries.group_by(&:model).each do |model, entries|
14
+ lines.concat(lines_for_entries(model, entries))
15
+ end
16
+
17
+ if lines.any?
18
+ ColumnsTrace.logger.info("#{title}\n#{lines.join("\n")}")
19
+ end
20
+ end
21
+
22
+ private
23
+ def lines_for_entries(model, entries)
24
+ lines = []
25
+
26
+ entries.group_by(&:backtrace).each do |backtrace, grouped_entries|
27
+ records = grouped_entries.map(&:record)
28
+
29
+ accessed = accessed_fields(records)
30
+ unused = records.first.attributes.keys - accessed
31
+ unused.reject! { |column| ColumnsTrace.ignored_column?(model, column) }
32
+
33
+ if unused.any?
34
+ records_text = "record".pluralize(records.size)
35
+ lines << <<-MSG
36
+ #{records.size} #{model.name} #{records_text}: unused columns - #{format_columns(unused)}; used columns - #{format_columns(accessed)}
37
+ ↳ #{backtrace.join("\n ")}
38
+ MSG
39
+ end
40
+ end
41
+
42
+ lines
43
+ end
44
+
45
+ def accessed_fields(records)
46
+ records.map(&:accessed_fields).flatten.uniq
47
+ end
48
+
49
+ def format_columns(columns)
50
+ columns.map(&:inspect).join(", ")
51
+ end
52
+ end
53
+ end
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ColumnsTrace
4
+ # @private
5
+ class SidekiqMiddleware
6
+ def call(worker, _job, _queue)
7
+ Registry.clear
8
+ yield
9
+ Reporter.report(worker.class.name)
10
+ end
11
+ end
12
+ end
13
+
14
+ Sidekiq.configure_server do |config|
15
+ config.server_middleware do |chain|
16
+ chain.add(ColumnsTrace::SidekiqMiddleware)
17
+ end
18
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ColumnsTrace
4
+ VERSION = "0.1.0"
5
+ end
@@ -0,0 +1,100 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record"
4
+
5
+ require_relative "columns_trace/registry"
6
+ require_relative "columns_trace/rails_integration"
7
+ require_relative "columns_trace/reporter"
8
+ require_relative "columns_trace/version"
9
+ require_relative "columns_trace/railtie" if defined?(Rails)
10
+
11
+ module ColumnsTrace
12
+ class << self
13
+ # @private
14
+ attr_reader :ignored_models
15
+
16
+ # Configures models that will be ignored.
17
+ #
18
+ # @example
19
+ # ColumnsTrace.ignored_models = [Settings]
20
+ #
21
+ # Always adds Rails' internal `ActiveRecord::SchemaMigration`
22
+ # and `ActiveRecord::InternalMetadata` models by default.
23
+ #
24
+ def ignored_models=(models)
25
+ @ignored_models = Array(models) | [ActiveRecord::SchemaMigration, ActiveRecord::InternalMetadata]
26
+ end
27
+
28
+ # @private
29
+ def ignored_model?(model)
30
+ ignored_models.include?(model)
31
+ end
32
+
33
+ # Configures columns that will be ignored.
34
+ #
35
+ # @example Global setting
36
+ # ColumnsTrace.ignored_columns = [:updated_at]
37
+ # @example Per-model setting
38
+ # ColumnsTrace.ignored_columns = [:updated_at, { User => :admin }]
39
+ #
40
+ attr_accessor :ignored_columns
41
+
42
+ # @private
43
+ def ignored_column?(model, column)
44
+ ignored_columns.any? do |value|
45
+ if value.is_a?(Hash)
46
+ columns = value[model] || value[model.name] || value[model.name.to_sym]
47
+ if columns
48
+ columns = Array(columns).map(&:to_s)
49
+ columns.include?(column)
50
+ end
51
+ else
52
+ value.to_s == column
53
+ end
54
+ end
55
+ end
56
+
57
+ # Allows to set the logger.
58
+ # Defaults to logger, that outputs to `log/columns_trace.log` file
59
+ # when inside a rails application.
60
+ #
61
+ attr_accessor :logger
62
+
63
+ # @private
64
+ attr_reader :backtrace_cleaner
65
+
66
+ # Allows to set a backtrace_cleaner used to clean backtrace before printing them.
67
+ # Defaults to `Rails.backtrace_cleaner` when inside a rails application.
68
+ #
69
+ def backtrace_cleaner=(cleaner)
70
+ @backtrace_cleaner =
71
+ if cleaner.respond_to?(:clean)
72
+ ->(backtrace) { cleaner.clean(backtrace) }
73
+ else
74
+ cleaner
75
+ end
76
+ end
77
+
78
+ # Enables integration with Sidekiq, which is disabled by default.
79
+ #
80
+ def enable_sidekiq_tracing!
81
+ require_relative "columns_trace/sidekiq_integration"
82
+ true
83
+ end
84
+
85
+ # A convenient method to configure this gem.
86
+ #
87
+ # @example
88
+ # ColumnsTrace.configure do |config|
89
+ # # ...
90
+ # end
91
+ #
92
+ def configure
93
+ yield self
94
+ end
95
+ end
96
+
97
+ self.ignored_models = []
98
+ self.ignored_columns = []
99
+ self.backtrace_cleaner = ->(backtrace) { backtrace }
100
+ end
@@ -0,0 +1 @@
1
+ 3a86c4ee2d1db41bbbe96c557949abd5be8b29f3577316d831a57bd81fe781ae84b631620ebdc3946c5b47747dc9aed358afbdfbf615608ad8c46996b05ab9a7
@@ -0,0 +1 @@
1
+ 93cbef26b77c38ff5046394a34240e582de1e6d96386235192ed44afd310938076d2f501309478ae03288fd98db9bd43c37f3c50503c0534cc011ac94fa75be4
metadata ADDED
@@ -0,0 +1,72 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: columns_trace
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - fatkodima
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2023-10-04 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: activerecord
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '6.0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '6.0'
27
+ description:
28
+ email:
29
+ - fatkodima123@gmail.com
30
+ executables: []
31
+ extensions: []
32
+ extra_rdoc_files: []
33
+ files:
34
+ - CHANGELOG.md
35
+ - LICENSE.txt
36
+ - README.md
37
+ - lib/columns_trace.rb
38
+ - lib/columns_trace/rails_integration.rb
39
+ - lib/columns_trace/railtie.rb
40
+ - lib/columns_trace/registry.rb
41
+ - lib/columns_trace/reporter.rb
42
+ - lib/columns_trace/sidekiq_integration.rb
43
+ - lib/columns_trace/version.rb
44
+ - test/dummy/tmp/development_secret.txt
45
+ - test/dummy/tmp/local_secret.txt
46
+ homepage: https://github.com/fatkodima/columns_trace
47
+ licenses:
48
+ - MIT
49
+ metadata:
50
+ homepage_uri: https://github.com/fatkodima/columns_trace
51
+ source_code_uri: https://github.com/fatkodima/columns_trace
52
+ changelog_uri: https://github.com/fatkodima/columns_trace/blob/master/CHANGELOG.md
53
+ post_install_message:
54
+ rdoc_options: []
55
+ require_paths:
56
+ - lib
57
+ required_ruby_version: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: 2.7.0
62
+ required_rubygems_version: !ruby/object:Gem::Requirement
63
+ requirements:
64
+ - - ">="
65
+ - !ruby/object:Gem::Version
66
+ version: '0'
67
+ requirements: []
68
+ rubygems_version: 3.4.10
69
+ signing_key:
70
+ specification_version: 4
71
+ summary: Find unnecessary selected database columns.
72
+ test_files: []