pg_objects 1.4.4 → 1.4.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/copilot-instructions.md +131 -0
- data/.github/workflows/ci.yml +1 -1
- data/.gitignore +1 -0
- data/Gemfile +1 -0
- data/Gemfile.lock +94 -96
- data/README.md +45 -0
- data/Rakefile +5 -0
- data/benchmark.txt +71 -0
- data/bin/benchmark +486 -0
- data/lib/pg_objects/version.rb +1 -1
- data/pg_objects.gemspec +1 -1
- metadata +7 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d237ef303ba52b2c6fbbf974ce269578863e83ce7c128bfb38e645a1c1603711
|
|
4
|
+
data.tar.gz: 2678a55d53168f225b63d042cde515a2d3955567261cbdfb9724782eaaa71c91
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d3ed0de00e000dd4ee275931a5fd356aa53119641a1db20b3cede6864c05e3dd0fb44c7021fff055f4ac8b35d156973435ff0deb6bdf686cde03cef3f66864c4
|
|
7
|
+
data.tar.gz: c6981b7a271149f406e1f85a4b81cc58a6b4aa420d40a87d9fcec3cf56270848f7854d9dd074bb92842099b619ad625107a524243e8019b817a972d26bc74aa2
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
# PG Objects
|
|
2
|
+
PG Objects is a Ruby gem for managing PostgreSQL database objects like triggers and functions. It provides a simple manager that handles dependencies between database objects and integrates with Rails applications.
|
|
3
|
+
|
|
4
|
+
Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
|
|
5
|
+
|
|
6
|
+
## Working Effectively
|
|
7
|
+
- Bootstrap, build, and test the repository:
|
|
8
|
+
- `gem install --user-install bundler`
|
|
9
|
+
- `export PATH="$HOME/.local/share/gem/ruby/3.2.0/bin:$PATH"`
|
|
10
|
+
- `bundle config set --local path 'vendor/bundle'`
|
|
11
|
+
- `bundle install` -- takes 45-60 seconds. NEVER CANCEL. Set timeout to 120+ seconds.
|
|
12
|
+
- `bundle exec rspec spec` -- takes 4 seconds. NEVER CANCEL. Set timeout to 60+ seconds.
|
|
13
|
+
- `bundle exec rubocop` -- takes 3 seconds. NEVER CANCEL. Set timeout to 60+ seconds.
|
|
14
|
+
- `bundle exec bundle-audit check` -- takes 2 seconds (first run downloads advisory database). NEVER CANCEL. Set timeout to 180+ seconds for first run.
|
|
15
|
+
- Performance benchmarking:
|
|
16
|
+
- `bundle exec rake benchmark` -- takes 1 second. NEVER CANCEL. Set timeout to 30+ seconds.
|
|
17
|
+
- Interactive console:
|
|
18
|
+
- `./bin/console` -- launches IRB with pg_objects loaded
|
|
19
|
+
- Install gem locally:
|
|
20
|
+
- `bundle exec rake install` -- takes 9 seconds. NEVER CANCEL. Set timeout to 120+ seconds.
|
|
21
|
+
|
|
22
|
+
## Validation
|
|
23
|
+
- Always run through the complete test suite after making changes: `bundle exec rspec spec`
|
|
24
|
+
- ALWAYS run linting before completing work: `bundle exec rubocop`
|
|
25
|
+
- Always run bundle audit to check for security vulnerabilities: `bundle exec bundle-audit check`
|
|
26
|
+
- Test parsing functionality with sample SQL files to ensure changes work correctly
|
|
27
|
+
- NEVER CANCEL builds or tests - they complete quickly (under 60 seconds)
|
|
28
|
+
|
|
29
|
+
## Common Tasks
|
|
30
|
+
The following are outputs from frequently run commands. Reference them instead of viewing, searching, or running bash commands to save time.
|
|
31
|
+
|
|
32
|
+
### Repository Root Structure
|
|
33
|
+
```
|
|
34
|
+
.
|
|
35
|
+
├── .github/ # CI/CD workflows (ci.yml, bundle_audit.yml, publish.yml)
|
|
36
|
+
├── .rspec # RSpec configuration
|
|
37
|
+
├── .rubocop.yml # RuboCop linting configuration
|
|
38
|
+
├── bin/
|
|
39
|
+
│ ├── setup # Setup script (runs bundle install)
|
|
40
|
+
│ ├── console # Interactive console
|
|
41
|
+
│ └── benchmark # Performance benchmark tool
|
|
42
|
+
├── lib/
|
|
43
|
+
│ ├── pg_objects.rb # Main entry point
|
|
44
|
+
│ ├── pg_objects/ # Core library files
|
|
45
|
+
│ │ ├── config.rb
|
|
46
|
+
│ │ ├── manager.rb
|
|
47
|
+
│ │ ├── parser.rb
|
|
48
|
+
│ │ └── parsed_object/ # SQL object parsers
|
|
49
|
+
│ └── generators/pg_objects/install/ # Rails generator
|
|
50
|
+
├── spec/ # RSpec test files
|
|
51
|
+
├── Gemfile # Gem dependencies
|
|
52
|
+
├── pg_objects.gemspec # Gem specification
|
|
53
|
+
├── Rakefile # Rake tasks (spec, benchmark)
|
|
54
|
+
└── README.md # Documentation
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### Key Files and Directories
|
|
58
|
+
- **lib/pg_objects.rb**: Main entry point that requires all components
|
|
59
|
+
- **lib/pg_objects/manager.rb**: Core manager for database objects
|
|
60
|
+
- **lib/pg_objects/parser.rb**: SQL parsing and dependency extraction
|
|
61
|
+
- **lib/pg_objects/parsed_object/**: Specific parsers for different SQL object types
|
|
62
|
+
- **spec/**: Complete test suite with fixtures
|
|
63
|
+
- **bin/benchmark**: Performance benchmarking tool with detailed metrics
|
|
64
|
+
- **Gemfile**: Development and test dependencies (RSpec, RuboCop, etc.)
|
|
65
|
+
|
|
66
|
+
### Gemfile Dependencies
|
|
67
|
+
- **Runtime**: activerecord, dry-auto_inject, dry-configurable, pg_query, railties
|
|
68
|
+
- **Development/Test**: rspec, rubocop, bundler-audit, faker, pry-byebug
|
|
69
|
+
|
|
70
|
+
### Common Command Outputs
|
|
71
|
+
#### `bundle exec rspec spec` (Expected: ~4 seconds, 54 examples, 0 failures)
|
|
72
|
+
```
|
|
73
|
+
54 examples, 0 failures
|
|
74
|
+
Finished in 2.24 seconds
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
#### `bundle exec rubocop` (Expected: ~3 seconds, 60 files, no offenses)
|
|
78
|
+
```
|
|
79
|
+
60 files inspected, no offenses detected
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
#### `bundle exec bundle-audit check` (Expected: ~2 seconds after initial setup)
|
|
83
|
+
```
|
|
84
|
+
No vulnerabilities found
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
#### `bundle exec rake benchmark` (Expected: ~1 second)
|
|
88
|
+
```
|
|
89
|
+
PG Objects Performance Benchmark
|
|
90
|
+
==================================================
|
|
91
|
+
File I/O Performance: ~100,000+ files/s
|
|
92
|
+
Parsing Performance: ~7,000 files/s
|
|
93
|
+
Full Workflow Performance: ~6,000 objects/s
|
|
94
|
+
Benchmark completed successfully!
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## Development Workflow
|
|
98
|
+
1. Always run `bundle install` after cloning or changing dependencies
|
|
99
|
+
2. Make changes to code in lib/ directory
|
|
100
|
+
3. Add or update tests in spec/ directory for any changes
|
|
101
|
+
4. Run `bundle exec rspec spec` to ensure all tests pass
|
|
102
|
+
5. Run `bundle exec rubocop` to ensure code style compliance
|
|
103
|
+
6. Use `bundle exec rake benchmark` to test performance impact
|
|
104
|
+
7. Run `bundle exec bundle-audit check` for security validation
|
|
105
|
+
|
|
106
|
+
## Troubleshooting
|
|
107
|
+
- If bundler is not found: `gem install --user-install bundler && export PATH="$HOME/.local/share/gem/ruby/3.2.0/bin:$PATH"`
|
|
108
|
+
- If bundle install fails with permission errors: `bundle config set --local path 'vendor/bundle'`
|
|
109
|
+
- Ruby version required: >= 3.2.0 (tested with 3.2, 3.3, 3.4)
|
|
110
|
+
- The gem requires PostgreSQL and uses pg_query for SQL parsing
|
|
111
|
+
- Dependencies include ActiveRecord, dry gems, and memery for caching
|
|
112
|
+
|
|
113
|
+
## Testing SQL Parsing
|
|
114
|
+
Create test SQL files with dependencies:
|
|
115
|
+
```sql
|
|
116
|
+
--!depends_on other_function
|
|
117
|
+
CREATE OR REPLACE FUNCTION my_function(param INTEGER)
|
|
118
|
+
RETURNS INTEGER AS $$
|
|
119
|
+
BEGIN
|
|
120
|
+
RETURN param * 2;
|
|
121
|
+
END;
|
|
122
|
+
$$ LANGUAGE plpgsql;
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Test parsing with:
|
|
126
|
+
```ruby
|
|
127
|
+
parser = PgObjects::Parser.new
|
|
128
|
+
content = File.read('path/to/file.sql')
|
|
129
|
+
object_name = parser.load(content).fetch_object_name
|
|
130
|
+
dependencies = parser.fetch_directives[:depends_on]
|
|
131
|
+
```
|
data/.github/workflows/ci.yml
CHANGED
data/.gitignore
CHANGED
data/Gemfile
CHANGED
data/Gemfile.lock
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
pg_objects (1.4.
|
|
4
|
+
pg_objects (1.4.6)
|
|
5
5
|
activerecord (>= 6.1.7.0, < 9)
|
|
6
6
|
dry-auto_inject (~> 1)
|
|
7
7
|
dry-configurable (~> 1)
|
|
8
8
|
dry-container (= 0.11.0)
|
|
9
9
|
dry-monads (~> 1.6)
|
|
10
|
-
memery (>= 1.5, < 1.
|
|
10
|
+
memery (>= 1.5, < 1.9)
|
|
11
11
|
pg_query (>= 5, < 7)
|
|
12
12
|
railties (>= 4, < 9)
|
|
13
13
|
rake-hooks (~> 1)
|
|
@@ -15,9 +15,9 @@ PATH
|
|
|
15
15
|
GEM
|
|
16
16
|
remote: https://rubygems.org/
|
|
17
17
|
specs:
|
|
18
|
-
actionpack (8.
|
|
19
|
-
actionview (= 8.
|
|
20
|
-
activesupport (= 8.
|
|
18
|
+
actionpack (8.1.2)
|
|
19
|
+
actionview (= 8.1.2)
|
|
20
|
+
activesupport (= 8.1.2)
|
|
21
21
|
nokogiri (>= 1.8.5)
|
|
22
22
|
rack (>= 2.2.4)
|
|
23
23
|
rack-session (>= 1.0.1)
|
|
@@ -25,50 +25,51 @@ GEM
|
|
|
25
25
|
rails-dom-testing (~> 2.2)
|
|
26
26
|
rails-html-sanitizer (~> 1.6)
|
|
27
27
|
useragent (~> 0.16)
|
|
28
|
-
actionview (8.
|
|
29
|
-
activesupport (= 8.
|
|
28
|
+
actionview (8.1.2)
|
|
29
|
+
activesupport (= 8.1.2)
|
|
30
30
|
builder (~> 3.1)
|
|
31
31
|
erubi (~> 1.11)
|
|
32
32
|
rails-dom-testing (~> 2.2)
|
|
33
33
|
rails-html-sanitizer (~> 1.6)
|
|
34
|
-
activemodel (8.
|
|
35
|
-
activesupport (= 8.
|
|
36
|
-
activerecord (8.
|
|
37
|
-
activemodel (= 8.
|
|
38
|
-
activesupport (= 8.
|
|
34
|
+
activemodel (8.1.2)
|
|
35
|
+
activesupport (= 8.1.2)
|
|
36
|
+
activerecord (8.1.2)
|
|
37
|
+
activemodel (= 8.1.2)
|
|
38
|
+
activesupport (= 8.1.2)
|
|
39
39
|
timeout (>= 0.4.0)
|
|
40
|
-
activesupport (8.
|
|
40
|
+
activesupport (8.1.2)
|
|
41
41
|
base64
|
|
42
|
-
benchmark (>= 0.3)
|
|
43
42
|
bigdecimal
|
|
44
43
|
concurrent-ruby (~> 1.0, >= 1.3.1)
|
|
45
44
|
connection_pool (>= 2.2.5)
|
|
46
45
|
drb
|
|
47
46
|
i18n (>= 1.6, < 2)
|
|
47
|
+
json
|
|
48
48
|
logger (>= 1.4.2)
|
|
49
49
|
minitest (>= 5.1)
|
|
50
50
|
securerandom (>= 0.3)
|
|
51
51
|
tzinfo (~> 2.0, >= 2.0.5)
|
|
52
52
|
uri (>= 0.13.1)
|
|
53
53
|
ast (2.4.3)
|
|
54
|
-
base64 (0.
|
|
55
|
-
benchmark (0.
|
|
56
|
-
bigdecimal (
|
|
54
|
+
base64 (0.3.0)
|
|
55
|
+
benchmark (0.5.0)
|
|
56
|
+
bigdecimal (4.0.1)
|
|
57
57
|
binding_of_caller (1.0.1)
|
|
58
58
|
debug_inspector (>= 1.2.0)
|
|
59
59
|
builder (3.3.0)
|
|
60
|
-
bundler-audit (0.9.
|
|
61
|
-
bundler (>= 1.2.0
|
|
60
|
+
bundler-audit (0.9.3)
|
|
61
|
+
bundler (>= 1.2.0)
|
|
62
62
|
thor (~> 1.0)
|
|
63
|
-
byebug (
|
|
63
|
+
byebug (13.0.0)
|
|
64
|
+
reline (>= 0.6.0)
|
|
64
65
|
coderay (1.1.3)
|
|
65
|
-
concurrent-ruby (1.3.
|
|
66
|
-
connection_pool (
|
|
66
|
+
concurrent-ruby (1.3.6)
|
|
67
|
+
connection_pool (3.0.2)
|
|
67
68
|
crass (1.0.6)
|
|
68
|
-
date (3.
|
|
69
|
+
date (3.5.1)
|
|
69
70
|
debug_inspector (1.2.0)
|
|
70
71
|
diff-lcs (1.6.2)
|
|
71
|
-
drb (2.2.
|
|
72
|
+
drb (2.2.3)
|
|
72
73
|
dry-auto_inject (1.1.0)
|
|
73
74
|
dry-core (~> 1.1)
|
|
74
75
|
zeitwerk (~> 2.6)
|
|
@@ -77,7 +78,7 @@ GEM
|
|
|
77
78
|
zeitwerk (~> 2.6)
|
|
78
79
|
dry-container (0.11.0)
|
|
79
80
|
concurrent-ruby (~> 1.0)
|
|
80
|
-
dry-core (1.
|
|
81
|
+
dry-core (1.2.0)
|
|
81
82
|
concurrent-ruby (~> 1.0)
|
|
82
83
|
logger
|
|
83
84
|
zeitwerk (~> 2.6)
|
|
@@ -85,140 +86,134 @@ GEM
|
|
|
85
86
|
concurrent-ruby (~> 1.0)
|
|
86
87
|
dry-core (~> 1.1)
|
|
87
88
|
zeitwerk (~> 2.6)
|
|
89
|
+
erb (6.0.1)
|
|
88
90
|
erubi (1.13.1)
|
|
89
|
-
faker (3.5.
|
|
91
|
+
faker (3.5.3)
|
|
90
92
|
i18n (>= 1.8.11, < 2)
|
|
91
|
-
google-protobuf (4.
|
|
93
|
+
google-protobuf (4.33.4)
|
|
92
94
|
bigdecimal
|
|
93
95
|
rake (>= 13)
|
|
94
|
-
|
|
95
|
-
bigdecimal
|
|
96
|
-
rake (>= 13)
|
|
97
|
-
google-protobuf (4.30.2-arm64-darwin)
|
|
98
|
-
bigdecimal
|
|
99
|
-
rake (>= 13)
|
|
100
|
-
google-protobuf (4.30.2-x86_64-darwin)
|
|
101
|
-
bigdecimal
|
|
102
|
-
rake (>= 13)
|
|
103
|
-
google-protobuf (4.30.2-x86_64-linux)
|
|
104
|
-
bigdecimal
|
|
105
|
-
rake (>= 13)
|
|
106
|
-
i18n (1.14.7)
|
|
96
|
+
i18n (1.14.8)
|
|
107
97
|
concurrent-ruby (~> 1.0)
|
|
108
|
-
io-console (0.8.
|
|
109
|
-
irb (1.
|
|
98
|
+
io-console (0.8.2)
|
|
99
|
+
irb (1.16.0)
|
|
110
100
|
pp (>= 0.6.0)
|
|
111
101
|
rdoc (>= 4.0.0)
|
|
112
102
|
reline (>= 0.4.2)
|
|
113
|
-
json (2.
|
|
103
|
+
json (2.18.0)
|
|
114
104
|
language_server-protocol (3.17.0.5)
|
|
115
105
|
lint_roller (1.1.0)
|
|
116
106
|
logger (1.7.0)
|
|
117
|
-
loofah (2.
|
|
107
|
+
loofah (2.25.0)
|
|
118
108
|
crass (~> 1.0.2)
|
|
119
109
|
nokogiri (>= 1.12.0)
|
|
120
|
-
memery (1.
|
|
110
|
+
memery (1.8.0)
|
|
121
111
|
method_source (1.1.0)
|
|
122
|
-
minitest (
|
|
123
|
-
|
|
112
|
+
minitest (6.0.1)
|
|
113
|
+
prism (~> 1.5)
|
|
114
|
+
nokogiri (1.19.0-aarch64-linux-gnu)
|
|
124
115
|
racc (~> 1.4)
|
|
125
|
-
nokogiri (1.
|
|
116
|
+
nokogiri (1.19.0-aarch64-linux-musl)
|
|
126
117
|
racc (~> 1.4)
|
|
127
|
-
nokogiri (1.
|
|
118
|
+
nokogiri (1.19.0-arm-linux-gnu)
|
|
128
119
|
racc (~> 1.4)
|
|
129
|
-
nokogiri (1.
|
|
120
|
+
nokogiri (1.19.0-arm-linux-musl)
|
|
130
121
|
racc (~> 1.4)
|
|
131
|
-
nokogiri (1.
|
|
122
|
+
nokogiri (1.19.0-arm64-darwin)
|
|
132
123
|
racc (~> 1.4)
|
|
133
|
-
nokogiri (1.
|
|
124
|
+
nokogiri (1.19.0-x86_64-darwin)
|
|
134
125
|
racc (~> 1.4)
|
|
135
|
-
nokogiri (1.
|
|
126
|
+
nokogiri (1.19.0-x86_64-linux-gnu)
|
|
136
127
|
racc (~> 1.4)
|
|
137
|
-
nokogiri (1.
|
|
128
|
+
nokogiri (1.19.0-x86_64-linux-musl)
|
|
138
129
|
racc (~> 1.4)
|
|
139
130
|
parallel (1.27.0)
|
|
140
|
-
parser (3.3.
|
|
131
|
+
parser (3.3.10.1)
|
|
141
132
|
ast (~> 2.4.1)
|
|
142
133
|
racc
|
|
143
134
|
pg_query (6.1.0)
|
|
144
135
|
google-protobuf (>= 3.25.3)
|
|
145
|
-
pp (0.6.
|
|
136
|
+
pp (0.6.3)
|
|
146
137
|
prettyprint
|
|
147
138
|
prettyprint (0.2.0)
|
|
148
|
-
prism (1.
|
|
139
|
+
prism (1.8.0)
|
|
149
140
|
proc_to_ast (0.2.0)
|
|
150
141
|
parser
|
|
151
142
|
rouge
|
|
152
143
|
unparser
|
|
153
|
-
pry (0.
|
|
144
|
+
pry (0.16.0)
|
|
154
145
|
coderay (~> 1.1)
|
|
155
146
|
method_source (~> 1.0)
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
147
|
+
reline (>= 0.6.0)
|
|
148
|
+
pry-byebug (3.12.0)
|
|
149
|
+
byebug (~> 13.0)
|
|
150
|
+
pry (>= 0.13, < 0.17)
|
|
151
|
+
psych (5.3.1)
|
|
160
152
|
date
|
|
161
153
|
stringio
|
|
162
154
|
racc (1.8.1)
|
|
163
|
-
rack (3.
|
|
155
|
+
rack (3.2.4)
|
|
164
156
|
rack-session (2.1.1)
|
|
165
157
|
base64 (>= 0.1.0)
|
|
166
158
|
rack (>= 3.0.0)
|
|
167
159
|
rack-test (2.2.0)
|
|
168
160
|
rack (>= 1.3)
|
|
169
|
-
rackup (2.
|
|
161
|
+
rackup (2.3.1)
|
|
170
162
|
rack (>= 3)
|
|
171
|
-
rails-dom-testing (2.
|
|
163
|
+
rails-dom-testing (2.3.0)
|
|
172
164
|
activesupport (>= 5.0.0)
|
|
173
165
|
minitest
|
|
174
166
|
nokogiri (>= 1.6)
|
|
175
167
|
rails-html-sanitizer (1.6.2)
|
|
176
168
|
loofah (~> 2.21)
|
|
177
169
|
nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0)
|
|
178
|
-
railties (8.
|
|
179
|
-
actionpack (= 8.
|
|
180
|
-
activesupport (= 8.
|
|
170
|
+
railties (8.1.2)
|
|
171
|
+
actionpack (= 8.1.2)
|
|
172
|
+
activesupport (= 8.1.2)
|
|
181
173
|
irb (~> 1.13)
|
|
182
174
|
rackup (>= 1.0.0)
|
|
183
175
|
rake (>= 12.2)
|
|
184
176
|
thor (~> 1.0, >= 1.2.2)
|
|
177
|
+
tsort (>= 0.2)
|
|
185
178
|
zeitwerk (~> 2.6)
|
|
186
179
|
rainbow (3.1.1)
|
|
187
|
-
rake (13.3.
|
|
180
|
+
rake (13.3.1)
|
|
188
181
|
rake-hooks (1.2.3)
|
|
189
182
|
rake
|
|
190
|
-
rdoc (
|
|
183
|
+
rdoc (7.1.0)
|
|
184
|
+
erb
|
|
191
185
|
psych (>= 4.0.0)
|
|
192
|
-
|
|
193
|
-
|
|
186
|
+
tsort
|
|
187
|
+
regexp_parser (2.11.3)
|
|
188
|
+
reline (0.6.3)
|
|
194
189
|
io-console (~> 0.5)
|
|
195
|
-
rouge (4.
|
|
196
|
-
rspec (3.13.
|
|
190
|
+
rouge (4.7.0)
|
|
191
|
+
rspec (3.13.2)
|
|
197
192
|
rspec-core (~> 3.13.0)
|
|
198
193
|
rspec-expectations (~> 3.13.0)
|
|
199
194
|
rspec-mocks (~> 3.13.0)
|
|
200
|
-
rspec-core (3.13.
|
|
195
|
+
rspec-core (3.13.6)
|
|
201
196
|
rspec-support (~> 3.13.0)
|
|
202
197
|
rspec-expectations (3.13.5)
|
|
203
198
|
diff-lcs (>= 1.2.0, < 2.0)
|
|
204
199
|
rspec-support (~> 3.13.0)
|
|
205
|
-
rspec-mocks (3.13.
|
|
200
|
+
rspec-mocks (3.13.7)
|
|
206
201
|
diff-lcs (>= 1.2.0, < 2.0)
|
|
207
202
|
rspec-support (~> 3.13.0)
|
|
208
|
-
rspec-parameterized (2.0.
|
|
203
|
+
rspec-parameterized (2.0.1)
|
|
209
204
|
rspec-parameterized-core (>= 2, < 3)
|
|
210
205
|
rspec-parameterized-table_syntax (>= 2, < 3)
|
|
211
|
-
rspec-parameterized-core (2.0.
|
|
206
|
+
rspec-parameterized-core (2.0.1)
|
|
212
207
|
parser
|
|
213
208
|
prism
|
|
214
209
|
proc_to_ast (>= 0.2.0)
|
|
215
210
|
rspec (>= 2.13, < 4)
|
|
216
211
|
unparser
|
|
217
|
-
rspec-parameterized-table_syntax (2.
|
|
212
|
+
rspec-parameterized-table_syntax (2.1.0)
|
|
218
213
|
binding_of_caller
|
|
219
214
|
rspec-parameterized-core (>= 2, < 3)
|
|
220
|
-
rspec-support (3.13.
|
|
221
|
-
rubocop (1.
|
|
215
|
+
rspec-support (3.13.6)
|
|
216
|
+
rubocop (1.82.1)
|
|
222
217
|
json (~> 2.3)
|
|
223
218
|
language_server-protocol (~> 3.17.0.2)
|
|
224
219
|
lint_roller (~> 1.1.0)
|
|
@@ -226,13 +221,13 @@ GEM
|
|
|
226
221
|
parser (>= 3.3.0.2)
|
|
227
222
|
rainbow (>= 2.2.2, < 4.0)
|
|
228
223
|
regexp_parser (>= 2.9.3, < 3.0)
|
|
229
|
-
rubocop-ast (>= 1.
|
|
224
|
+
rubocop-ast (>= 1.48.0, < 2.0)
|
|
230
225
|
ruby-progressbar (~> 1.7)
|
|
231
226
|
unicode-display_width (>= 2.4.0, < 4.0)
|
|
232
|
-
rubocop-ast (1.
|
|
227
|
+
rubocop-ast (1.49.0)
|
|
233
228
|
parser (>= 3.3.7.2)
|
|
234
|
-
prism (~> 1.
|
|
235
|
-
rubocop-rails (2.
|
|
229
|
+
prism (~> 1.7)
|
|
230
|
+
rubocop-rails (2.34.3)
|
|
236
231
|
activesupport (>= 4.2.0)
|
|
237
232
|
lint_roller (~> 1.1)
|
|
238
233
|
rack (>= 1.1)
|
|
@@ -241,25 +236,27 @@ GEM
|
|
|
241
236
|
rubocop-rake (0.7.1)
|
|
242
237
|
lint_roller (~> 1.1)
|
|
243
238
|
rubocop (>= 1.72.1)
|
|
244
|
-
rubocop-rspec (3.
|
|
239
|
+
rubocop-rspec (3.9.0)
|
|
245
240
|
lint_roller (~> 1.1)
|
|
246
|
-
rubocop (~> 1.
|
|
241
|
+
rubocop (~> 1.81)
|
|
247
242
|
ruby-progressbar (1.13.0)
|
|
248
243
|
securerandom (0.4.1)
|
|
249
|
-
stringio (3.
|
|
250
|
-
thor (1.
|
|
251
|
-
timeout (0.
|
|
244
|
+
stringio (3.2.0)
|
|
245
|
+
thor (1.5.0)
|
|
246
|
+
timeout (0.6.0)
|
|
247
|
+
tsort (0.2.0)
|
|
252
248
|
tzinfo (2.0.6)
|
|
253
249
|
concurrent-ruby (~> 1.0)
|
|
254
|
-
unicode-display_width (3.
|
|
255
|
-
unicode-emoji (~> 4.
|
|
256
|
-
unicode-emoji (4.0
|
|
257
|
-
unparser (0.
|
|
250
|
+
unicode-display_width (3.2.0)
|
|
251
|
+
unicode-emoji (~> 4.1)
|
|
252
|
+
unicode-emoji (4.2.0)
|
|
253
|
+
unparser (0.8.1)
|
|
258
254
|
diff-lcs (~> 1.6)
|
|
259
255
|
parser (>= 3.3.0)
|
|
260
|
-
|
|
256
|
+
prism (>= 1.5.1)
|
|
257
|
+
uri (1.1.1)
|
|
261
258
|
useragent (0.16.11)
|
|
262
|
-
zeitwerk (2.7.
|
|
259
|
+
zeitwerk (2.7.4)
|
|
263
260
|
|
|
264
261
|
PLATFORMS
|
|
265
262
|
aarch64-linux
|
|
@@ -273,6 +270,7 @@ PLATFORMS
|
|
|
273
270
|
x86_64-linux-musl
|
|
274
271
|
|
|
275
272
|
DEPENDENCIES
|
|
273
|
+
benchmark
|
|
276
274
|
bundler
|
|
277
275
|
bundler-audit
|
|
278
276
|
faker
|
data/README.md
CHANGED
|
@@ -97,6 +97,51 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
|
|
|
97
97
|
|
|
98
98
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
99
99
|
|
|
100
|
+
### Performance Benchmarking
|
|
101
|
+
|
|
102
|
+
You can measure the performance of query parsing including file I/O operations using the included benchmark tool:
|
|
103
|
+
|
|
104
|
+
```shell
|
|
105
|
+
# Run the benchmark
|
|
106
|
+
bundle exec rake benchmark
|
|
107
|
+
|
|
108
|
+
# Or run directly
|
|
109
|
+
bundle exec bin/benchmark
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
The benchmark measures:
|
|
113
|
+
|
|
114
|
+
- **File I/O Performance**: Time to read SQL files from disk
|
|
115
|
+
- **SQL Parsing Performance**: Time to parse SQL queries using pg_query
|
|
116
|
+
- **Dependency Extraction Performance**: Time to extract `--!depends_on` directives from comments
|
|
117
|
+
- **Full Workflow Performance**: Combined time for file I/O, parsing, and dependency extraction
|
|
118
|
+
- **Memory Usage**: Memory consumption during the parsing process
|
|
119
|
+
|
|
120
|
+
The benchmark creates temporary SQL files of various sizes and complexities to provide realistic performance metrics. Results include throughput (files/objects per second) and average processing time per file.
|
|
121
|
+
|
|
122
|
+
Example output:
|
|
123
|
+
```
|
|
124
|
+
PG Objects Performance Benchmark
|
|
125
|
+
==================================================
|
|
126
|
+
|
|
127
|
+
File I/O Performance:
|
|
128
|
+
Files processed: 59
|
|
129
|
+
Time: 0.0005s
|
|
130
|
+
Throughput: 108574.84 files/s
|
|
131
|
+
|
|
132
|
+
Parsing Performance:
|
|
133
|
+
Files processed: 59
|
|
134
|
+
Successful parses: 59
|
|
135
|
+
Parse errors: 0
|
|
136
|
+
Time: 0.0092s
|
|
137
|
+
Throughput: 6411.8 files/s
|
|
138
|
+
|
|
139
|
+
Full Workflow Performance:
|
|
140
|
+
Objects processed: 59
|
|
141
|
+
Time: 0.0083s
|
|
142
|
+
Throughput: 7105.78 objects/s
|
|
143
|
+
```
|
|
144
|
+
|
|
100
145
|
## Contributing
|
|
101
146
|
|
|
102
147
|
Bug reports and pull requests are welcome on GitHub at https://github.com/marinazzio/pg_objects.
|
data/Rakefile
CHANGED
data/benchmark.txt
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
PG Objects Performance Benchmark
|
|
2
|
+
==================================================
|
|
3
|
+
|
|
4
|
+
Setting up test files...
|
|
5
|
+
|
|
6
|
+
Benchmarking File I/O Operations:
|
|
7
|
+
------------------------------
|
|
8
|
+
Read 59 files in 0.0005s
|
|
9
|
+
Throughput: 109963.02 files/second
|
|
10
|
+
Average time per file: 0.01ms
|
|
11
|
+
|
|
12
|
+
Benchmarking SQL Parsing:
|
|
13
|
+
------------------------------
|
|
14
|
+
Parsed 59 files in 0.0186s
|
|
15
|
+
Successful parses: 59
|
|
16
|
+
Parse errors: 0
|
|
17
|
+
Throughput: 3166.05 files/second
|
|
18
|
+
Average time per file: 0.32ms
|
|
19
|
+
|
|
20
|
+
Benchmarking Dependency Extraction:
|
|
21
|
+
------------------------------
|
|
22
|
+
Extracted dependencies from 59 files in 0.0009s
|
|
23
|
+
Total dependencies found: 15
|
|
24
|
+
Throughput: 68904.22 files/second
|
|
25
|
+
Average time per file: 0.01ms
|
|
26
|
+
|
|
27
|
+
Benchmarking Full Workflow (File I/O + Parsing + Dependencies):
|
|
28
|
+
------------------------------
|
|
29
|
+
Processed 59 objects in 0.0108s
|
|
30
|
+
Throughput: 5445.8 objects/second
|
|
31
|
+
Average time per object: 0.18ms
|
|
32
|
+
|
|
33
|
+
Memory Usage Analysis:
|
|
34
|
+
------------------------------
|
|
35
|
+
Memory usage:
|
|
36
|
+
Start: 36.59 MB
|
|
37
|
+
End: 36.71 MB
|
|
38
|
+
Used: 128.0 KB
|
|
39
|
+
Per object: 2.17 KB
|
|
40
|
+
|
|
41
|
+
Performance Summary:
|
|
42
|
+
==================================================
|
|
43
|
+
File I/O Performance:
|
|
44
|
+
Files processed: 59
|
|
45
|
+
Time: 0.0005s
|
|
46
|
+
Throughput: 109963.02 files/s
|
|
47
|
+
|
|
48
|
+
Parsing Performance:
|
|
49
|
+
Files processed: 59
|
|
50
|
+
Successful parses: 59
|
|
51
|
+
Parse errors: 0
|
|
52
|
+
Time: 0.0186s
|
|
53
|
+
Throughput: 3166.05 files/s
|
|
54
|
+
|
|
55
|
+
Dependency Extraction Performance:
|
|
56
|
+
Files processed: 59
|
|
57
|
+
Dependencies found: 15
|
|
58
|
+
Time: 0.0009s
|
|
59
|
+
Throughput: 68904.22 files/s
|
|
60
|
+
|
|
61
|
+
Full Workflow Performance:
|
|
62
|
+
Objects processed: 59
|
|
63
|
+
Time: 0.0108s
|
|
64
|
+
Throughput: 5445.8 objects/s
|
|
65
|
+
|
|
66
|
+
Memory Usage:
|
|
67
|
+
Objects processed: 59
|
|
68
|
+
Memory used: 128.0 KB
|
|
69
|
+
Memory per object: 2.17 KB
|
|
70
|
+
|
|
71
|
+
Benchmark completed successfully!
|
data/bin/benchmark
ADDED
|
@@ -0,0 +1,486 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
|
|
3
|
+
require 'bundler/setup'
|
|
4
|
+
require 'pg_objects'
|
|
5
|
+
require 'benchmark'
|
|
6
|
+
require 'fileutils'
|
|
7
|
+
require 'tmpdir'
|
|
8
|
+
require 'optparse'
|
|
9
|
+
|
|
10
|
+
# rubocop: disable Metrics/MethodLength, Style/Documentation, Metrics/AbcSize
|
|
11
|
+
class PgObjectsBenchmark
|
|
12
|
+
DEFAULT_OPTIONS = {
|
|
13
|
+
file_count: 59,
|
|
14
|
+
large_files: 10,
|
|
15
|
+
verbose: false,
|
|
16
|
+
quiet: false
|
|
17
|
+
}.freeze
|
|
18
|
+
|
|
19
|
+
SAMPLE_SQLS = {
|
|
20
|
+
simple_function: <<~SQL,
|
|
21
|
+
CREATE OR REPLACE FUNCTION add_numbers(a INTEGER, b INTEGER)
|
|
22
|
+
RETURNS INTEGER AS $$
|
|
23
|
+
BEGIN
|
|
24
|
+
RETURN a + b;
|
|
25
|
+
END;
|
|
26
|
+
$$ LANGUAGE plpgsql;
|
|
27
|
+
SQL
|
|
28
|
+
|
|
29
|
+
complex_function_with_deps: <<~SQL,
|
|
30
|
+
--!depends_on add_numbers
|
|
31
|
+
CREATE OR REPLACE FUNCTION calculate_total(x INTEGER, y INTEGER, z INTEGER)
|
|
32
|
+
RETURNS INTEGER AS $$
|
|
33
|
+
DECLARE
|
|
34
|
+
result INTEGER;
|
|
35
|
+
BEGIN
|
|
36
|
+
result := add_numbers(x, y);
|
|
37
|
+
result := add_numbers(result, z);
|
|
38
|
+
RETURN result;
|
|
39
|
+
END;
|
|
40
|
+
$$ LANGUAGE plpgsql;
|
|
41
|
+
SQL
|
|
42
|
+
|
|
43
|
+
trigger: <<~SQL,
|
|
44
|
+
--!depends_on audit_function
|
|
45
|
+
CREATE TRIGGER user_audit_trigger
|
|
46
|
+
AFTER INSERT OR UPDATE OR DELETE ON users
|
|
47
|
+
FOR EACH ROW
|
|
48
|
+
EXECUTE FUNCTION audit_function();
|
|
49
|
+
SQL
|
|
50
|
+
|
|
51
|
+
view: <<~SQL,
|
|
52
|
+
--!depends_on users_table
|
|
53
|
+
--!depends_on orders_table
|
|
54
|
+
CREATE OR REPLACE VIEW user_orders AS
|
|
55
|
+
SELECT
|
|
56
|
+
u.id,
|
|
57
|
+
u.name,
|
|
58
|
+
u.email,
|
|
59
|
+
COUNT(o.id) as order_count,
|
|
60
|
+
SUM(o.total) as total_spent
|
|
61
|
+
FROM users u
|
|
62
|
+
LEFT JOIN orders o ON u.id = o.user_id
|
|
63
|
+
GROUP BY u.id, u.name, u.email;
|
|
64
|
+
SQL
|
|
65
|
+
|
|
66
|
+
large_function: <<~SQL,
|
|
67
|
+
CREATE OR REPLACE FUNCTION complex_calculation(input_data JSONB)
|
|
68
|
+
RETURNS TABLE(
|
|
69
|
+
id INTEGER,
|
|
70
|
+
calculated_value NUMERIC,
|
|
71
|
+
status TEXT,
|
|
72
|
+
created_at TIMESTAMP
|
|
73
|
+
) AS $$
|
|
74
|
+
DECLARE
|
|
75
|
+
item JSONB;
|
|
76
|
+
temp_value NUMERIC;
|
|
77
|
+
temp_status TEXT;
|
|
78
|
+
BEGIN
|
|
79
|
+
FOR item IN SELECT jsonb_array_elements(input_data)
|
|
80
|
+
LOOP
|
|
81
|
+
temp_value := (item->>'value')::NUMERIC;
|
|
82
|
+
|
|
83
|
+
IF temp_value > 1000 THEN
|
|
84
|
+
temp_value := temp_value * 0.95;
|
|
85
|
+
temp_status := 'discounted';
|
|
86
|
+
ELSIF temp_value > 500 THEN
|
|
87
|
+
temp_value := temp_value * 0.98;
|
|
88
|
+
temp_status := 'reduced';
|
|
89
|
+
ELSE
|
|
90
|
+
temp_status := 'standard';
|
|
91
|
+
END IF;
|
|
92
|
+
|
|
93
|
+
RETURN QUERY SELECT
|
|
94
|
+
(item->>'id')::INTEGER,
|
|
95
|
+
temp_value,
|
|
96
|
+
temp_status,
|
|
97
|
+
NOW();
|
|
98
|
+
END LOOP;
|
|
99
|
+
END;
|
|
100
|
+
$$ LANGUAGE plpgsql;
|
|
101
|
+
SQL
|
|
102
|
+
|
|
103
|
+
table: <<~SQL,
|
|
104
|
+
CREATE TABLE users (
|
|
105
|
+
id SERIAL PRIMARY KEY,
|
|
106
|
+
name VARCHAR(255) NOT NULL,
|
|
107
|
+
email VARCHAR(255) UNIQUE NOT NULL,
|
|
108
|
+
created_at TIMESTAMP DEFAULT NOW(),
|
|
109
|
+
updated_at TIMESTAMP DEFAULT NOW()
|
|
110
|
+
);
|
|
111
|
+
SQL
|
|
112
|
+
|
|
113
|
+
materialized_view: <<~SQL,
|
|
114
|
+
--!depends_on user_orders
|
|
115
|
+
CREATE MATERIALIZED VIEW top_customers AS
|
|
116
|
+
SELECT *
|
|
117
|
+
FROM user_orders
|
|
118
|
+
WHERE total_spent > 1000
|
|
119
|
+
ORDER BY total_spent DESC
|
|
120
|
+
LIMIT 100;
|
|
121
|
+
SQL
|
|
122
|
+
|
|
123
|
+
type: <<~SQL
|
|
124
|
+
CREATE TYPE address_type AS (
|
|
125
|
+
street VARCHAR(255),
|
|
126
|
+
city VARCHAR(100),
|
|
127
|
+
state VARCHAR(50),
|
|
128
|
+
zip_code VARCHAR(10),
|
|
129
|
+
country VARCHAR(50)
|
|
130
|
+
);
|
|
131
|
+
SQL
|
|
132
|
+
}.freeze
|
|
133
|
+
|
|
134
|
+
def initialize(options = {})
|
|
135
|
+
@options = DEFAULT_OPTIONS.merge(options)
|
|
136
|
+
@temp_dir = File.join(Dir.tmpdir, 'pg_objects_benchmark')
|
|
137
|
+
@results = {}
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
def run
|
|
141
|
+
unless @options[:quiet]
|
|
142
|
+
puts 'PG Objects Performance Benchmark'
|
|
143
|
+
puts '=' * 50
|
|
144
|
+
puts
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
setup_test_files
|
|
148
|
+
|
|
149
|
+
run_benchmarks
|
|
150
|
+
|
|
151
|
+
cleanup
|
|
152
|
+
print_summary unless @options[:quiet]
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
private
|
|
156
|
+
|
|
157
|
+
def run_benchmarks
|
|
158
|
+
benchmark_file_io
|
|
159
|
+
benchmark_parsing
|
|
160
|
+
benchmark_dependency_extraction
|
|
161
|
+
benchmark_full_workflow
|
|
162
|
+
benchmark_memory_usage
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
def setup_test_files
|
|
166
|
+
puts 'Setting up test files...' unless @options[:quiet]
|
|
167
|
+
|
|
168
|
+
FileUtils.rm_rf(@temp_dir)
|
|
169
|
+
FileUtils.mkdir_p(File.join(@temp_dir, 'before'))
|
|
170
|
+
FileUtils.mkdir_p(File.join(@temp_dir, 'after'))
|
|
171
|
+
|
|
172
|
+
# Create multiple files of different sizes
|
|
173
|
+
SAMPLE_SQLS.each_with_index do |(name, content), index|
|
|
174
|
+
file_path = File.join(@temp_dir, index.even? ? 'before' : 'after', "#{name}.sql")
|
|
175
|
+
File.write(file_path, content)
|
|
176
|
+
end
|
|
177
|
+
|
|
178
|
+
# Create additional files for stress testing
|
|
179
|
+
create_large_files
|
|
180
|
+
|
|
181
|
+
puts "Created #{count_files} test SQL files in #{@temp_dir}" if @options[:verbose]
|
|
182
|
+
puts unless @options[:quiet]
|
|
183
|
+
end
|
|
184
|
+
|
|
185
|
+
def create_large_files
|
|
186
|
+
# Create a large function with many dependencies
|
|
187
|
+
large_content = "-- Large file with many dependencies\n"
|
|
188
|
+
@options[:large_files].times do |i|
|
|
189
|
+
large_content += "--!depends_on function_#{i}\n"
|
|
190
|
+
end
|
|
191
|
+
large_content += SAMPLE_SQLS[:large_function]
|
|
192
|
+
|
|
193
|
+
File.write(File.join(@temp_dir, 'before', 'large_with_deps.sql'), large_content)
|
|
194
|
+
|
|
195
|
+
# Create multiple small files (based on file_count option)
|
|
196
|
+
additional_files = [@options[:file_count] - SAMPLE_SQLS.size - 1, 0].max
|
|
197
|
+
additional_files.times do |i|
|
|
198
|
+
content = <<~SQL
|
|
199
|
+
CREATE OR REPLACE FUNCTION generated_function_#{i}(param INTEGER)
|
|
200
|
+
RETURNS INTEGER AS $$
|
|
201
|
+
BEGIN
|
|
202
|
+
RETURN param * #{i + 1};
|
|
203
|
+
END;
|
|
204
|
+
$$ LANGUAGE plpgsql;
|
|
205
|
+
SQL
|
|
206
|
+
|
|
207
|
+
File.write(File.join(@temp_dir, 'after', "generated_#{i}.sql"), content)
|
|
208
|
+
end
|
|
209
|
+
end
|
|
210
|
+
|
|
211
|
+
def count_files
|
|
212
|
+
Dir[File.join(@temp_dir, '**', '*.sql')].size
|
|
213
|
+
end
|
|
214
|
+
|
|
215
|
+
def benchmark_file_io
|
|
216
|
+
puts 'Benchmarking File I/O Operations:'
|
|
217
|
+
puts '-' * 30
|
|
218
|
+
|
|
219
|
+
files = Dir[File.join(@temp_dir, '**', '*.sql')]
|
|
220
|
+
|
|
221
|
+
result = Benchmark.measure do
|
|
222
|
+
files.each { |file| File.read(file) }
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
@results[:file_io] = {
|
|
226
|
+
time: result.real,
|
|
227
|
+
files_count: files.size,
|
|
228
|
+
throughput: files.size / result.real
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
puts "Read #{files.size} files in #{result.real.round(4)}s"
|
|
232
|
+
puts "Throughput: #{(files.size / result.real).round(2)} files/second"
|
|
233
|
+
puts "Average time per file: #{(result.real / files.size * 1000).round(2)}ms"
|
|
234
|
+
puts
|
|
235
|
+
end
|
|
236
|
+
|
|
237
|
+
def benchmark_parsing
|
|
238
|
+
puts 'Benchmarking SQL Parsing:'
|
|
239
|
+
puts '-' * 30
|
|
240
|
+
|
|
241
|
+
files = Dir[File.join(@temp_dir, '**', '*.sql')]
|
|
242
|
+
parser = PgObjects::Parser.new
|
|
243
|
+
successful_parses = 0
|
|
244
|
+
parse_errors = 0
|
|
245
|
+
|
|
246
|
+
result = Benchmark.measure do
|
|
247
|
+
files.each do |file|
|
|
248
|
+
content = File.read(file)
|
|
249
|
+
begin
|
|
250
|
+
parser.load(content).fetch_object_name
|
|
251
|
+
successful_parses += 1
|
|
252
|
+
rescue StandardError
|
|
253
|
+
parse_errors += 1
|
|
254
|
+
end
|
|
255
|
+
end
|
|
256
|
+
end
|
|
257
|
+
|
|
258
|
+
@results[:parsing] = {
|
|
259
|
+
time: result.real,
|
|
260
|
+
files_count: files.size,
|
|
261
|
+
successful_parses: successful_parses,
|
|
262
|
+
parse_errors: parse_errors,
|
|
263
|
+
throughput: files.size / result.real
|
|
264
|
+
}
|
|
265
|
+
|
|
266
|
+
puts "Parsed #{files.size} files in #{result.real.round(4)}s"
|
|
267
|
+
puts "Successful parses: #{successful_parses}"
|
|
268
|
+
puts "Parse errors: #{parse_errors}"
|
|
269
|
+
puts "Throughput: #{(files.size / result.real).round(2)} files/second"
|
|
270
|
+
puts "Average time per file: #{(result.real / files.size * 1000).round(2)}ms"
|
|
271
|
+
puts
|
|
272
|
+
end
|
|
273
|
+
|
|
274
|
+
def benchmark_dependency_extraction
|
|
275
|
+
puts 'Benchmarking Dependency Extraction:'
|
|
276
|
+
puts '-' * 30
|
|
277
|
+
|
|
278
|
+
files = Dir[File.join(@temp_dir, '**', '*.sql')]
|
|
279
|
+
parser = PgObjects::Parser.new
|
|
280
|
+
total_dependencies = 0
|
|
281
|
+
|
|
282
|
+
result = Benchmark.measure do
|
|
283
|
+
files.each do |file|
|
|
284
|
+
content = File.read(file)
|
|
285
|
+
dependencies = parser.load(content).fetch_directives[:depends_on]
|
|
286
|
+
total_dependencies += dependencies.size
|
|
287
|
+
end
|
|
288
|
+
end
|
|
289
|
+
|
|
290
|
+
@results[:dependency_extraction] = {
|
|
291
|
+
time: result.real,
|
|
292
|
+
files_count: files.size,
|
|
293
|
+
total_dependencies: total_dependencies,
|
|
294
|
+
throughput: files.size / result.real
|
|
295
|
+
}
|
|
296
|
+
|
|
297
|
+
puts "Extracted dependencies from #{files.size} files in #{result.real.round(4)}s"
|
|
298
|
+
puts "Total dependencies found: #{total_dependencies}"
|
|
299
|
+
puts "Throughput: #{(files.size / result.real).round(2)} files/second"
|
|
300
|
+
puts "Average time per file: #{(result.real / files.size * 1000).round(2)}ms"
|
|
301
|
+
puts
|
|
302
|
+
end
|
|
303
|
+
|
|
304
|
+
def benchmark_full_workflow
|
|
305
|
+
puts 'Benchmarking Full Workflow (File I/O + Parsing + Dependencies):'
|
|
306
|
+
puts '-' * 30
|
|
307
|
+
|
|
308
|
+
# Test the parsing workflow without database operations
|
|
309
|
+
files = Dir[File.join(@temp_dir, '**', '*.sql')]
|
|
310
|
+
db_objects = []
|
|
311
|
+
|
|
312
|
+
result = Benchmark.measure do
|
|
313
|
+
files.each do |file_path|
|
|
314
|
+
# Simulate the DbObject creation process
|
|
315
|
+
content = File.read(file_path)
|
|
316
|
+
|
|
317
|
+
# Parse the content
|
|
318
|
+
parser = PgObjects::Parser.new
|
|
319
|
+
object_name = parser.load(content).fetch_object_name
|
|
320
|
+
dependencies = parser.fetch_directives[:depends_on]
|
|
321
|
+
|
|
322
|
+
# Create a mock DbObject-like structure
|
|
323
|
+
db_objects << {
|
|
324
|
+
name: File.basename(file_path, '.sql'),
|
|
325
|
+
full_name: file_path,
|
|
326
|
+
object_name: object_name,
|
|
327
|
+
dependencies: dependencies,
|
|
328
|
+
content: content
|
|
329
|
+
}
|
|
330
|
+
rescue StandardError
|
|
331
|
+
# Count parse errors but continue
|
|
332
|
+
end
|
|
333
|
+
end
|
|
334
|
+
|
|
335
|
+
@results[:full_workflow] = {
|
|
336
|
+
time: result.real,
|
|
337
|
+
objects_count: db_objects.size,
|
|
338
|
+
throughput: db_objects.size / result.real
|
|
339
|
+
}
|
|
340
|
+
|
|
341
|
+
puts "Processed #{db_objects.size} objects in #{result.real.round(4)}s"
|
|
342
|
+
puts "Throughput: #{(db_objects.size / result.real).round(2)} objects/second"
|
|
343
|
+
puts "Average time per object: #{(result.real / db_objects.size * 1000).round(2)}ms"
|
|
344
|
+
puts
|
|
345
|
+
end
|
|
346
|
+
|
|
347
|
+
def benchmark_memory_usage
|
|
348
|
+
puts 'Memory Usage Analysis:'
|
|
349
|
+
puts '-' * 30
|
|
350
|
+
|
|
351
|
+
start_memory = memory_usage
|
|
352
|
+
|
|
353
|
+
# Load all files and parse them
|
|
354
|
+
files = Dir[File.join(@temp_dir, '**', '*.sql')]
|
|
355
|
+
parser = PgObjects::Parser.new
|
|
356
|
+
parsed_objects = []
|
|
357
|
+
|
|
358
|
+
files.each do |file|
|
|
359
|
+
content = File.read(file)
|
|
360
|
+
begin
|
|
361
|
+
object_name = parser.load(content).fetch_object_name
|
|
362
|
+
dependencies = parser.fetch_directives[:depends_on]
|
|
363
|
+
parsed_objects << { file: file, name: object_name, deps: dependencies }
|
|
364
|
+
rescue StandardError
|
|
365
|
+
# Ignore parse errors for memory analysis
|
|
366
|
+
end
|
|
367
|
+
end
|
|
368
|
+
|
|
369
|
+
end_memory = memory_usage
|
|
370
|
+
memory_diff = end_memory - start_memory
|
|
371
|
+
|
|
372
|
+
@results[:memory] = {
|
|
373
|
+
start_memory: start_memory,
|
|
374
|
+
end_memory: end_memory,
|
|
375
|
+
memory_used: memory_diff,
|
|
376
|
+
objects_count: parsed_objects.size,
|
|
377
|
+
memory_per_object: memory_diff.to_f / parsed_objects.size
|
|
378
|
+
}
|
|
379
|
+
|
|
380
|
+
puts 'Memory usage:'
|
|
381
|
+
puts " Start: #{format_memory(start_memory)}"
|
|
382
|
+
puts " End: #{format_memory(end_memory)}"
|
|
383
|
+
puts " Used: #{format_memory(memory_diff)}"
|
|
384
|
+
puts " Per object: #{format_memory(memory_diff.to_f / parsed_objects.size)}"
|
|
385
|
+
puts
|
|
386
|
+
end
|
|
387
|
+
|
|
388
|
+
def memory_usage
|
|
389
|
+
`ps -o rss= -p #{Process.pid}`.to_i * 1024 # Convert KB to bytes
|
|
390
|
+
rescue StandardError
|
|
391
|
+
0
|
|
392
|
+
end
|
|
393
|
+
|
|
394
|
+
def format_memory(bytes)
|
|
395
|
+
if bytes < 1024
|
|
396
|
+
"#{bytes} B"
|
|
397
|
+
elsif bytes < 1024 * 1024
|
|
398
|
+
"#{(bytes / 1024.0).round(2)} KB"
|
|
399
|
+
else
|
|
400
|
+
"#{(bytes / (1024.0 * 1024)).round(2)} MB"
|
|
401
|
+
end
|
|
402
|
+
end
|
|
403
|
+
|
|
404
|
+
def cleanup
|
|
405
|
+
FileUtils.rm_rf(@temp_dir)
|
|
406
|
+
end
|
|
407
|
+
|
|
408
|
+
def print_summary
|
|
409
|
+
puts 'Performance Summary:'
|
|
410
|
+
puts '=' * 50
|
|
411
|
+
|
|
412
|
+
if @results[:file_io]
|
|
413
|
+
puts 'File I/O Performance:'
|
|
414
|
+
puts " Files processed: #{@results[:file_io][:files_count]}"
|
|
415
|
+
puts " Time: #{@results[:file_io][:time].round(4)}s"
|
|
416
|
+
puts " Throughput: #{@results[:file_io][:throughput].round(2)} files/s"
|
|
417
|
+
puts
|
|
418
|
+
end
|
|
419
|
+
|
|
420
|
+
if @results[:parsing]
|
|
421
|
+
puts 'Parsing Performance:'
|
|
422
|
+
puts " Files processed: #{@results[:parsing][:files_count]}"
|
|
423
|
+
puts " Successful parses: #{@results[:parsing][:successful_parses]}"
|
|
424
|
+
puts " Parse errors: #{@results[:parsing][:parse_errors]}"
|
|
425
|
+
puts " Time: #{@results[:parsing][:time].round(4)}s"
|
|
426
|
+
puts " Throughput: #{@results[:parsing][:throughput].round(2)} files/s"
|
|
427
|
+
puts
|
|
428
|
+
end
|
|
429
|
+
|
|
430
|
+
if @results[:dependency_extraction]
|
|
431
|
+
puts 'Dependency Extraction Performance:'
|
|
432
|
+
puts " Files processed: #{@results[:dependency_extraction][:files_count]}"
|
|
433
|
+
puts " Dependencies found: #{@results[:dependency_extraction][:total_dependencies]}"
|
|
434
|
+
puts " Time: #{@results[:dependency_extraction][:time].round(4)}s"
|
|
435
|
+
puts " Throughput: #{@results[:dependency_extraction][:throughput].round(2)} files/s"
|
|
436
|
+
puts
|
|
437
|
+
end
|
|
438
|
+
|
|
439
|
+
if @results[:full_workflow]
|
|
440
|
+
puts 'Full Workflow Performance:'
|
|
441
|
+
puts " Objects processed: #{@results[:full_workflow][:objects_count]}"
|
|
442
|
+
puts " Time: #{@results[:full_workflow][:time].round(4)}s"
|
|
443
|
+
puts " Throughput: #{@results[:full_workflow][:throughput].round(2)} objects/s"
|
|
444
|
+
puts
|
|
445
|
+
end
|
|
446
|
+
|
|
447
|
+
if @results[:memory]
|
|
448
|
+
puts 'Memory Usage:'
|
|
449
|
+
puts " Objects processed: #{@results[:memory][:objects_count]}"
|
|
450
|
+
puts " Memory used: #{format_memory(@results[:memory][:memory_used])}"
|
|
451
|
+
puts " Memory per object: #{format_memory(@results[:memory][:memory_per_object])}"
|
|
452
|
+
puts
|
|
453
|
+
end
|
|
454
|
+
|
|
455
|
+
puts 'Benchmark completed successfully!'
|
|
456
|
+
end
|
|
457
|
+
end
|
|
458
|
+
# rubocop: enable Metrics/MethodLength, Style/Documentation, Metrics/AbcSize
|
|
459
|
+
|
|
460
|
+
options = {}
|
|
461
|
+
OptionParser.new do |opts|
|
|
462
|
+
opts.banner = "Usage: #{$PROGRAM_NAME} [options]"
|
|
463
|
+
|
|
464
|
+
opts.on('-f', '--files COUNT', Integer, 'Number of test files to generate (default: 59)') do |count|
|
|
465
|
+
options[:file_count] = count
|
|
466
|
+
end
|
|
467
|
+
|
|
468
|
+
opts.on('-l', '--large-files COUNT', Integer, 'Number of large dependency files (default: 10)') do |count|
|
|
469
|
+
options[:large_files] = count
|
|
470
|
+
end
|
|
471
|
+
|
|
472
|
+
opts.on('-v', '--verbose', 'Verbose output') do
|
|
473
|
+
options[:verbose] = true
|
|
474
|
+
end
|
|
475
|
+
|
|
476
|
+
opts.on('-q', '--quiet', 'Quiet mode - minimal output') do
|
|
477
|
+
options[:quiet] = true
|
|
478
|
+
end
|
|
479
|
+
|
|
480
|
+
opts.on('-h', '--help', 'Show this help message') do
|
|
481
|
+
puts opts
|
|
482
|
+
exit
|
|
483
|
+
end
|
|
484
|
+
end.parse!
|
|
485
|
+
|
|
486
|
+
PgObjectsBenchmark.new(options).run
|
data/lib/pg_objects/version.rb
CHANGED
data/pg_objects.gemspec
CHANGED
|
@@ -48,7 +48,7 @@ Gem::Specification.new do |spec|
|
|
|
48
48
|
spec.add_dependency 'dry-configurable', '~> 1'
|
|
49
49
|
spec.add_dependency 'dry-container', '0.11.0'
|
|
50
50
|
spec.add_dependency 'dry-monads', '~> 1.6'
|
|
51
|
-
spec.add_dependency 'memery', '>= 1.5', '< 1.
|
|
51
|
+
spec.add_dependency 'memery', '>= 1.5', '< 1.9'
|
|
52
52
|
spec.add_dependency 'pg_query', '>= 5', '< 7'
|
|
53
53
|
spec.add_dependency 'railties', '>= 4', '< 9'
|
|
54
54
|
spec.add_dependency 'rake-hooks', '~> 1'
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: pg_objects
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.4.
|
|
4
|
+
version: 1.4.6
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Denis Kiselyov
|
|
@@ -94,7 +94,7 @@ dependencies:
|
|
|
94
94
|
version: '1.5'
|
|
95
95
|
- - "<"
|
|
96
96
|
- !ruby/object:Gem::Version
|
|
97
|
-
version: '1.
|
|
97
|
+
version: '1.9'
|
|
98
98
|
type: :runtime
|
|
99
99
|
prerelease: false
|
|
100
100
|
version_requirements: !ruby/object:Gem::Requirement
|
|
@@ -104,7 +104,7 @@ dependencies:
|
|
|
104
104
|
version: '1.5'
|
|
105
105
|
- - "<"
|
|
106
106
|
- !ruby/object:Gem::Version
|
|
107
|
-
version: '1.
|
|
107
|
+
version: '1.9'
|
|
108
108
|
- !ruby/object:Gem::Dependency
|
|
109
109
|
name: pg_query
|
|
110
110
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -165,6 +165,7 @@ executables: []
|
|
|
165
165
|
extensions: []
|
|
166
166
|
extra_rdoc_files: []
|
|
167
167
|
files:
|
|
168
|
+
- ".github/copilot-instructions.md"
|
|
168
169
|
- ".github/dependabot.yml"
|
|
169
170
|
- ".github/workflows/bundle_audit.yml"
|
|
170
171
|
- ".github/workflows/ci.yml"
|
|
@@ -178,6 +179,8 @@ files:
|
|
|
178
179
|
- LICENSE
|
|
179
180
|
- README.md
|
|
180
181
|
- Rakefile
|
|
182
|
+
- benchmark.txt
|
|
183
|
+
- bin/benchmark
|
|
181
184
|
- bin/console
|
|
182
185
|
- bin/setup
|
|
183
186
|
- lib/generators/pg_objects/install/USAGE
|
|
@@ -240,7 +243,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
240
243
|
- !ruby/object:Gem::Version
|
|
241
244
|
version: '0'
|
|
242
245
|
requirements: []
|
|
243
|
-
rubygems_version:
|
|
246
|
+
rubygems_version: 4.0.3
|
|
244
247
|
specification_version: 4
|
|
245
248
|
summary: Simple manager for PostgreSQL objects like triggers and functions
|
|
246
249
|
test_files: []
|