queue_classic 3.2.0.RC1 → 4.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (43) hide show
  1. checksums.yaml +5 -5
  2. data/.github/workflows/codeql-analysis.yml +72 -0
  3. data/.github/workflows/main.yaml +71 -0
  4. data/.gitignore +2 -0
  5. data/{changelog → CHANGELOG.md} +87 -34
  6. data/CODE_OF_CONDUCT.md +46 -0
  7. data/Gemfile +9 -5
  8. data/README.md +103 -126
  9. data/Rakefile +2 -0
  10. data/lib/generators/queue_classic/install_generator.rb +6 -0
  11. data/lib/generators/queue_classic/templates/add_queue_classic.rb +3 -1
  12. data/lib/generators/queue_classic/templates/update_queue_classic_3_0_0.rb +3 -1
  13. data/lib/generators/queue_classic/templates/update_queue_classic_3_0_2.rb +3 -1
  14. data/lib/generators/queue_classic/templates/update_queue_classic_3_1_0.rb +3 -1
  15. data/lib/generators/queue_classic/templates/update_queue_classic_4_0_0.rb +11 -0
  16. data/lib/queue_classic/config.rb +2 -1
  17. data/lib/queue_classic/conn_adapter.rb +29 -15
  18. data/lib/queue_classic/queue.rb +66 -12
  19. data/lib/queue_classic/railtie.rb +2 -0
  20. data/lib/queue_classic/setup.rb +24 -7
  21. data/lib/queue_classic/tasks.rb +4 -5
  22. data/lib/queue_classic/version.rb +3 -1
  23. data/lib/queue_classic/worker.rb +15 -6
  24. data/lib/queue_classic.rb +4 -11
  25. data/queue_classic.gemspec +3 -2
  26. data/sql/create_table.sql +7 -16
  27. data/sql/ddl.sql +6 -82
  28. data/sql/downgrade_from_4_0_0.sql +88 -0
  29. data/sql/update_to_3_0_0.sql +5 -5
  30. data/sql/update_to_3_1_0.sql +6 -6
  31. data/sql/update_to_4_0_0.sql +6 -0
  32. data/test/benchmark_test.rb +2 -0
  33. data/test/config_test.rb +2 -0
  34. data/test/helper.rb +34 -0
  35. data/test/lib/queue_classic_rails_connection_test.rb +9 -6
  36. data/test/lib/queue_classic_test.rb +2 -0
  37. data/test/lib/queue_classic_test_with_activerecord_typecast.rb +21 -0
  38. data/test/queue_test.rb +62 -2
  39. data/test/rails-tests/.gitignore +2 -0
  40. data/test/rails-tests/rails523.sh +23 -0
  41. data/test/worker_test.rb +138 -17
  42. metadata +43 -13
  43. data/.travis.yml +0 -15
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 173f9534b0c2b7318cccdf7de96b762ee63306d3
4
- data.tar.gz: c808e6bc17371a57e997c13800ba803666be969e
2
+ SHA256:
3
+ metadata.gz: 85cbe23c9e438cd6a641325666d15b2b9195e804807301551c2daf557edf5f1a
4
+ data.tar.gz: 5a83731ab2ee80532dc4e4e5af535e58301003a8064f92851793120520b88cff
5
5
  SHA512:
6
- metadata.gz: 7065d297515c3cd9bc40ac5f93a36f164d124f812dc2c53a649b163aceecf9995ebc07bee273660a4540defd08b95afc5b0b69f7b6af1491fd25f8c61cdbb927
7
- data.tar.gz: 9f4abe2f2f6c58a5bc4f049b0851d63e730c2922dcf8141a14222e01e2900c7993db939033925ebb684c88b6452e689269c3bbccb2e7386a60a4eacf24308aed
6
+ metadata.gz: 9862ac7ed5c2d45d30524636cba00ac60b0939c240fc631a9e60561353ac115d49ce147d783cc454ddcb025676eaac7958368d286877e57d38bf1889924b3574
7
+ data.tar.gz: 83010a3b0450dd5128c72fdc41ac834baa7af97e3166a2a3e7bc8592213a7682f434b23c34c89635f2246112cbe240faa97d7bcb6bda4579703ed0d2e613cc71
@@ -0,0 +1,72 @@
1
+ # For most projects, this workflow file will not need changing; you simply need
2
+ # to commit it to your repository.
3
+ #
4
+ # You may wish to alter this file to override the set of languages analyzed,
5
+ # or to provide custom queries or build logic.
6
+ #
7
+ # ******** NOTE ********
8
+ # We have attempted to detect the languages in your repository. Please check
9
+ # the `language` matrix defined below to confirm you have the correct set of
10
+ # supported CodeQL languages.
11
+ #
12
+ name: "CodeQL"
13
+
14
+ on:
15
+ push:
16
+ branches: [ master ]
17
+ pull_request:
18
+ # The branches below must be a subset of the branches above
19
+ branches: [ master ]
20
+ schedule:
21
+ - cron: '30 21 * * 0'
22
+
23
+ jobs:
24
+ analyze:
25
+ name: Analyze
26
+ runs-on: ubuntu-latest
27
+ permissions:
28
+ actions: read
29
+ contents: read
30
+ security-events: write
31
+
32
+ strategy:
33
+ fail-fast: false
34
+ matrix:
35
+ language: [ 'ruby' ]
36
+ # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
37
+ # Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
38
+
39
+ steps:
40
+ - name: Checkout repository
41
+ uses: actions/checkout@v3
42
+
43
+ # Initializes the CodeQL tools for scanning.
44
+ - name: Initialize CodeQL
45
+ uses: github/codeql-action/init@v2
46
+ with:
47
+ languages: ${{ matrix.language }}
48
+ # If you wish to specify custom queries, you can do so here or in a config file.
49
+ # By default, queries listed here will override any specified in a config file.
50
+ # Prefix the list here with "+" to use these queries and those in the config file.
51
+
52
+ # Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
53
+ # queries: security-extended,security-and-quality
54
+
55
+
56
+ # Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
57
+ # If this step fails, then you should remove it and run the build manually (see below)
58
+ - name: Autobuild
59
+ uses: github/codeql-action/autobuild@v2
60
+
61
+ # ℹ️ Command-line programs to run using the OS shell.
62
+ # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
63
+
64
+ # If the Autobuild fails above, remove it and uncomment the following three lines.
65
+ # modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
66
+
67
+ # - run: |
68
+ # echo "Run, Build Application using script"
69
+ # ./location_of_script_within_repo/buildscript.sh
70
+
71
+ - name: Perform CodeQL Analysis
72
+ uses: github/codeql-action/analyze@v2
@@ -0,0 +1,71 @@
1
+ name: Test
2
+ on: push
3
+ jobs:
4
+ # test_rails:
5
+ # runs-on: ubuntu-latest
6
+ # services:
7
+ # postgres:
8
+ # image: postgres
9
+ # env:
10
+ # POSTGRES_PASSWORD: postgres
11
+ # options: >-
12
+ # --health-cmd pg_isready
13
+ # --health-interval 10s
14
+ # --health-timeout 5s
15
+ # --health-retries 5
16
+ # ports:
17
+ # - 5432:5432
18
+ # steps:
19
+ # - uses: actions/checkout@v3
20
+ # - name: Install postgresql-client
21
+ # run: |
22
+ # sudo apt-get update
23
+ # sudo apt-get install -y libpq-dev
24
+ # - uses: ruby/setup-ruby@v1
25
+ # with:
26
+ # ruby-version: "2.6"
27
+ # bundler-cache: true # runs 'bundle install' and caches installed gems automatically
28
+ # - name: Test clean install in Rails
29
+ # env:
30
+ # DATABASE_URL: postgres://postgres:postgres@localhost/qctest523
31
+ # run: |
32
+ # cd test/rails-tests/
33
+ # sh rails523.sh
34
+ test:
35
+ runs-on: ubuntu-latest
36
+ strategy:
37
+ fail-fast: false
38
+ matrix:
39
+ ruby_version: ['2.6', '2.7', '3.0', '3.1']
40
+ services:
41
+ postgres:
42
+ image: postgres
43
+ env:
44
+ POSTGRES_PASSWORD: postgres
45
+ options: >-
46
+ --health-cmd pg_isready
47
+ --health-interval 10s
48
+ --health-timeout 5s
49
+ --health-retries 5
50
+ ports:
51
+ - 5432:5432
52
+ steps:
53
+ - uses: actions/checkout@v3
54
+ - name: Install postgresql-client
55
+ run: |
56
+ sudo apt-get update
57
+ sudo apt-get install -y libpq-dev
58
+ - uses: ruby/setup-ruby@v1
59
+ with:
60
+ ruby-version: ${{ matrix.ruby_version }}
61
+ bundler-cache: true # runs 'bundle install' and caches installed gems automatically
62
+ - name: Run tests
63
+ run: bundle exec rake
64
+ env:
65
+ RAILS_ENV: test
66
+ RACK_ENV: test
67
+ QC_DATABASE_URL: postgres://postgres:postgres@localhost/postgres
68
+ DATABASE_URL: postgres://postgres:postgres@localhost/postgres
69
+ QC_BENCHMARK: true
70
+ QC_BENCHMARK_MAX_TIME_DEQUEUE: 60
71
+ QC_BENCHMARK_MAX_TIME_ENQUEUE: 10
data/.gitignore CHANGED
@@ -7,3 +7,5 @@ etc/
7
7
  .env
8
8
  Gemfile.lock
9
9
  tips-and-tricks.md
10
+ test/reports/
11
+ vendor/
@@ -1,31 +1,60 @@
1
- Unreleased
2
- - Fixed a bug in the offset calculation of `.enqueue_at`.
1
+ # Changelog
2
+
3
+ All notable changes to this project will be documented in this file.
4
+
5
+ ## [4.0.0] - 2022-05-17
6
+
7
+ Updates:
8
+ - [Allow overriding of job successes](https://github.com/QueueClassic/queue_classic/pull/338)
9
+ - [Move to @github actions - @circleci ended up being slower. Test for Ruby 2.6, 2.7, 3.0 and 3.1. Require a more modern, but still old PG gem.](https://github.com/QueueClassic/queue_classic/pull/335)
10
+ - [Setup some code analysis](https://github.com/QueueClassic/queue_classic/pull/337)
11
+
12
+ ## [4.0.0-alpha1] - 2019-07-18
13
+
14
+ Updates:
15
+ - [Change to only support >= Postgres 9.6. We will be bringing in newer changes and testing on only 9.6+ going forward.](https://github.com/QueueClassic/queue_classic/pull/307)
16
+ - [Change to only support currently supported Ruby versions: 2.4, 2.5 and 2.6.](https://github.com/QueueClassic/queue_classic/pull/305)
17
+ - [Use skip-locked](https://github.com/QueueClassic/queue_classic/pull/303)
18
+ - [Add abilty to count ready and scheduled jobs](https://github.com/QueueClassic/queue_classic/pull/255)
19
+
20
+ Bug fixes:
21
+ - [Switched project to use CircleCI, as it's way more consistent speed wise](https://github.com/QueueClassic/queue_classic/pull/304)
22
+ - [Automatically retry after a connection reset #294](https://github.com/QueueClassic/queue_classic/pull/294)
23
+ - [Add tests for installing fresh on rails 5.2.3 + running migrations](https://github.com/QueueClassic/queue_classic/pull/308)
24
+ - [Don't recuse Time.now errors](https://github.com/QueueClassic/queue_classic/pull/310)
3
25
  - Use the jsonb type for the args column from now on. If not available, fall back to json or text.
4
26
  - `enqueue`, `enqueue_at`, `enqueue_in` return job hash with id.
5
- - Fixed unlock query for versions below Postgres 9.2
27
+ - Fixed a bug in the offset calculation of `.enqueue_at`.
28
+
29
+ ## [3.0.0rc] - 2014-01-07
6
30
 
7
- Version 3.0.0rc
8
31
  - Improved signal handling
9
32
 
10
- Version 3.0.0beta
33
+ ## [3.0.0beta] - 2014-01-06
34
+
11
35
  - Workers can process many queues.
12
36
 
13
- Version 2.2.3
37
+ ## [2.2.3] - 2013-10-24
38
+
14
39
  - Update pg dependency to 0.17.0
15
40
 
16
- Version 2.3.0beta [YANKED]
41
+ ## [2.3.0beta] - 2013-09-05 YANKED
42
+
17
43
  - Concurrent job processing.
18
44
 
19
- Version 2.2.2
45
+ ## [2.2.2] - 2013-08-04
46
+
20
47
  - Update pg dependency to 0.16.0
21
48
 
22
- Version 2.2.1
49
+ ## [2.2.1] - 2013-07-12
50
+
23
51
  - Force listen/notify on worker
24
52
  - Notifications happen inside PostgreSQL trigger
25
53
  - Add rake task for generating rails migrations
26
54
  - Fix bug related to listening worker
27
55
 
28
- Version 2.2.0
56
+ ## [2.2.0] - 2013-07-02
57
+
29
58
  - Use json from the stdlib in place of MultiJson.
30
59
  - Use postgresql's json type for the args column if json type is available
31
60
  - QC::Worker#handle_failure logs the job and the error
@@ -33,53 +62,65 @@ Version 2.2.0
33
62
  in testing to configure a mock queue)
34
63
  - QC.log now reports time elapsed in milliseconds.
35
64
 
36
- Version 2.1.4
65
+ ## [2.1.4]
66
+
37
67
  - Update pg dependency to 0.15.1
38
68
  - Document logging behaviour
39
69
 
40
- Version 2.1.3
70
+ ## [2.1.3]
71
+
41
72
  - Use MultiJson (Ezekiel Templin: #106)
42
73
 
43
- Version 2.1.2
74
+ ## [2.1.2]
75
+
44
76
  - Use 64bit ints as default data types in PostgreSQL
45
77
  - Add process method in worker
46
78
  - Allow percent-encoded socket paths in DATABASE_URL
47
79
 
48
- Version 2.1.1
80
+ ## [2.1.1]
81
+
49
82
  - Update pg gem version
50
83
 
51
- Version 2.1.0
84
+ ## [2.1.0]
85
+
52
86
  - Wrap connection execution in mutex making it thread safe
53
87
  - Cleanup logging
54
88
  - Refactor worker class making it more extensible
55
89
  - Added rdoc style docs for worker class
56
90
 
57
- Version 2.0.5
91
+ ## [2.0.5]
92
+
58
93
  - Allow term signal to halt the lock_job function
59
94
 
60
- Version 2.0.4
95
+ ## [2.0.4]
96
+
61
97
  - Provider a connection setter.
62
98
 
63
- Version 2.0.3
99
+ ## [2.0.3]
100
+
64
101
  - Fix typo :(
65
102
 
66
- Version 2.0.2
103
+ ## [2.0.2]
104
+
67
105
  - Remove scrolls dependency
68
106
  - Fix issue with notify not working on non-default queues
69
107
 
70
- Version 2.0.1
108
+ ## [2.0.1]
109
+
110
+ ## [2.0.0]
71
111
 
72
- Version 2.0.0
73
112
  - Simpler setup via QC::Setup.create (rake qc:create) & QC::Setup.drop (rake
74
113
  qc:drop)
75
114
  - Simpler abstractions in implementation
76
115
  - Better support for instrumentation via log_yield hook in QC module
77
116
  - Multiple queues use one table with a queue_name column
78
117
 
79
- Version 1.0.2
118
+ ## [1.0.2]
119
+
80
120
  - Update to latest okjson as the current has bugs
81
121
 
82
- Version 1.0.1
122
+ ## [1.0.1]
123
+
83
124
  - Using OkJson instead of any sort of rubygem
84
125
  - Remove html from docs
85
126
  - Use parameterised queries
@@ -87,35 +128,42 @@ Version 1.0.1
87
128
  - Injection attack bug fixed in lock_head()
88
129
  - Notificaiton get sent on seperate chans for disjoint queues
89
130
 
90
- Version 1.0.0rc1
131
+ ## [1.0.0rc1] - 2011-08-29
132
+
91
133
  - Removed json gem and relying on ruby 1.9.2's stdlib
92
134
  - Added better documentation
93
135
 
94
- Version 0.3.6pre
136
+ ## [0.3.6pre]
137
+
95
138
  - Added listen/notify support configured by $QC_LISTENING_WORKER otherwise uses Kernel.sleep()
96
139
 
97
- Version 0.3.5pre
140
+ ## [0.3.5pre] - 2011-08-27
141
+
98
142
  - Removed debug statement. Mistake!
99
143
 
100
- Version 0.3.4pre
144
+ ## [0.3.4pre]
145
+
101
146
  - Added logging configured by $VERBOSE or $QC_VERBOSE.
102
147
  - Added a method setup_child that gets called right after a worker forks.
103
148
  - Removed database helper methods: create_table, drop_table, silence_warnings.
104
149
  - Removed queue connection helper methods. Status should be discoverd by psql or the likes.
105
150
 
106
- Version 0.3.3pre
151
+ ## [0.3.3pre]
152
+
107
153
  - Removed PUB/SUB
108
154
  - Added GC after working a job
109
155
  - Added support for a database_url other than $DATABASE_URL. $QC_DATABASE_URL
110
156
  - Added exp backoff configured by $QC_MAX_LOCK_ATTEMPTS (default = 5)
111
157
  - Added option for forking worker configured by $QC_FORK_WORKER (default = false)
112
158
 
113
- Version 0.3.2
159
+ ## [0.3.2] - 2011-08-03
160
+
114
161
  - Fixed bug which caused workers to consume 2 connections. Now they only consume 1
115
162
  - Added a rake file for tests
116
163
  - Added support for postgres:///db_name DATABASE_URLs
117
164
 
118
- Version 0.3.1
165
+ ## [0.3.1] - 2011-04-27
166
+
119
167
  - Added query interface for introspection success
120
168
  - Moved the locking of jobs into the DB as a PG function. SELECT lock_head()
121
169
  - Added requirement for DB connection. MUST BE URI i.e. DATABASE_URL=postgres://user:pass@localhost/db_name
@@ -126,21 +174,26 @@ Version 0.3.1
126
174
  - Renamed the jobs channel to queue_classic_jobs
127
175
  - Added support for multiple queues
128
176
 
129
- Version 0.2.2
177
+ ## [0.2.2] - 2011-02-26
178
+
130
179
  - Fixed problems with enqueueing a list of parameters.
131
180
 
132
- Version 0.2.1
181
+ ## [0.2.1] - 2011-02-22
182
+
133
183
  - Added method for handling errors.
134
184
  - Added ability to enqueue a Job instance. Makes retrying jobs easier.
135
185
  - Added delete_all.
136
186
  - Fixed connection algorithm. 1 connection per process.
137
187
  - Fixed API for enqueue. Now accepting 1 arg or many args.
138
188
 
139
- Version 0.2.0
189
+ ## [0.2.0] - 2011-02-17
190
+
140
191
  - Beta Release
141
192
  - Added method for handling failed jobs
142
193
  - Added Benchmarks
143
194
  - Removed logging
144
195
  - Moved the Job class into it's own file
145
196
 
146
- 0.1.6
197
+ ## [0.1.6] - 2011-02-03
198
+
199
+ - Early release
@@ -0,0 +1,46 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
6
+
7
+ ## Our Standards
8
+
9
+ Examples of behavior that contributes to creating a positive environment include:
10
+
11
+ * Using welcoming and inclusive language
12
+ * Being respectful of differing viewpoints and experiences
13
+ * Gracefully accepting constructive criticism
14
+ * Focusing on what is best for the community
15
+ * Showing empathy towards other community members
16
+
17
+ Examples of unacceptable behavior by participants include:
18
+
19
+ * The use of sexualized language or imagery and unwelcome sexual attention or advances
20
+ * Trolling, insulting/derogatory comments, and personal or political attacks
21
+ * Public or private harassment
22
+ * Publishing others' private information, such as a physical or electronic address, without explicit permission
23
+ * Other conduct which could reasonably be considered inappropriate in a professional setting
24
+
25
+ ## Our Responsibilities
26
+
27
+ Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
28
+
29
+ Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
30
+
31
+ ## Scope
32
+
33
+ This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
34
+
35
+ ## Enforcement
36
+
37
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at isitaboat+qc@gmail.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
38
+
39
+ Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
40
+
41
+ ## Attribution
42
+
43
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
44
+
45
+ [homepage]: http://contributor-covenant.org
46
+ [version]: http://contributor-covenant.org/version/1/4/
data/Gemfile CHANGED
@@ -1,9 +1,13 @@
1
- source "https://rubygems.org"
1
+ # frozen_string_literal: true
2
+ source 'https://rubygems.org'
2
3
 
3
- gem "rake"
4
+ source 'https://rubygems.org' do
5
+ gem 'rake'
4
6
 
5
- gemspec
7
+ gemspec
6
8
 
7
- group :test do
8
- gem 'minitest', '~> 5.5.1'
9
+ group :test do
10
+ gem 'minitest', '~> 5.8'
11
+ gem 'minitest-reporters'
12
+ end
9
13
  end