que 1.0.0.beta5 → 1.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 90ae5c7c65d93f4ea3833d80351c2e0d3ed3f706ba00411937e8aba2137a41d7
4
- data.tar.gz: 58aba934ee2735522e602dd5777de666c6acbee1cb531d9918ce6f2b75f3b693
3
+ metadata.gz: 967eded27a81df945b65911275dcdc7925113727bfd06fbc42cb780a3bfb6fcf
4
+ data.tar.gz: 87607b262263b13623b4b09822a478c02179e57cfa503f253e3e678ac8b1490a
5
5
  SHA512:
6
- metadata.gz: 1cbfa5f3c6d13868897ac9f1fcf079aaeaa29c9eadfd231044a484878fbf8e99f4b159a4458690027b181b29c5da3abafcc1c9de2d33b3c487b1fac4a15c7e50
7
- data.tar.gz: 1ed4b71f51cd72e3968de6ec745e2de592dc021e9d334526630b3011d3f5526388b8562da4bf513066cb68d4dee033043115027ddfc0e9bef5e816ed2c35ede5
6
+ metadata.gz: 3d8242fe07445a6f4658f322dd1a7cdb8918c8a56c2bfad2db49c58695b4ebae183e6a56159abc4ff74316006e7d95459ab6cfd8a4c98410eea50a2e4fb52509
7
+ data.tar.gz: 24fdf63cab7ff6e8a2ffa6ed598718d3c2ae2cbc6ef61de22ab6f1541b0cc23bab980fe5d10bbcb54823b2a529cd1cdc9d57437db3905e802bec92a04ea3ed94
data/CHANGELOG.md CHANGED
@@ -1,3 +1,71 @@
1
+ ### 1.2.0 (2022-02-23)
2
+
3
+ - **Deprecated**
4
+ + Providing job options as top level keyword arguments to Job.enqueue is now deprecated. Support will be dropped in `2.0`. Job options should be nested under the `job_options` keyword arg instead. See [#336](https://github.com/que-rb/que/pull/336)
5
+
6
+ ### 1.1.0 (2022-02-21)
7
+
8
+ - **Features**:
9
+ + Add backtrace to errors, by [@trammel](https://github.com/trammel) in [#328](https://github.com/que-rb/que/pull/328)
10
+ - **Internal**:
11
+ + Add Dockerised development environment, in [#324](https://github.com/que-rb/que/pull/324)
12
+
13
+ ### 1.0.0 (2022-01-24)
14
+
15
+ _This release does not add any changes on top of 1.0.0.beta5._
16
+
17
+ ### 1.0.0.beta5 (2021-12-23)
18
+
19
+ * **Bug fixes and improvements**
20
+ * Add more context to error message when config files fail to load. by [@trammel](https://github.com/trammel) in [#293](https://github.com/que-rb/que/pull/293)
21
+ * Fix lock leak on PostgreSQL 12 and later by [@jasoncodes](https://github.com/jasoncodes) in [#298](https://github.com/que-rb/que/pull/298)
22
+ * Fix deadlock issue [#318](https://github.com/que-rb/que/pull/318)
23
+ * Fix thread attrition issue [#321](https://github.com/que-rb/que/pull/321)
24
+
25
+ * **Rails fixes:**
26
+ * Set schema in table_name for ActiveRecord model by [@nikitug](https://github.com/nikitug) in [#274](https://github.com/que-rb/que/pull/274)
27
+
28
+ * **Documentation:**
29
+ * Add link to que-locks for exclusive job locking by [@airhorns](https://github.com/airhorns) in [#263](https://github.com/que-rb/que/pull/263)
30
+ [`5259303`](https://github.com/que-rb/que/commit/52593031a7eef2d52ac38eceb2d8df776ec74090)
31
+ * Fix links to Writing Reliable Jobs by [@nikitug](https://github.com/nikitug) in [#273](https://github.com/que-rb/que/pull/273)
32
+ * Add build badge to README by [@jonathanhefner](https://github.com/jonathanhefner) in [#278](https://github.com/que-rb/que/pull/278)
33
+ * Fix ToC links in docs by [@swrobel](https://github.com/swrobel) in [#287](https://github.com/que-rb/que/pull/287)
34
+ * Note all Rails queue names that must be changed by [@swrobel](https://github.com/swrobel) in [#296](https://github.com/que-rb/que/pull/296)
35
+ * Add instructions for how to start Que by [@xcskier56](https://github.com/xcskier56) in [#292](https://github.com/que-rb/que/pull/292)
36
+
37
+ * **CI/tests**
38
+ * Fix CI failure from Docker Postgres image by [@jonathanhefner](https://github.com/jonathanhefner) in [#275](https://github.com/que-rb/que/pull/275)
39
+ * Test with Ruby 2.7 by [@jonathanhefner](https://github.com/jonathanhefner) in [#276](https://github.com/que-rb/que/pull/276)
40
+ * Run GitHub build workflow on push by [@jonathanhefner](https://github.com/jonathanhefner) in [#277](https://github.com/que-rb/que/pull/277)
41
+
42
+
43
+ **Full Changelog**: [`v1.0.0.beta4...v1.0.0.beta5`](https://github.com/que-rb/que/compare/v1.0.0.beta4...v1.0.0.beta5)
44
+
45
+ **Unless an issue is found we intend for this release to become v1.0.0 proper.**
46
+
47
+ ---
48
+
49
+ ### 1.0.0.beta4 (2020-01-17)
50
+
51
+ - Rails 6 compatibility: Fix time parsing #249 and https://github.com/que-rb/que/commit/5ddddd5ebac6153d7a683ef08c86bced8e03fb51
52
+ - Cleaner sequel usage #257
53
+ - Documentation improvements #264 #269 #261 #231
54
+
55
+ ---
56
+
57
+ ### 1.0.0.beta3 (2018-05-18)
58
+
59
+ * Added support for customizing log levels for `job_worked` events (#217).
60
+
61
+ * Began logging all `job_errored` events at the `ERROR` log level.
62
+
63
+ * Fixed the Railtie when running in test mode (#214).
64
+
65
+ * Tweaked the meanings of worker-priorities and worker-count options in the CLI, to better support use cases with low worker counts (#216).
66
+
67
+ ---
68
+
1
69
  ### 1.0.0.beta2 (2018-04-13)
2
70
 
3
71
  * **A schema upgrade to version 4 will be required for this release.** See [the migration doc](https://github.com/que-rb/que/blob/master/docs/migrating.md) for information if you're upgrading from a previous release.
@@ -111,8 +179,123 @@
111
179
  * Names for queues and job classes are now limited to 500 characters, which is still far longer than either of these values should reasonably be.
112
180
 
113
181
  * There is now a `data` JSONB column which is used to support various ways of organizing jobs (setting tags on them, etc).
182
+ ---
183
+
184
+ ### 1.0.0.beta (2017-10-25)
185
+
186
+ * **A schema upgrade to version 4 will be required for this release.** See [the migration doc](https://github.com/que-rb/que/blob/master/docs/migrating.md) for information if you're upgrading from a previous release.
187
+
188
+ * Please note that this migration requires a rewrite of the jobs table, which makes it O(n) with the size of the table. If you have a very large backlog of jobs you may want to schedule downtime for this migration.
189
+
190
+ * Que's implementation has been changed from one in which worker threads hold their own PG connections and lock their own jobs to one in which a single thread (and PG connection) locks jobs through LISTEN/NOTIFY and batch polling, and passes jobs along to worker threads. This has many benefits, including:
191
+
192
+ * Jobs queued for immediate processing can be actively distributed to workers with LISTEN/NOTIFY, which is more efficient than having workers repeatedly poll for new jobs.
193
+
194
+ * When polling is necessary (to pick up jobs that are scheduled for the future or that need to be retried due to errors), jobs can be locked and fetched in batches, rather than one at a time.
195
+
196
+ * Individual workers no longer need to monopolize their own (usually idle) connections while working jobs, so Ruby processes will require fewer Postgres connections.
197
+
198
+ * PgBouncer or another external connection pool can be used for workers' connections (though not for the connection used to lock and listen for jobs).
199
+
200
+ * Other features introduced in this version include:
201
+
202
+ * Much better support for all versions of ActiveJob.
114
203
 
115
- For a detailed list of the changes between each beta release of 1.0.0, see [the beta Changelog](CHANGELOG.1.0.beta.md).
204
+ * In particular, you may (optionally) include `Que::ActiveJob::JobExtensions` into `ApplicationJob` to get support for all of Que's job helper methods.
205
+
206
+ * Custom middleware that wrap running jobs are now supported.
207
+
208
+ * Support for categorizing jobs with tags.
209
+
210
+ * Support for configuring a `maximum_retry_count` on individual job classes.
211
+
212
+ * Job configuration options are now inheritable, so job class hierarchies are more useful.
213
+
214
+ * There are now built-in models for ActiveRecord and Sequel to allow inspecting the queue easily.
215
+
216
+ * Jobs that have finished working may optionally be retained in the database indefinitely.
217
+
218
+ * To keep a job record, replace the `destroy` calls in your jobs with `finish`. `destroy` will still delete records entirely, for jobs that you don't want to keep.
219
+
220
+ * If you don't resolve a job yourself one way or another, Que will still `destroy` the job for you by default.
221
+
222
+ * Finished jobs have a timestamp set in the finished_at column.
223
+
224
+ * Jobs that have errored too many times will now be marked as expired, and won't be retried again.
225
+
226
+ * You can configure a maximum_retry_count in your job classes, to set the threshold at which a job will be marked expired. The default is 15.
227
+
228
+ * To manually mark a job as expired (and keep it in the database but not try to run it again) you can call `expire` helper in your job.
229
+
230
+ * You can now set job priority thresholds for individual workers, to ensure that there will always be space available for high-priority jobs.
231
+
232
+ * `Que.job_states` returns a list of locked jobs and the hostname/pid of the Ruby processes that have locked them.
233
+
234
+ * `Que.connection_proc=` has been added, to allow for the easy integration of custom connection pools.
235
+
236
+ * In keeping with semantic versioning, the major version is being bumped since the new implementation requires some backwards-incompatible changes. These changes include:
237
+
238
+ * Support for MRI Rubies before 2.2 has been dropped.
239
+
240
+ * Support for Postgres versions before 9.5 has been dropped (JSONB and upsert support is required).
241
+
242
+ * JRuby support has been dropped. It will be reintroduced whenever the jruby-pg gem is production-ready.
243
+
244
+ * The `que:work` rake task has been removed. Use the `que` executable instead.
245
+
246
+ * Therefore, configuring workers using QUE_* environment variables is no longer supported. Please pass the appropriate options to the `que` executable instead.
247
+
248
+ * The `mode` setter has been removed.
249
+
250
+ * To run jobs synchronously when they are enqueued (the old `:sync` behavior) you can set `Que.run_synchronously = true`.
251
+
252
+ * To start up the worker pool (the old :async behavior) you should use the `que` executable to start up a worker process. There's no longer a supported API for running workers outside of the `que` executable.
253
+
254
+ * The following methods are not meaningful under the new implementation and have been removed:
255
+
256
+ * The `Que.wake_interval` getter and setter.
257
+
258
+ * The `Que.worker_count` getter and setter.
259
+
260
+ * `Que.wake!`
261
+
262
+ * `Que.wake_all!`
263
+
264
+ * Since Que needs a dedicated Postgres connection to manage job locks, running Que through a single PG connection is no longer supported.
265
+
266
+ * It's not clear that anyone ever actually did this.
267
+
268
+ * `Que.worker_states` has been removed, as the connection that locks a job is no longer the one that the job is using to run. Its functionality has been partially replaced with `Que.job_states`.
269
+
270
+ * When using Rails, for simplicity, job attributes and keys in argument hashes are now converted to symbols when retrieved from the database, rather than being converted to instances of HashWithIndifferentAccess.
271
+
272
+ * Arguments passed to jobs are now deep-frozen, to prevent unexpected behavior when the args are mutated and the job is reenqueued.
273
+
274
+ * Since JSONB is now used to store arguments, the order of argument hashes is no longer maintained.
275
+
276
+ * It wouldn't have been a good idea to rely on this anyway.
277
+
278
+ * Calling Que.log() directly is no longer supported/recommended.
279
+
280
+ * Features marked as deprecated in the final 0.x releases have been removed.
281
+
282
+ * Finally, if you've built up your own tooling and customizations around Que, you may need to be aware of some DB schema changes made in the migration to schema version #4.
283
+
284
+ * The `job_id` column has been renamed `id` and is now the primary key. This makes it easier to manage the queue using an ActiveRecord model.
285
+
286
+ * Finished jobs are now kept in the DB, unless you explicitly call `destroy`. If you want to query the DB for only jobs that haven't finished yet, add a `WHERE finished_at IS NULL` condition to your query, or use the not_finished scope on one of the provided ORM models.
287
+
288
+ * There is now an `expired_at` timestamp column, which is set when a job reaches its maximum number of retries and will not be attempted again.
289
+
290
+ * Due to popular demand, the default queue name is now "default" rather than an empty string. The migration will move pending jobs under the "" queue to the "default" queue.
291
+
292
+ * The `last_error` column has been split in two, to `last_error_message` and `last_error_backtrace`. These two columns are now limited to 500 and 10,000 characters, respectively. The migration will split old error data correctly, and truncate it if necessary.
293
+
294
+ * Names for queues and job classes are now limited to 500 characters, which is still far longer than either of these values should reasonably be.
295
+
296
+ * There is now a `data` JSONB column which is used to support various ways of organizing jobs (setting tags on them, etc).
297
+
298
+ ---
116
299
 
117
300
  ### 0.14.3 (2018-03-02)
118
301
 
@@ -120,6 +303,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
120
303
 
121
304
  * Recorded error messages are now truncated to the first 500 characters.
122
305
 
306
+ ---
307
+
123
308
  ### 0.14.2 (2018-01-05)
124
309
 
125
310
  * Deprecate the Que.disable_prepared_statements= accessors.
@@ -128,24 +313,34 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
128
313
 
129
314
  * Update the generated Rails migration to declare a version. (NARKOZ)
130
315
 
316
+ ---
317
+
131
318
  ### 0.14.1 (2017-12-14)
132
319
 
133
320
  * Fix a bug with typecasting boolean values on Rails 5+.
134
321
 
322
+ ---
323
+
135
324
  ### 0.14.0 (2017-08-11)
136
325
 
137
326
  * Fix incompatibility with Rails 5.1.
138
327
 
139
328
  * Drop support for waking an in-process worker when an ActiveRecord transaction commits.
140
329
 
330
+ ---
331
+
141
332
  ### 0.13.1 (2017-07-05)
142
333
 
143
334
  * Fix issue that caused error stacktraces to not be persisted in most cases.
144
335
 
336
+ ---
337
+
145
338
  ### 0.13.0 (2017-06-08)
146
339
 
147
340
  * Fix recurring JSON issues by dropping MultiJson support. Previously MultiJson was detected and used automatically, and now it's just ignored and stdlib JSON used instead, so this shouldn't require any code changes.
148
341
 
342
+ ---
343
+
149
344
  ### 0.12.3 (2017-06-01)
150
345
 
151
346
  * Fix incompatibility with MultiJson introduced by the previous release.
@@ -154,28 +349,40 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
154
349
 
155
350
  * Fix security vulnerability in parsing JSON from the DB (by specifying create_additions: false). This shouldn't be a concern unless you were passing untrusted user input in your job arguments. (hmac)
156
351
 
352
+ ---
353
+
157
354
  ### 0.12.1 (2017-01-22)
158
355
 
159
356
  * Fix incompatibility with Rails 5.0. (#166) (nbibler, thedarkone)
160
357
 
358
+ ---
359
+
161
360
  ### 0.12.0 (2016-09-09)
162
361
 
163
362
  * The error_handler configuration option has been renamed to error_notifier, which is more descriptive of what it's actually supposed to do. You can still use error_handler for configuration, but you'll get a warning.
164
363
 
165
364
  * Introduced a new framework for handling errors on a per-job basis. See the docs for more information. (#106, #147)
166
365
 
366
+ ---
367
+
167
368
  ### 0.11.6 (2016-07-01)
168
369
 
169
370
  * Fix for operating in nested transactions in Rails 5.0. (#160) (greysteil)
170
371
 
372
+ ---
373
+
171
374
  ### 0.11.5 (2016-05-13)
172
375
 
173
376
  * Fix error when running `que -v`. (#154) (hardbap)
174
377
 
378
+ ---
379
+
175
380
  ### 0.11.4 (2016-03-03)
176
381
 
177
382
  * Fix incompatibility with ActiveRecord 5.0.0.beta3. (#143, #144) (joevandyk)
178
383
 
384
+ ---
385
+
179
386
  ### 0.11.3 (2016-02-26)
180
387
 
181
388
  * Fixed bug with displaying the current version of the que executable. (#122) (hardbap)
@@ -186,14 +393,20 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
186
393
 
187
394
  * String literals are now frozen on Ruby 2.3.
188
395
 
396
+ ---
397
+
189
398
  ### 0.11.2 (2015-09-09)
190
399
 
191
400
  * Fix Job class constantizing when ActiveSupport isn't loaded. (#121) (godfat)
192
401
 
402
+ ---
403
+
193
404
  ### 0.11.1 (2015-09-04)
194
405
 
195
406
  * The `rake que:work` rake task that was specific to Rails has been deprecated and will be removed in Que 1.0. A deprecation warning will display when it is run.
196
407
 
408
+ ---
409
+
197
410
  ### 0.11.0 (2015-09-04)
198
411
 
199
412
  * A command-line program has been added that can be used to work jobs in a more flexible manner than the previous rake task. Run `que -h` for more information.
@@ -208,6 +421,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
208
421
 
209
422
  * If it exists, use String#constantize to constantize job classes, since ActiveSupport's constantize method behaves better with Rails' autoloading. (#115, #120) (joevandyk)
210
423
 
424
+ ---
425
+
211
426
  ### 0.10.0 (2015-03-18)
212
427
 
213
428
  * When working jobs via the rake task, Rails applications are now eager-loaded if present, to avoid problems with multithreading and autoloading. (#96) (hmarr)
@@ -216,28 +431,40 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
216
431
 
217
432
  * Add Que.transaction() helper method, to aid in transaction management in migrations or when the user's ORM doesn't provide one. (#81)
218
433
 
434
+ ---
435
+
219
436
  ### 0.9.2 (2015-02-05)
220
437
 
221
438
  * Fix a bug wherein the at_exit hook in the railtie wasn't waiting for jobs to finish before exiting.
222
439
 
223
440
  * Fix a bug wherein the que:work rake task wasn't waiting for jobs to finish before exiting. (#85) (tycooon)
224
441
 
442
+ ---
443
+
225
444
  ### 0.9.1 (2015-01-11)
226
445
 
227
446
  * Use now() rather than 'now' when inserting jobs, to avoid using an old value as the default run_at in prepared statements. (#74) (bgentry)
228
447
 
448
+ ---
449
+
229
450
  ### 0.9.0 (2014-12-16)
230
451
 
231
452
  * The error_handler callable is now passed two objects, the error and the job that raised it. If your current error_handler is a proc, as recommended in the docs, you shouldn't need to make any code changes, unless you want to use the job in your error handling. If your error_handler is a lambda, or another callable with a strict arity requirement, you'll want to change it before upgrading. (#69) (statianzo)
232
453
 
454
+ ---
455
+
233
456
  ### 0.8.2 (2014-10-12)
234
457
 
235
458
  * Fix errors raised during rollbacks in the ActiveRecord adapter, which remained silent until Rails 4.2. (#64, #65) (Strech)
236
459
 
460
+ ---
461
+
237
462
  ### 0.8.1 (2014-07-28)
238
463
 
239
464
  * Fix regression introduced in the `que:work` rake task by the `mode` / `worker_count` disentangling in 0.8.0. (#50)
240
465
 
466
+ ---
467
+
241
468
  ### 0.8.0 (2014-07-12)
242
469
 
243
470
  * A callable can now be set as the logger, like `Que.logger = proc { MyLogger.new }`. Que uses this in its Railtie for cleaner initialization, but it is also available for public use.
@@ -246,20 +473,28 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
246
473
 
247
474
  * Fixed a similar bug wherein setting a wake_interval during application startup would break worker awakening after the process was forked.
248
475
 
476
+ ---
477
+
249
478
  ### 0.7.3 (2014-05-19)
250
479
 
251
480
  * When mode = :sync, don't touch the database at all when running jobs inline. Needed for ActiveJob compatibility (#46).
252
481
 
482
+ ---
483
+
253
484
  ### 0.7.2 (2014-05-18)
254
485
 
255
486
  * Fix issue wherein intermittent worker wakeups would not work after forking (#44).
256
487
 
488
+ ---
489
+
257
490
  ### 0.7.1 (2014-04-29)
258
491
 
259
492
  * Fix errors with prepared statements when ActiveRecord reconnects to the database. (dvrensk)
260
493
 
261
494
  * Don't use prepared statements when inside a transaction. This negates the risk of a prepared statement error harming the entire transaction. The query that benefits the most from preparation is the job-lock CTE, which is never run in a transaction, so the performance impact should be negligible.
262
495
 
496
+ ---
497
+
263
498
  ### 0.7.0 (2014-04-09)
264
499
 
265
500
  * `JobClass.queue(*args)` has been deprecated and will be removed in version 1.0.0. Please use `JobClass.enqueue(*args)` instead.
@@ -270,6 +505,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
270
505
 
271
506
  * The [Pond gem](https://github.com/chanks/pond) is now supported as a connection. It is very similar to the ConnectionPool gem, but creates connections lazily and is dynamically resizable.
272
507
 
508
+ ---
509
+
273
510
  ### 0.6.0 (2014-02-04)
274
511
 
275
512
  * **A schema upgrade to version 3 is required for this release.** See [the migration doc](https://github.com/que-rb/que/blob/master/docs/migrating.md) for information if you're upgrading from a previous release.
@@ -290,6 +527,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
290
527
 
291
528
  * Job classes may now define their own logic for determining the retry interval when a job raises an error. See [error handling](https://github.com/que-rb/que/blob/master/docs/error_handling.md) for more information.
292
529
 
530
+ ---
531
+
293
532
  ### 0.5.0 (2014-01-14)
294
533
 
295
534
  * When running a worker pool inside your web process on ActiveRecord, Que will now wake a worker once a transaction containing a queued job is committed. (joevandyk, chanks)
@@ -312,6 +551,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
312
551
 
313
552
  Now, when a process exits, if the worker pool is running (whether in a rake task or in a web process) the exit will be stalled until all workers have finished their current jobs. If you have long-running jobs, this may take a long time. If you need the process to exit immediately, you can SIGKILL without any threat of commiting prematurely.
314
553
 
554
+ ---
555
+
315
556
  ### 0.4.0 (2014-01-05)
316
557
 
317
558
  * Que.wake_all! was added, as a simple way to wake up all workers in the pool.
@@ -332,6 +573,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
332
573
 
333
574
  * Much more internal cleanup.
334
575
 
576
+ ---
577
+
335
578
  ### 0.3.0 (2013-12-21)
336
579
 
337
580
  * Add Que.stop!, which immediately kills all jobs being worked in the process.
@@ -344,6 +587,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
344
587
 
345
588
  * Clean up internals and hammer out several race conditions.
346
589
 
590
+ ---
591
+
347
592
  ### 0.2.0 (2013-11-30)
348
593
 
349
594
  * Officially support JRuby 1.7.5+. Earlier versions may work.
@@ -356,6 +601,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
356
601
 
357
602
  * :sync mode now ignores scheduled jobs (jobs queued with a specific run_at).
358
603
 
604
+ ---
605
+
359
606
  ### 0.1.0 (2013-11-18)
360
607
 
361
608
  * Initial public release, after a test-driven rewrite.
@@ -366,6 +613,8 @@ For a detailed list of the changes between each beta release of 1.0.0, see [the
366
613
 
367
614
  Added a Railtie for easier setup with Rails, as well as a migration generator.
368
615
 
616
+ ---
617
+
369
618
  ### 0.0.1 (2013-11-07)
370
619
 
371
620
  * Copy-pasted from an app of mine. Very Sequel-specific. Nobody look at it, let's pretend it never happened.
data/Dockerfile ADDED
@@ -0,0 +1,20 @@
1
+ FROM ruby:2.7.5-slim-buster@sha256:4cbbe2fba099026b243200aa8663f56476950cc64ccd91d7aaccddca31e445b5 AS base
2
+
3
+ # Install libpq-dev in our base layer, as it's needed in all environments
4
+ RUN apt-get update \
5
+ && apt-get install -y libpq-dev \
6
+ && rm -rf /var/lib/apt/lists/*
7
+
8
+ ENV RUBY_BUNDLER_VERSION 2.3.7
9
+ RUN gem install bundler -v $RUBY_BUNDLER_VERSION
10
+
11
+ ENV BUNDLE_PATH /usr/local/bundle
12
+
13
+ ENV RUBYOPT=-W:deprecated
14
+
15
+ FROM base AS dev-environment
16
+
17
+ # Install build-essential and git, as we'd need them for building gems that have native code components
18
+ RUN apt-get update \
19
+ && apt-get install -y build-essential git \
20
+ && rm -rf /var/lib/apt/lists/*
data/README.md CHANGED
@@ -1,54 +1,59 @@
1
1
  # Que ![tests](https://github.com/que-rb/que/workflows/tests/badge.svg)
2
2
 
3
- **This README and the rest of the docs on the master branch all refer to Que 1.0, which is currently in beta. If you're using version 0.x, please refer to the docs on [the 0.x branch](https://github.com/que-rb/que/tree/0.x).**
3
+ **This README and the rest of the docs on the master branch all refer to Que 1.0. If you're using version 0.x, please refer to the docs on [the 0.x branch](https://github.com/que-rb/que/tree/0.x).**
4
4
 
5
5
  *TL;DR: Que is a high-performance job queue that improves the reliability of your application by protecting your jobs with the same [ACID guarantees](https://en.wikipedia.org/wiki/ACID) as the rest of your data.*
6
6
 
7
7
  Que ("keɪ", or "kay") is a queue for Ruby and PostgreSQL that manages jobs using [advisory locks](http://www.postgresql.org/docs/current/static/explicit-locking.html#ADVISORY-LOCKS), which gives it several advantages over other RDBMS-backed queues:
8
8
 
9
- * **Concurrency** - Workers don't block each other when trying to lock jobs, as often occurs with "SELECT FOR UPDATE"-style locking. This allows for very high throughput with a large number of workers.
10
- * **Efficiency** - Locks are held in memory, so locking a job doesn't incur a disk write. These first two points are what limit performance with other queues. Under heavy load, Que's bottleneck is CPU, not I/O.
11
- * **Safety** - If a Ruby process dies, the jobs it's working won't be lost, or left in a locked or ambiguous state - they immediately become available for any other worker to pick up.
9
+ - **Concurrency** - Workers don't block each other when trying to lock jobs, as often occurs with "SELECT FOR UPDATE"-style locking. This allows for very high throughput with a large number of workers.
10
+ - **Efficiency** - Locks are held in memory, so locking a job doesn't incur a disk write. These first two points are what limit performance with other queues. Under heavy load, Que's bottleneck is CPU, not I/O.
11
+ - **Safety** - If a Ruby process dies, the jobs it's working won't be lost, or left in a locked or ambiguous state - they immediately become available for any other worker to pick up.
12
12
 
13
13
  Additionally, there are the general benefits of storing jobs in Postgres, alongside the rest of your data, rather than in Redis or a dedicated queue:
14
14
 
15
- * **Transactional Control** - Queue a job along with other changes to your database, and it'll commit or rollback with everything else. If you're using ActiveRecord or Sequel, Que can piggyback on their connections, so setup is simple and jobs are protected by the transactions you're already using.
16
- * **Atomic Backups** - Your jobs and data can be backed up together and restored as a snapshot. If your jobs relate to your data (and they usually do), there's no risk of jobs falling through the cracks during a recovery.
17
- * **Fewer Dependencies** - If you're already using Postgres (and you probably should be), a separate queue is another moving part that can break.
18
- * **Security** - Postgres' support for SSL connections keeps your data safe in transport, for added protection when you're running workers on cloud platforms that you can't completely control.
15
+ - **Transactional Control** - Queue a job along with other changes to your database, and it'll commit or rollback with everything else. If you're using ActiveRecord or Sequel, Que can piggyback on their connections, so setup is simple and jobs are protected by the transactions you're already using.
16
+ - **Atomic Backups** - Your jobs and data can be backed up together and restored as a snapshot. If your jobs relate to your data (and they usually do), there's no risk of jobs falling through the cracks during a recovery.
17
+ - **Fewer Dependencies** - If you're already using Postgres (and you probably should be), a separate queue is another moving part that can break.
18
+ - **Security** - Postgres' support for SSL connections keeps your data safe in transport, for added protection when you're running workers on cloud platforms that you can't completely control.
19
19
 
20
20
  Que's primary goal is reliability. You should be able to leave your application running indefinitely without worrying about jobs being lost due to a lack of transactional support, or left in limbo due to a crashing process. Que does everything it can to ensure that jobs you queue are performed exactly once (though the occasional repetition of a job can be impossible to avoid - see the docs on [how to write a reliable job](/docs/README.md#writing-reliable-jobs)).
21
21
 
22
22
  Que's secondary goal is performance. The worker process is multithreaded, so that a single process can run many jobs simultaneously.
23
23
 
24
24
  Compatibility:
25
+
25
26
  - MRI Ruby 2.2+
26
27
  - PostgreSQL 9.5+
27
28
  - Rails 4.1+ (optional)
28
29
 
29
30
  **Please note** - Que's job table undergoes a lot of churn when it is under high load, and like any heavily-written table, is susceptible to bloat and slowness if Postgres isn't able to clean it up. The most common cause of this is long-running transactions, so it's recommended to try to keep all transactions against the database housing Que's job table as short as possible. This is good advice to remember for any high-activity database, but bears emphasizing when using tables that undergo a lot of writes.
30
31
 
31
-
32
32
  ## Installation
33
33
 
34
34
  Add this line to your application's Gemfile:
35
35
 
36
- gem 'que'
36
+ ```ruby
37
+ gem 'que'
38
+ ```
37
39
 
38
40
  And then execute:
39
41
 
40
- $ bundle
42
+ ```bash
43
+ bundle
44
+ ```
41
45
 
42
46
  Or install it yourself as:
43
47
 
44
- $ gem install que
45
-
48
+ ```bash
49
+ gem install que
50
+ ```
46
51
 
47
52
  ## Usage
48
53
 
49
54
  First, create the queue schema in a migration. For example:
50
55
 
51
- ``` ruby
56
+ ```ruby
52
57
  class CreateQueSchema < ActiveRecord::Migration[5.0]
53
58
  def up
54
59
  # Whenever you use Que in a migration, always specify the version you're
@@ -66,7 +71,7 @@ end
66
71
 
67
72
  Create a class for each type of job you want to run:
68
73
 
69
- ``` ruby
74
+ ```ruby
70
75
  # app/jobs/charge_credit_card.rb
71
76
  class ChargeCreditCard < Que::Job
72
77
  # Default settings for this job. These are optional - without them, jobs
@@ -101,7 +106,7 @@ end
101
106
 
102
107
  Queue your job. Again, it's best to do this in a transaction with other changes you're making. Also note that any arguments you pass will be serialized to JSON and back again, so stick to simple types (strings, integers, floats, hashes, and arrays).
103
108
 
104
- ``` ruby
109
+ ```ruby
105
110
  CreditCard.transaction do
106
111
  # Persist credit card information
107
112
  card = CreditCard.create(params[:credit_card])
@@ -111,17 +116,18 @@ end
111
116
 
112
117
  You can also add options to run the job after a specific time, or with a specific priority:
113
118
 
114
- ``` ruby
119
+ ```ruby
115
120
  ChargeCreditCard.enqueue card.id, user_id: current_user.id, run_at: 1.day.from_now, priority: 5
116
121
  ```
117
122
  ## Running the Que Worker
118
123
  In order to process jobs, you must start a separate worker process outside of your main server.
119
124
 
120
- ```
125
+ ```bash
121
126
  bundle exec que
122
127
  ```
123
128
 
124
129
  Try running `que -h` to get a list of runtime options:
130
+
125
131
  ```
126
132
  $ que -h
127
133
  usage: que [options] [file/to/require] ...
@@ -138,21 +144,24 @@ If you're using ActiveRecord to dump your database's schema, please [set your sc
138
144
 
139
145
  Pre-1.0, the default queue name needed to be configured in order for Que to work out of the box with Rails. In 1.0 the default queue name is now 'default', as Rails expects, but when Rails enqueues some types of jobs it may try to use another queue name that isn't worked by default. You can either:
140
146
 
141
- * [Configure Rails](https://guides.rubyonrails.org/configuring.html) to send all internal job types to the 'default' queue by adding the following to `config/application.rb`:
142
- ```ruby
143
- config.action_mailer.deliver_later_queue_name = :default
144
- config.action_mailbox.queues.incineration = :default
145
- config.action_mailbox.queues.routing = :default
146
- config.active_storage.queues.analysis = :default
147
- config.active_storage.queues.purge = :default
148
- ```
147
+ - [Configure Rails](https://guides.rubyonrails.org/configuring.html) to send all internal job types to the 'default' queue by adding the following to `config/application.rb`:
149
148
 
150
- * [Tell que](/docs#multiple-queues) to work all of these queues (less efficient because it requires polling all of them):
151
- ```
152
- que -q default -q mailers -q action_mailbox_incineration -q action_mailbox_routing -q active_storage_analysis -q active_storage_purge
153
- ```
149
+ ```ruby
150
+ config.action_mailer.deliver_later_queue_name = :default
151
+ config.action_mailbox.queues.incineration = :default
152
+ config.action_mailbox.queues.routing = :default
153
+ config.active_storage.queues.analysis = :default
154
+ config.active_storage.queues.purge = :default
155
+ ```
156
+
157
+ - [Tell que](/docs#multiple-queues) to work all of these queues (less efficient because it requires polling all of them):
158
+
159
+ ```bash
160
+ que -q default -q mailers -q action_mailbox_incineration -q action_mailbox_routing -q active_storage_analysis -q active_storage_purge
161
+ ```
154
162
 
155
163
  Also, if you would like to integrate Que with Active Job, you can do it by setting the adapter in `config/application.rb` or in a specific environment by setting it in `config/environments/production.rb`, for example:
164
+
156
165
  ```ruby
157
166
  config.active_job.queue_adapter = :que
158
167
  ```
@@ -183,9 +192,9 @@ If you have a project that uses or relates to Que, feel free to submit a PR addi
183
192
 
184
193
  ## Community and Contributing
185
194
 
186
- * For bugs in the library, please feel free to [open an issue](https://github.com/que-rb/que/issues/new).
187
- * For general discussion and questions/concerns that don't relate to obvious bugs, join our [Discord Server](https://discord.gg/B3EW32H).
188
- * For contributions, pull requests submitted via Github are welcome.
195
+ - For bugs in the library, please feel free to [open an issue](https://github.com/que-rb/que/issues/new).
196
+ - For general discussion and questions/concerns that don't relate to obvious bugs, join our [Discord Server](https://discord.gg/B3EW32H).
197
+ - For contributions, pull requests submitted via Github are welcome.
189
198
 
190
199
  Regarding contributions, one of the project's priorities is to keep Que as simple, lightweight and dependency-free as possible, and pull requests that change too much or wouldn't be useful to the majority of Que's users have a good chance of being rejected. If you're thinking of submitting a pull request that adds a new feature, consider starting a discussion in [que-talk](https://groups.google.com/forum/#!forum/que-talk) first about what it would do and how it would be implemented. If it's a sufficiently large feature, or if most of Que's users wouldn't find it useful, it may be best implemented as a standalone gem, like some of the related projects above.
191
200
 
@@ -193,12 +202,54 @@ Regarding contributions, one of the project's priorities is to keep Que as simpl
193
202
 
194
203
  A note on running specs - Que's worker system is multithreaded and therefore prone to race conditions. As such, if you've touched that code, a single spec run passing isn't a guarantee that any changes you've made haven't introduced bugs. One thing I like to do before pushing changes is rerun the specs many times and watching for hangs. You can do this from the command line with something like:
195
204
 
196
- for i in {1..1000}; do SEED=$i bundle exec rake; done
205
+ ```bash
206
+ for i in {1..1000}; do SEED=$i bundle exec rake; done
207
+ ```
197
208
 
198
209
  This will iterate the specs one thousand times, each with a different ordering. If the specs hang, note what the seed number was on that iteration. For example, if the previous specs finished with a "Randomized with seed 328", you know that there's a hang with seed 329, and you can narrow it down to a specific spec with:
199
210
 
200
- for i in {1..1000}; do LOG_SPEC=true SEED=328 bundle exec rake; done
211
+ ```bash
212
+ for i in {1..1000}; do LOG_SPEC=true SEED=328 bundle exec rake; done
213
+ ```
201
214
 
202
215
  Note that we iterate because there's no guarantee that the hang would reappear with a single additional run, so we need to rerun the specs until it reappears. The LOG_SPEC parameter will output the name and file location of each spec before it is run, so you can easily tell which spec is hanging, and you can continue narrowing things down from there.
203
216
 
204
217
  Another helpful technique is to replace an `it` spec declaration with `hit` - this will run that particular spec 100 times during the run.
218
+
219
+ #### With Docker
220
+
221
+ We've provided a Dockerised environment to avoid the need to manually: install Ruby, install the gem bundle, set up Postgres, and connect to the database.
222
+
223
+ To run the specs using this environment, run:
224
+
225
+ ```bash
226
+ ./auto/test
227
+ ```
228
+
229
+ To get a shell in the environment, run:
230
+
231
+ ```bash
232
+ ./auto/dev
233
+ ```
234
+
235
+ The [Docker Compose config](docker-compose.yml) provides a convenient way to inject your local shell aliases into the Docker container. Simply create a file containing your alias definitions (or which sources them from other files) at `~/.docker-rc.d/.docker-bashrc`, and they will be available inside the container.
236
+
237
+ #### Without Docker
238
+
239
+ You'll need to have Postgres running. Assuming you have it running on port 5697, with a `que-test` database, and a username & password of `que`, you can run:
240
+
241
+ ```bash
242
+ DATABASE_URL=postgres://que:que@localhost:5697/que-test bundle exec rake
243
+ ```
244
+
245
+ If you don't already have Postgres, you could use Docker Compose to run just the database:
246
+
247
+ ```bash
248
+ docker compose up -d db
249
+ ```
250
+
251
+ If you want to try a different version of Postgres, e.g. 12:
252
+
253
+ ```bash
254
+ export POSTGRES_VERSION=12
255
+ ```
data/auto/dev ADDED
@@ -0,0 +1,21 @@
1
+ #!/bin/bash
2
+ #
3
+ # Operate in development environment
4
+
5
+ set -Eeuo pipefail
6
+
7
+ cd "$(dirname "$0")/.."
8
+
9
+ docker compose build dev
10
+
11
+ # Delete containers and DB volume afterwards on CI
12
+ if [[ "${CI-}" == "true" ]]; then
13
+ trap '{
14
+ echo "Stopping containers..."
15
+ docker compose down
16
+ docker volume rm -f que_db-data
17
+ }' EXIT
18
+ fi
19
+
20
+ set -x
21
+ docker compose run --rm dev "${@-bash}"
data/auto/psql ADDED
@@ -0,0 +1,9 @@
1
+ #!/bin/bash
2
+ #
3
+ # Open a database shell
4
+
5
+ set -Eeuo pipefail
6
+
7
+ cd "$(dirname "$0")/.."
8
+
9
+ docker compose run --rm pg-dev psql "${@-}"
data/auto/test ADDED
@@ -0,0 +1,5 @@
1
+ #!/bin/bash
2
+
3
+ set -Eeuo pipefail
4
+
5
+ "$(dirname "$0")"/dev ./scripts/test "$@"
@@ -0,0 +1,17 @@
1
+ #!/bin/bash
2
+
3
+ set -Eeuo pipefail
4
+
5
+ export POSTGRES_VERSION=14
6
+
7
+ delete_db() {
8
+ docker compose down
9
+ docker volume rm -f que_db-data
10
+ }
11
+
12
+ trap delete_db EXIT
13
+
14
+ # pre-test cleanup is necessary as the existing db container will be used if it's running (potentially with the wrong PG version)
15
+ delete_db
16
+ "$(dirname "$0")"/test "$@"
17
+ delete_db
@@ -0,0 +1,46 @@
1
+ version: "3.7"
2
+
3
+ services:
4
+
5
+ dev:
6
+ build:
7
+ context: .
8
+ target: dev-environment
9
+ depends_on:
10
+ - db
11
+ volumes:
12
+ - .:/work
13
+ - ruby-2.7.5-gem-cache:/usr/local/bundle
14
+ - ~/.docker-rc.d/:/.docker-rc.d/:ro
15
+ working_dir: /work
16
+ entrypoint: /work/scripts/docker-entrypoint
17
+ command: bash
18
+ environment:
19
+ DATABASE_URL: postgres://que:que@db/que-test
20
+
21
+ db:
22
+ image: "postgres:${POSTGRES_VERSION-13}"
23
+ volumes:
24
+ - db-data:/var/lib/postgresql/data
25
+ environment:
26
+ POSTGRES_USER: que
27
+ POSTGRES_PASSWORD: que
28
+ POSTGRES_DB: que-test
29
+ ports:
30
+ - 5697:5432
31
+
32
+ pg-dev:
33
+ image: "postgres:${POSTGRES_VERSION-13}"
34
+ depends_on:
35
+ - db
36
+ entrypoint: []
37
+ command: psql
38
+ environment:
39
+ PGHOST: db
40
+ PGUSER: que
41
+ PGPASSWORD: que
42
+ PGDATABASE: que-test
43
+
44
+ volumes:
45
+ db-data: ~
46
+ ruby-2.7.5-gem-cache: ~
data/lib/que/job.rb CHANGED
@@ -57,22 +57,18 @@ module Que
57
57
 
58
58
  def enqueue(
59
59
  *args,
60
- queue: nil,
61
- priority: nil,
62
- run_at: nil,
63
- job_class: nil,
64
- tags: nil,
60
+ job_options: {},
65
61
  **arg_opts
66
62
  )
67
-
63
+ arg_opts, job_options = _extract_job_options(arg_opts, job_options.dup)
68
64
  args << arg_opts if arg_opts.any?
69
65
 
70
- if tags
71
- if tags.length > MAXIMUM_TAGS_COUNT
72
- raise Que::Error, "Can't enqueue a job with more than #{MAXIMUM_TAGS_COUNT} tags! (passed #{tags.length})"
66
+ if job_options[:tags]
67
+ if job_options[:tags].length > MAXIMUM_TAGS_COUNT
68
+ raise Que::Error, "Can't enqueue a job with more than #{MAXIMUM_TAGS_COUNT} tags! (passed #{job_options[:tags].length})"
73
69
  end
74
70
 
75
- tags.each do |tag|
71
+ job_options[:tags].each do |tag|
76
72
  if tag.length > MAXIMUM_TAG_LENGTH
77
73
  raise Que::Error, "Can't enqueue a job with a tag longer than 100 characters! (\"#{tag}\")"
78
74
  end
@@ -80,13 +76,13 @@ module Que
80
76
  end
81
77
 
82
78
  attrs = {
83
- queue: queue || resolve_que_setting(:queue) || Que.default_queue,
84
- priority: priority || resolve_que_setting(:priority),
85
- run_at: run_at || resolve_que_setting(:run_at),
79
+ queue: job_options[:queue] || resolve_que_setting(:queue) || Que.default_queue,
80
+ priority: job_options[:priority] || resolve_que_setting(:priority),
81
+ run_at: job_options[:run_at] || resolve_que_setting(:run_at),
86
82
  args: Que.serialize_json(args),
87
- data: tags ? Que.serialize_json(tags: tags) : "{}",
83
+ data: job_options[:tags] ? Que.serialize_json(tags: job_options[:tags]) : "{}",
88
84
  job_class: \
89
- job_class || name ||
85
+ job_options[:job_class] || name ||
90
86
  raise(Error, "Can't enqueue an anonymous subclass of Que::Job"),
91
87
  }
92
88
 
@@ -139,6 +135,27 @@ module Que
139
135
  end
140
136
  end
141
137
  end
138
+
139
+ def _extract_job_options(arg_opts, job_options)
140
+ deprecated_job_option_names = []
141
+
142
+ %i[queue priority run_at job_class tags].each do |option_name|
143
+ next unless arg_opts.key?(option_name) && job_options[option_name].nil?
144
+
145
+ job_options[option_name] = arg_opts.delete(option_name)
146
+ deprecated_job_option_names << option_name
147
+ end
148
+
149
+ _log_job_options_deprecation(deprecated_job_option_names)
150
+
151
+ [arg_opts, job_options]
152
+ end
153
+
154
+ def _log_job_options_deprecation(deprecated_job_option_names)
155
+ return unless deprecated_job_option_names.any?
156
+
157
+ warn "Passing job options like (#{deprecated_job_option_names.join(', ')}) to `JobClass.enqueue` as top level keyword args has been deprecated and will be removed in version 2.0. Please wrap job options in an explicit `job_options` keyword arg instead."
158
+ end
142
159
  end
143
160
 
144
161
  # Set up some defaults.
data/lib/que/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Que
4
- VERSION = '1.0.0.beta5'
4
+ VERSION = '1.2.0'
5
5
  end
data/lib/que/worker.rb CHANGED
@@ -137,6 +137,7 @@ module Que
137
137
  error: {
138
138
  class: error.class.to_s,
139
139
  message: error.message,
140
+ backtrace: (error.backtrace || []).join("\n").slice(0, 10000),
140
141
  },
141
142
  )
142
143
 
@@ -164,7 +165,7 @@ module Que
164
165
  Que.execute :set_error, [
165
166
  delay,
166
167
  "#{error.class}: #{error.message}".slice(0, 500),
167
- error.backtrace.join("\n").slice(0, 10000),
168
+ (error.backtrace || []).join("\n").slice(0, 10000),
168
169
  job.fetch(:id),
169
170
  ]
170
171
  end
@@ -0,0 +1,14 @@
1
+ #!/bin/bash
2
+
3
+ set -Eeuo pipefail
4
+
5
+ # For using your own dotfiles within the Docker container
6
+ if [ -f /.docker-rc.d/.docker-bashrc ]; then
7
+ echo "source /.docker-rc.d/.docker-bashrc" >> ~/.bashrc
8
+ fi
9
+
10
+ gem list -i -e bundler -v "$RUBY_BUNDLER_VERSION" >/dev/null || gem install bundler -v "$RUBY_BUNDLER_VERSION"
11
+
12
+ bundle check --dry-run || bundle install
13
+
14
+ exec "${@-bash}"
data/scripts/test ADDED
@@ -0,0 +1,5 @@
1
+ #!/bin/bash
2
+
3
+ set -Eeuo pipefail
4
+
5
+ bundle exec rake spec "$@"
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: que
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0.beta5
4
+ version: 1.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Chris Hanks
8
- autorequire:
8
+ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-12-23 00:00:00.000000000 Z
11
+ date: 2022-02-23 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -34,13 +34,18 @@ extra_rdoc_files: []
34
34
  files:
35
35
  - ".github/workflows/tests.yml"
36
36
  - ".gitignore"
37
- - CHANGELOG.1.0.beta.md
38
37
  - CHANGELOG.md
38
+ - Dockerfile
39
39
  - LICENSE.txt
40
40
  - README.md
41
41
  - Rakefile
42
+ - auto/dev
43
+ - auto/psql
44
+ - auto/test
45
+ - auto/test-postgres-14
42
46
  - bin/command_line_interface.rb
43
47
  - bin/que
48
+ - docker-compose.yml
44
49
  - docs/README.md
45
50
  - lib/que.rb
46
51
  - lib/que/active_job/extensions.rb
@@ -80,11 +85,13 @@ files:
80
85
  - lib/que/version.rb
81
86
  - lib/que/worker.rb
82
87
  - que.gemspec
88
+ - scripts/docker-entrypoint
89
+ - scripts/test
83
90
  homepage: https://github.com/que-rb/que
84
91
  licenses:
85
92
  - MIT
86
93
  metadata: {}
87
- post_install_message:
94
+ post_install_message:
88
95
  rdoc_options: []
89
96
  require_paths:
90
97
  - lib
@@ -95,12 +102,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
95
102
  version: '0'
96
103
  required_rubygems_version: !ruby/object:Gem::Requirement
97
104
  requirements:
98
- - - ">"
105
+ - - ">="
99
106
  - !ruby/object:Gem::Version
100
- version: 1.3.1
107
+ version: '0'
101
108
  requirements: []
102
109
  rubygems_version: 3.1.6
103
- signing_key:
110
+ signing_key:
104
111
  specification_version: 4
105
112
  summary: A PostgreSQL-based Job Queue
106
113
  test_files: []
@@ -1,137 +0,0 @@
1
- ### 1.0.0.beta3 (2018-05-18)
2
-
3
- * Added support for customizing log levels for `job_worked` events (#217).
4
-
5
- * Began logging all `job_errored` events at the `ERROR` log level.
6
-
7
- * Fixed the Railtie when running in test mode (#214).
8
-
9
- * Tweaked the meanings of worker-priorities and worker-count options in the CLI, to better support use cases with low worker counts (#216).
10
-
11
- ### 1.0.0.beta2 (2018-04-13)
12
-
13
- * Fixed an incompatibility that caused the new locker to hang when using Rails in development mode (#213).
14
-
15
- * Fixed a bug with setting the log level via the CLI when the configured logger was based on a callable (#210).
16
-
17
- * Renamed Que.middleware to Que.job_middleware.
18
-
19
- * Added Que.sql_middleware.
20
-
21
- * Officially added support for Ruby 2.5.
22
-
23
- * Internal cleanup and renamings.
24
-
25
- ### 1.0.0.beta (2017-10-25)
26
-
27
- * **A schema upgrade to version 4 will be required for this release.** See [the migration doc](https://github.com/que-rb/que/blob/master/docs/migrating.md) for information if you're upgrading from a previous release.
28
-
29
- * Please note that this migration requires a rewrite of the jobs table, which makes it O(n) with the size of the table. If you have a very large backlog of jobs you may want to schedule downtime for this migration.
30
-
31
- * Que's implementation has been changed from one in which worker threads hold their own PG connections and lock their own jobs to one in which a single thread (and PG connection) locks jobs through LISTEN/NOTIFY and batch polling, and passes jobs along to worker threads. This has many benefits, including:
32
-
33
- * Jobs queued for immediate processing can be actively distributed to workers with LISTEN/NOTIFY, which is more efficient than having workers repeatedly poll for new jobs.
34
-
35
- * When polling is necessary (to pick up jobs that are scheduled for the future or that need to be retried due to errors), jobs can be locked and fetched in batches, rather than one at a time.
36
-
37
- * Individual workers no longer need to monopolize their own (usually idle) connections while working jobs, so Ruby processes will require fewer Postgres connections.
38
-
39
- * PgBouncer or another external connection pool can be used for workers' connections (though not for the connection used to lock and listen for jobs).
40
-
41
- * Other features introduced in this version include:
42
-
43
- * Much better support for all versions of ActiveJob.
44
-
45
- * In particular, you may (optionally) include `Que::ActiveJob::JobExtensions` into `ApplicationJob` to get support for all of Que's job helper methods.
46
-
47
- * Custom middleware that wrap running jobs are now supported.
48
-
49
- * Support for categorizing jobs with tags.
50
-
51
- * Support for configuring a `maximum_retry_count` on individual job classes.
52
-
53
- * Job configuration options are now inheritable, so job class hierarchies are more useful.
54
-
55
- * There are now built-in models for ActiveRecord and Sequel to allow inspecting the queue easily.
56
-
57
- * Jobs that have finished working may optionally be retained in the database indefinitely.
58
-
59
- * To keep a job record, replace the `destroy` calls in your jobs with `finish`. `destroy` will still delete records entirely, for jobs that you don't want to keep.
60
-
61
- * If you don't resolve a job yourself one way or another, Que will still `destroy` the job for you by default.
62
-
63
- * Finished jobs have a timestamp set in the finished_at column.
64
-
65
- * Jobs that have errored too many times will now be marked as expired, and won't be retried again.
66
-
67
- * You can configure a maximum_retry_count in your job classes, to set the threshold at which a job will be marked expired. The default is 15.
68
-
69
- * To manually mark a job as expired (and keep it in the database but not try to run it again) you can call `expire` helper in your job.
70
-
71
- * You can now set job priority thresholds for individual workers, to ensure that there will always be space available for high-priority jobs.
72
-
73
- * `Que.job_states` returns a list of locked jobs and the hostname/pid of the Ruby processes that have locked them.
74
-
75
- * `Que.connection_proc=` has been added, to allow for the easy integration of custom connection pools.
76
-
77
- * In keeping with semantic versioning, the major version is being bumped since the new implementation requires some backwards-incompatible changes. These changes include:
78
-
79
- * Support for MRI Rubies before 2.2 has been dropped.
80
-
81
- * Support for Postgres versions before 9.5 has been dropped (JSONB and upsert support is required).
82
-
83
- * JRuby support has been dropped. It will be reintroduced whenever the jruby-pg gem is production-ready.
84
-
85
- * The `que:work` rake task has been removed. Use the `que` executable instead.
86
-
87
- * Therefore, configuring workers using QUE_* environment variables is no longer supported. Please pass the appropriate options to the `que` executable instead.
88
-
89
- * The `mode` setter has been removed.
90
-
91
- * To run jobs synchronously when they are enqueued (the old `:sync` behavior) you can set `Que.run_synchronously = true`.
92
-
93
- * To start up the worker pool (the old :async behavior) you should use the `que` executable to start up a worker process. There's no longer a supported API for running workers outside of the `que` executable.
94
-
95
- * The following methods are not meaningful under the new implementation and have been removed:
96
-
97
- * The `Que.wake_interval` getter and setter.
98
-
99
- * The `Que.worker_count` getter and setter.
100
-
101
- * `Que.wake!`
102
-
103
- * `Que.wake_all!`
104
-
105
- * Since Que needs a dedicated Postgres connection to manage job locks, running Que through a single PG connection is no longer supported.
106
-
107
- * It's not clear that anyone ever actually did this.
108
-
109
- * `Que.worker_states` has been removed, as the connection that locks a job is no longer the one that the job is using to run. Its functionality has been partially replaced with `Que.job_states`.
110
-
111
- * When using Rails, for simplicity, job attributes and keys in argument hashes are now converted to symbols when retrieved from the database, rather than being converted to instances of HashWithIndifferentAccess.
112
-
113
- * Arguments passed to jobs are now deep-frozen, to prevent unexpected behavior when the args are mutated and the job is reenqueued.
114
-
115
- * Since JSONB is now used to store arguments, the order of argument hashes is no longer maintained.
116
-
117
- * It wouldn't have been a good idea to rely on this anyway.
118
-
119
- * Calling Que.log() directly is no longer supported/recommended.
120
-
121
- * Features marked as deprecated in the final 0.x releases have been removed.
122
-
123
- * Finally, if you've built up your own tooling and customizations around Que, you may need to be aware of some DB schema changes made in the migration to schema version #4.
124
-
125
- * The `job_id` column has been renamed `id` and is now the primary key. This makes it easier to manage the queue using an ActiveRecord model.
126
-
127
- * Finished jobs are now kept in the DB, unless you explicitly call `destroy`. If you want to query the DB for only jobs that haven't finished yet, add a `WHERE finished_at IS NULL` condition to your query, or use the not_finished scope on one of the provided ORM models.
128
-
129
- * There is now an `expired_at` timestamp column, which is set when a job reaches its maximum number of retries and will not be attempted again.
130
-
131
- * Due to popular demand, the default queue name is now "default" rather than an empty string. The migration will move pending jobs under the "" queue to the "default" queue.
132
-
133
- * The `last_error` column has been split in two, to `last_error_message` and `last_error_backtrace`. These two columns are now limited to 500 and 10,000 characters, respectively. The migration will split old error data correctly, and truncate it if necessary.
134
-
135
- * Names for queues and job classes are now limited to 500 characters, which is still far longer than either of these values should reasonably be.
136
-
137
- * There is now a `data` JSONB column which is used to support various ways of organizing jobs (setting tags on them, etc).