quebert 3.0.2 → 3.0.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.ruby-version +1 -1
- data/.travis.yml +1 -1
- data/README.md +42 -21
- data/lib/quebert/controller/beanstalk.rb +2 -2
- data/lib/quebert/version.rb +1 -1
- metadata +3 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 107451b4a0b3a15d3e42285f65c1c53be6700c52
|
4
|
+
data.tar.gz: ce2f9fd96496d6701eadad9abd94acc33a7fa6d4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 06fe749f66920ee7ebb436b72d82c3d0f2f68049cf5dfbba10c9ca3e7ac76eebfb60644cab07634d23fb9d8863b9b5c16880c45b3191f91a6ee187604fc2adea
|
7
|
+
data.tar.gz: 3523257233d8d1f7732be791ae48a6bb7f359ba063231908c4c5957e9411333b48ea32ecd70daec03a0dd915847b087abf4247f21b33c78fadab3575e44e5b84
|
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
2.
|
1
|
+
2.2.5
|
data/.travis.yml
CHANGED
data/README.md
CHANGED
@@ -1,35 +1,39 @@
|
|
1
1
|
# Quebert
|
2
2
|
|
3
|
-
[![Build Status](https://travis-ci.org/polleverywhere/quebert.png?branch=master)](https://travis-ci.org/polleverywhere/quebert) [![Code Climate](https://codeclimate.com/
|
3
|
+
[![Build Status](https://travis-ci.org/polleverywhere/quebert.png?branch=master)](https://travis-ci.org/polleverywhere/quebert) [![Code Climate](https://codeclimate.com/github/polleverywhere/quebert/badges/gpa.svg)](https://codeclimate.com/github/polleverywhere/quebert)
|
4
4
|
|
5
|
-
|
5
|
+
Quebert is a ruby background worker library that works with the very fast and simple [beanstalkd](http://kr.github.io/beanstalkd/) deamon.
|
6
6
|
|
7
|
-
|
7
|
+
## Why Quebert?
|
8
8
|
|
9
|
-
Because it has really low latency. Other Ruby queuing frameworks, like DJ or Resque, have to poll their queue servers periodicly. You could think of it as a "pull" queue. Quebert is a "push" queue. It maintains a persistent connection with beanstalkd and when is enqueud, its instantly pushed to the workers and executed.
|
9
|
+
Because it has really low latency. Other Ruby queuing frameworks, like [DJ](https://github.com/collectiveidea/delayed_job) or [Resque](https://github.com/resque/resque), have to poll their queue servers periodicly. You could think of it as a "pull" queue. Quebert is a "push" queue. It maintains a persistent connection with beanstalkd and when is enqueud, its instantly pushed to the workers and executed.
|
10
10
|
|
11
|
-
|
11
|
+
[Sidekiq](http://sidekiq.org) uses Redis's "push" primitives so it has low latency, but it doesn't support class reloading in a development environment. Sidekiq is also threaded, which means there are no guarantees of reliability when running non-threadsafe code.
|
12
12
|
|
13
|
-
|
13
|
+
[Backburner](https://github.com/nesquena/backburner) is very similar to Quebert. It offers more options for concurrency (threading, forking, etc.) than queubert but lacks pluggable back-ends, which means you'll be stubbing and mocking async calls.
|
14
14
|
|
15
|
-
|
15
|
+
## Who uses it?
|
16
16
|
|
17
|
-
|
18
|
-
* Rails/ActiveRecord integration similar to async_observer
|
19
|
-
* Pluggable exception handling (for Hoptoad integration)
|
20
|
-
* Run workers with pid, log, and config files. These do not daemonize (do it yourself punk!)
|
21
|
-
* Provide custom hooks to be called before, after & around jobs are run
|
17
|
+
Quebert is a serious project. Its used in a production environment at [Poll Everywhere](https://www.polleverywhere.com/) to handle everything from SMS message processing to account downgrades.
|
22
18
|
|
23
|
-
|
19
|
+
## Features
|
20
|
+
|
21
|
+
* **Multiple back-ends** (InProcess, Sync, and Beanstalk)
|
22
|
+
* **Rails/ActiveRecord integration** similar to async_observer
|
23
|
+
* **Pluggable exception handling** (for Hoptoad integration)
|
24
|
+
* **Run workers with pid, log, and config files**. These do not daemonize (do it yourself punk!)
|
25
|
+
* **Custom hooks** that may be called before, after & around jobs are run
|
26
|
+
|
27
|
+
Some features that are currently *missing* that I will soon add include:
|
24
28
|
|
25
29
|
* Rails plugin support (The AR integrations have to be done manually today)
|
26
30
|
* Auto-detecting serializers. Enhanced ClassRegistry to more efficiently look up serializers for objects.
|
27
31
|
|
28
|
-
|
32
|
+
## How to use
|
29
33
|
|
30
34
|
There are two ways to enqueue jobs with Quebert: through the Job itself, provided you set a default back-end for the job, or put it on the backend.
|
31
35
|
|
32
|
-
|
36
|
+
### Jobs
|
33
37
|
|
34
38
|
Quebert includes a Job class so you can implement how you want certain types of Jobs performed.
|
35
39
|
|
@@ -52,7 +56,10 @@ Quebert.backend.put WackyMathWizard.new(1, 2, 3)
|
|
52
56
|
Or drop it in right from the job:
|
53
57
|
|
54
58
|
```ruby
|
59
|
+
# Run job right away!
|
55
60
|
WackyMathWizard.new(4, 5, 6).enqueue
|
61
|
+
# Run a lower priority job in 10 seconds for a max of 120 seconds
|
62
|
+
WackyMathWizard.new(10, 10, 10).enqueue(ttr: 120, priority: 100, delay: 10)
|
56
63
|
```
|
57
64
|
|
58
65
|
Then perform the jobs!
|
@@ -60,9 +67,10 @@ Then perform the jobs!
|
|
60
67
|
```ruby
|
61
68
|
Quebert.backend.reserve.perform # => 6
|
62
69
|
Quebert.backend.reserve.perform # => 15
|
70
|
+
Quebert.backend.reserve.perform # => 30
|
63
71
|
```
|
64
72
|
|
65
|
-
|
73
|
+
### Rails integration
|
66
74
|
|
67
75
|
config/quebert.yml:
|
68
76
|
|
@@ -83,7 +91,7 @@ Quebert.config.from_hash(Rails.application.config.quebert)
|
|
83
91
|
Quebert.config.logger = Rails.logger
|
84
92
|
```
|
85
93
|
|
86
|
-
|
94
|
+
### Job & Worker hooks
|
87
95
|
|
88
96
|
Quebert has support for providing custom hooks to be called before, after & around your jobs are being run.
|
89
97
|
A common example is making sure that any active ActiveRecord database connections are put back on the connection pool after a job is done:
|
@@ -104,7 +112,7 @@ Quebert.config.around_job do |job|
|
|
104
112
|
end
|
105
113
|
```
|
106
114
|
|
107
|
-
|
115
|
+
### Async sender
|
108
116
|
|
109
117
|
Take any ol' class and include the Quebert::AsyncSender.
|
110
118
|
|
@@ -162,13 +170,13 @@ Quebert.backend.reserve.perform # => "waazup Coraline!"
|
|
162
170
|
|
163
171
|
* Only basic data types are included for serialization. Serializers may be customized to include support for different types.
|
164
172
|
|
165
|
-
|
173
|
+
### Backends
|
166
174
|
|
167
175
|
* Beanstalk: Enqueue jobs in a beanstalkd service. The workers run in a separate process. Typically used in production environments.
|
168
176
|
* Sync: Perform jobs immediately upon enqueuing. Typically used in testing environments.
|
169
177
|
* InProcess: Enqueue jobs in an in-memory array. A worker will need to reserve a job to perform.
|
170
178
|
|
171
|
-
|
179
|
+
### Multiple queues
|
172
180
|
|
173
181
|
To start a worker pointed at a non-default queue (e.g., a Quebert "tube"), start the process with `-q`:
|
174
182
|
|
@@ -190,16 +198,29 @@ class FooJob < Quebert::Job
|
|
190
198
|
end
|
191
199
|
```
|
192
200
|
|
193
|
-
|
201
|
+
### Setting job defaults
|
202
|
+
|
203
|
+
A `Quebert::Job` is a Plain Ol' Ruby Object. The defaults of a job, including its `ttr`, `queue_name`, and `delay` may be overridden in a super class as follows:
|
194
204
|
|
195
205
|
```ruby
|
206
|
+
# Assuming you're in Rails or using ActiveSupport
|
196
207
|
class FooJob < Quebert::Job
|
197
208
|
def ttr
|
198
209
|
5.minutes
|
199
210
|
end
|
200
211
|
|
212
|
+
def delay
|
213
|
+
30.seconds
|
214
|
+
end
|
215
|
+
|
216
|
+
def queue_name
|
217
|
+
"long-running-delayed-jobs"
|
218
|
+
end
|
219
|
+
|
201
220
|
def perform(args)
|
202
221
|
# ...
|
203
222
|
end
|
204
223
|
end
|
205
224
|
```
|
225
|
+
|
226
|
+
Take a look at the [`Quebert::Job` class](https://github.com/polleverywhere/quebert/blob/master/lib/quebert/job.rb) code for more details on methods you may ovveride.
|
@@ -22,7 +22,7 @@ module Quebert
|
|
22
22
|
end
|
23
23
|
|
24
24
|
def perform
|
25
|
-
logger.error(job) { "Performing with args #{job.args.inspect}" }
|
25
|
+
logger.error(job) { "Performing #{job.class.name} with args #{job.args.inspect}" }
|
26
26
|
logger.error(job) { "Beanstalk Job Stats: #{beanstalk_job.stats.inspect}" }
|
27
27
|
|
28
28
|
result = false
|
@@ -31,7 +31,7 @@ module Quebert
|
|
31
31
|
beanstalk_job.delete
|
32
32
|
end
|
33
33
|
|
34
|
-
logger.
|
34
|
+
logger.info(job) { "Completed in #{(time*1000*1000).to_i/1000.to_f} ms\n" }
|
35
35
|
result
|
36
36
|
rescue Job::Delete
|
37
37
|
logger.info(job) { "Deleting job" }
|
data/lib/quebert/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: quebert
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.0.
|
4
|
+
version: 3.0.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Brad Gessler
|
@@ -10,7 +10,7 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2016-11-29 00:00:00.000000000 Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
16
|
name: json
|
@@ -131,7 +131,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
131
131
|
version: '0'
|
132
132
|
requirements: []
|
133
133
|
rubyforge_project: quebert
|
134
|
-
rubygems_version: 2.
|
134
|
+
rubygems_version: 2.4.5.1
|
135
135
|
signing_key:
|
136
136
|
specification_version: 4
|
137
137
|
summary: A worker queue framework built around beanstalkd
|
@@ -149,4 +149,3 @@ test_files:
|
|
149
149
|
- spec/support/jobs.rb
|
150
150
|
- spec/support_spec.rb
|
151
151
|
- spec/worker_spec.rb
|
152
|
-
has_rdoc:
|