updater 0.9.1 → 0.9.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. data/README.markdown +188 -0
  2. data/Rakefile +6 -4
  3. data/VERSION +1 -1
  4. data/lib/updater/update.rb +1 -1
  5. metadata +48 -25
  6. data/README +0 -7
data/README.markdown ADDED
@@ -0,0 +1,188 @@
1
+ Updater
2
+ =======
3
+
4
+ Updater is a background job queue processor.
5
+ It works a bit like delayed_job or rescue,
6
+ processing jobs in the background to allow user facing processes to stay responsive.
7
+ It also allow jobs to be scheduled and run at a particular time.
8
+ It is intended to work with a number of different ORM layers,
9
+ but at the moment only DataMapper is implemented.
10
+ Native support for MongoDB and ActiveRecord are planed.
11
+ Get it on GemCutter with `gem install updater`.
12
+
13
+ Main Features
14
+ -------------
15
+
16
+ * Handles both immediate and time delayed jobs
17
+ * Intelligent Automatic Scaling
18
+ * Does not poll the database
19
+ * Uses minimal resources when not under load
20
+ * Flexible Job Chaining allows for intelligent error handling and rescheduling
21
+ * Powerful configuration with intelligent defaults
22
+ * Comes with `rake` tasks
23
+
24
+ These feature are as of ver 0.9.1. See the change log for addational features
25
+
26
+ Use Cases
27
+ ---------
28
+
29
+ Web based applications face two restrictions on their functionality
30
+ as it pertains to data processing tasks.
31
+ First, the application will only run code in response to a user request.
32
+ Updater handles the case of actions or events that need to be triggered
33
+ without a request coming in to the web application.
34
+ Second, web applications, particularly those under heavy load,
35
+ need to handle as many request as possible in a given time frame.
36
+ Updater allows data processing and communication tasks
37
+ to happen outside of the request response cycle
38
+ and makes it possible to move these tasks onto dedicated hardware
39
+
40
+ Updater is also useful for circumstances where code needs to be run
41
+ in a time dependent fashion,
42
+ particularly when these events need to be closely controlled by the code.
43
+
44
+ Updater is not a replacement for `cron`.
45
+ Jobs that are regular and repeating can be run
46
+ more consistently and with fewer resources with `cron`.
47
+ Updater should be considered when the application generates
48
+ a large number of one time events,
49
+ and/or the events need to be regularly manipulated bu the application.
50
+
51
+ Updater is also not the optimal solution if the only goal
52
+ is to offload large numbers of immediate tasks.
53
+ For this the author recommends
54
+ [resque](http://github.com/defunkt/resque)
55
+ by Chris Wanstrath.
56
+
57
+ Resque lacks a number of Updater's more powerful features,
58
+ and as of this writing we are not aware of any ability in resque
59
+ to set the time the job is run.
60
+ But rescue does offer much higher potential throughput, and
61
+ a more robust queue structure backed by the Redis key-value store.
62
+
63
+ Using Updater
64
+ =============
65
+
66
+ Initial Installation
67
+ --------------------
68
+
69
+ Updater comes packaged as a gem and is published on GemCutter.
70
+ Get the latest version with `gem install updater`
71
+
72
+ Consepts
73
+ --------
74
+
75
+ Updater is not complex to use but it does, of necessity, have a number of *moving parts*.
76
+ In order to use updater successfully two tasks must be accomplished:
77
+ your application (referred to as the client) must be able to add jobs to the queue,
78
+ and a separate process (the server) must be setup and run
79
+ which will preform the actions specified in those jobs.
80
+
81
+ Jobs are stored in a data store that is shared between client and server.
82
+ Usually this data store will be a table in your database.
83
+ Other data stores are possible, but require significantly configuration.
84
+ Updater is designed to have a minimal impact on the load of the data store,
85
+ and it therefore should be a reasonable solution for most applications.
86
+ (For a discussion of when this is not a reasonable solution see
87
+ [the Rescue Blog Post](http://github.com/blog/542-introducing-resque))
88
+
89
+ Updater is very flexible about what can be run in as a background job.
90
+ The Job will call a single method on either a Class or an instance.
91
+ The method must be public.
92
+ Any arguments can be passed to the method so long as the can be marshaled.
93
+ It is important to keep in mind that the job will be run in a completely separate process.
94
+ Any global state will have to be recreated,
95
+ and data must be persisted in some form in order to be seen by the client.
96
+ For web applications this is usually not an issue.
97
+
98
+ Calling a class method is fairly strait forward,
99
+ but calling a method on in instance take a little more work.
100
+ Instances must be somehow persisted in the client
101
+ then reinstantiated on the worker process.
102
+ The assumption is that this will be don through the ORM and data store.
103
+ Each ORM adapter in Updater lists a defaults methods
104
+ for retrieving particular instances from the data store.
105
+ When an instance is scheduled as the target of a job,
106
+ its class and id will be stored in the `updates` table.
107
+ When the job is run it will first use the it to pull the instance out of the data store,
108
+ then call the appropriate method on that instance.
109
+
110
+ Client Setup
111
+ ------------
112
+
113
+ The client application needs to do two things to be able to effectively schedule jobs:
114
+ 1) require the `updater` library, and
115
+ 2) call the `Updater::Setup.client_setup` method during initialization.
116
+ The `client_setup` method is responsible for establishing interprocess communication with the server,
117
+ and selecting the correct ORM adapter for Updater.
118
+ It does this using the configuration file discussed later in this document.
119
+ This method can take an optional hash that will override the options in the configuration file.
120
+ It can also be passes a `:logger` option which will use the passed in logger instance for updater logging.
121
+
122
+ Scheduling Jobs
123
+ ---------------
124
+
125
+ The core of Updater is placing Jobs on the queue to be run.
126
+ Jobs are scheduled using methods on the `Updater::Update` class.
127
+ Please see the rdoc entry for `Updater::Update#at` for more details
128
+ The following methods can be used to schedule jobs:
129
+
130
+ * `at(time, target, method, args, options)`: Schedules a job to run at a given time
131
+ * `in(delay, target, method, args, options)`: Schedules a job to run after a given interval
132
+ * `immidiate(target, method, args, options)`: Schedules a job to run immidiatly.
133
+
134
+ **target**: is the object (class or instance) that the method will be called on.
135
+
136
+ **method**: the method name as a string or symbol.
137
+ If this is unspesified `:preform` is asumed (a la Resque)
138
+
139
+ **args**: an array of objects that will be passed as arguments to the method.
140
+ Either leave this blank, or set to `[]` to call without arguments.
141
+ All members of the array must be marshalable.
142
+
143
+ **options**: a hash of extra information, details can be found in the Options section.
144
+
145
+ We intend to add a module that can be included into a target class
146
+ that will allow scheduling in the same general manner as delayed_job.
147
+ This addation is planned for version 1.2.
148
+
149
+ The Configuration File
150
+ ----------------------
151
+
152
+ In updater both client and server use a single configuration file.
153
+ The location of this file can be set using the `UPDATE_CONFIG` environment variable.
154
+ When this is not set Updater will instead look for some intelligently chosen defaults.
155
+ These defaults are based on the assumption
156
+ that the client is one of a number of popular web frameworks
157
+ that use the rails directory structure.
158
+ It will look in either the current working directory or a subdirectory called `config`
159
+ (with preference for the latter) for a file called `updater.config`.
160
+ Failing that it will look for a .updater file in the current working directory.
161
+ Rake files should endeavor to set an appropriate working directory
162
+ before invoking the setup tasks.
163
+
164
+ The configuration itself is a ERb interpreted YAML file.
165
+ This is of use in limiting repetition,
166
+ and in changing options based on the environment (test/development/production)
167
+
168
+ Please see the options section for details about the various options in this file.
169
+
170
+ Starting Workers (Server)
171
+ -------------------------
172
+
173
+ In the parlance of background job processing,
174
+ a process that executes jobs is known as a worker.
175
+ The recommended way to start workers is through a rake task.
176
+ First, include `updater/tasks` in your application's Rakefile.
177
+ This will add start, stop and monitor tasks into the `updater` namespace.
178
+ `start` will use the options in your configuration file to start a worker process.
179
+ Likewise, `stop` will shut that process down.
180
+ The monitor task will start an http server
181
+ that you can use to monitor and control the job queue and workers.
182
+ (This feature is not currently implemented)
183
+
184
+ Individual workers are initialized and shutdown by a master process
185
+ which monitors the work load and starts or stops individual workers as needed
186
+ within the limits established in the configuration file.
187
+ You should, therefore, only need to use `start` once.
188
+
data/Rakefile CHANGED
@@ -1,7 +1,9 @@
1
1
  require 'rubygems'
2
2
  require 'rake/gempackagetask'
3
+ gem 'rspec', '=1.3.0'
3
4
  require 'spec/rake/spectask'
4
5
 
6
+
5
7
  VERSION_FILE = File.join(File.dirname(__FILE__), 'VERSION')
6
8
 
7
9
  GEM_NAME = "updater"
@@ -17,19 +19,19 @@ spec = Gem::Specification.new do |s|
17
19
  s.date = File.ctime(VERSION_FILE)
18
20
  s.platform = Gem::Platform::RUBY
19
21
  s.has_rdoc = true
20
- s.extra_rdoc_files = ["README", "LICENSE", "VERSION"]
22
+ s.extra_rdoc_files = ["README.markdown", "LICENSE", "VERSION"]
21
23
  s.summary = SUMMARY
22
24
  s.description = s.summary
23
25
  s.author = AUTHOR
24
26
  s.email = EMAIL
25
27
  s.homepage = HOMEPAGE
26
- s.add_dependency('datamapper', '>= 0.10.2')
27
- s.add_development_dependency('rspec', '>= 1.2.6')
28
+ s.add_development_dependency('datamapper', '>= 0.10.2')
29
+ s.add_development_dependency('rspec', '=1.3.0')
28
30
  s.add_development_dependency('timecop', '>= 0.2.1')
29
31
  s.add_development_dependency('chronic', '>= 0.2.3')
30
32
  s.require_path = 'lib'
31
33
  s.bindir = 'bin'
32
- s.files = %w(LICENSE README Rakefile VERSION) + Dir.glob("{lib,spec,bin}/**/*")
34
+ s.files = %w(LICENSE README.markdown Rakefile VERSION) + Dir.glob("{lib,spec,bin}/**/*")
33
35
 
34
36
  end
35
37
 
data/VERSION CHANGED
@@ -1 +1 @@
1
- 0.9.1
1
+ 0.9.2
@@ -202,7 +202,7 @@ module Updater
202
202
  hash[:finder] = options[:finder] || hash[:finder]
203
203
  hash[:finder_args] = options[:finder_args] || hash[:finder_args]
204
204
 
205
- hash[:method] = method || :process
205
+ hash[:method] = method || :perform
206
206
  hash[:method_args] = args
207
207
 
208
208
  [:name,:failure,:success,:ensure].each do |opt|
metadata CHANGED
@@ -1,7 +1,12 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: updater
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.9.1
4
+ prerelease: false
5
+ segments:
6
+ - 0
7
+ - 9
8
+ - 2
9
+ version: 0.9.2
5
10
  platform: ruby
6
11
  authors:
7
12
  - John F. Miller
@@ -9,49 +14,65 @@ autorequire:
9
14
  bindir: bin
10
15
  cert_chain: []
11
16
 
12
- date: 2010-03-27 00:00:00 -07:00
17
+ date: 2010-05-07 00:00:00 -07:00
13
18
  default_executable:
14
19
  dependencies:
15
20
  - !ruby/object:Gem::Dependency
16
21
  name: datamapper
17
- type: :runtime
18
- version_requirement:
19
- version_requirements: !ruby/object:Gem::Requirement
22
+ prerelease: false
23
+ requirement: &id001 !ruby/object:Gem::Requirement
20
24
  requirements:
21
25
  - - ">="
22
26
  - !ruby/object:Gem::Version
27
+ segments:
28
+ - 0
29
+ - 10
30
+ - 2
23
31
  version: 0.10.2
24
- version:
32
+ type: :development
33
+ version_requirements: *id001
25
34
  - !ruby/object:Gem::Dependency
26
35
  name: rspec
27
- type: :development
28
- version_requirement:
29
- version_requirements: !ruby/object:Gem::Requirement
36
+ prerelease: false
37
+ requirement: &id002 !ruby/object:Gem::Requirement
30
38
  requirements:
31
- - - ">="
39
+ - - "="
32
40
  - !ruby/object:Gem::Version
33
- version: 1.2.6
34
- version:
41
+ segments:
42
+ - 1
43
+ - 3
44
+ - 0
45
+ version: 1.3.0
46
+ type: :development
47
+ version_requirements: *id002
35
48
  - !ruby/object:Gem::Dependency
36
49
  name: timecop
37
- type: :development
38
- version_requirement:
39
- version_requirements: !ruby/object:Gem::Requirement
50
+ prerelease: false
51
+ requirement: &id003 !ruby/object:Gem::Requirement
40
52
  requirements:
41
53
  - - ">="
42
54
  - !ruby/object:Gem::Version
55
+ segments:
56
+ - 0
57
+ - 2
58
+ - 1
43
59
  version: 0.2.1
44
- version:
60
+ type: :development
61
+ version_requirements: *id003
45
62
  - !ruby/object:Gem::Dependency
46
63
  name: chronic
47
- type: :development
48
- version_requirement:
49
- version_requirements: !ruby/object:Gem::Requirement
64
+ prerelease: false
65
+ requirement: &id004 !ruby/object:Gem::Requirement
50
66
  requirements:
51
67
  - - ">="
52
68
  - !ruby/object:Gem::Version
69
+ segments:
70
+ - 0
71
+ - 2
72
+ - 3
53
73
  version: 0.2.3
54
- version:
74
+ type: :development
75
+ version_requirements: *id004
55
76
  description: A Gem for queuing methods for later calling which is ORM Agnostic, and has advanced Error Handling
56
77
  email: emperor@antarestrader.com
57
78
  executables: []
@@ -59,12 +80,12 @@ executables: []
59
80
  extensions: []
60
81
 
61
82
  extra_rdoc_files:
62
- - README
83
+ - README.markdown
63
84
  - LICENSE
64
85
  - VERSION
65
86
  files:
66
87
  - LICENSE
67
- - README
88
+ - README.markdown
68
89
  - Rakefile
69
90
  - VERSION
70
91
  - lib/updater.rb
@@ -106,18 +127,20 @@ required_ruby_version: !ruby/object:Gem::Requirement
106
127
  requirements:
107
128
  - - ">="
108
129
  - !ruby/object:Gem::Version
130
+ segments:
131
+ - 0
109
132
  version: "0"
110
- version:
111
133
  required_rubygems_version: !ruby/object:Gem::Requirement
112
134
  requirements:
113
135
  - - ">="
114
136
  - !ruby/object:Gem::Version
137
+ segments:
138
+ - 0
115
139
  version: "0"
116
- version:
117
140
  requirements: []
118
141
 
119
142
  rubyforge_project:
120
- rubygems_version: 1.3.5
143
+ rubygems_version: 1.3.6
121
144
  signing_key:
122
145
  specification_version: 3
123
146
  summary: A Gem for queuing methods for later calling which is ORM Agnostic, and has advanced Error Handling
data/README DELETED
@@ -1,7 +0,0 @@
1
- updater
2
- =======
3
-
4
- This plugin provides database driven delayed exicution of DataMapper model
5
- classes. It can call class method on any class availible class or it can find
6
- an instance of a class from a database and exicute any method on that instance.
7
- Error handleing is also provided.