spiderfw 0.6.9 → 0.6.10
Sign up to get free protection for your applications and to get access to all the features.
- data/CHANGELOG +5 -1
- data/README.rdoc +121 -0
- data/VERSION +1 -1
- data/apps/app_server/lib/app.rb +2 -2
- data/apps/messenger/messenger.rb +1 -1
- data/apps/servant/lib/commands_processor.rb +1 -1
- data/apps/worker/cmd.rb +6 -2
- data/apps/worker/config/options.rb +2 -0
- data/apps/worker/worker.rb +1 -1
- data/lib/spiderfw/app.rb +1 -0
- data/lib/spiderfw/cmd/commands/app.rb +2 -2
- data/lib/spiderfw/cmd/commands/create.rb +1 -1
- data/lib/spiderfw/controller/mixins/visual.rb +6 -4
- data/lib/spiderfw/create.rb +3 -2
- data/lib/spiderfw/i18n/gettext.rb +1 -0
- data/lib/spiderfw/model/element.rb +1 -1
- data/lib/spiderfw/requires.rb +1 -0
- data/lib/spiderfw/setup/app_manager.rb +49 -25
- data/lib/spiderfw/setup/spider_setup_wizard.rb +1 -1
- data/lib/spiderfw/spider.rb +18 -17
- data/lib/spiderfw/utils/gems.rb +12 -0
- data/lib/spiderfw/utils/logger.rb +8 -0
- data/spider.gemspec +3 -3
- metadata +7 -37
- data/README +0 -4
data/CHANGELOG
CHANGED
@@ -1,7 +1,11 @@
|
|
1
|
+
= 0.6.10
|
2
|
+
== 2 August, 2011
|
3
|
+
* Bugfixes
|
4
|
+
|
1
5
|
= 0.6.9
|
2
6
|
== 28 July, 2011
|
3
7
|
* App installer now uses Git gem instead of Grit for better Windows compatibility
|
4
|
-
* Switched from GetText to FastGettext
|
8
|
+
* Switched from GetText to FastGettext; text domains are now used for each app
|
5
9
|
|
6
10
|
= 0.6.8
|
7
11
|
== 26 July, 2011
|
data/README.rdoc
ADDED
@@ -0,0 +1,121 @@
|
|
1
|
+
== Welcome to Spider
|
2
|
+
|
3
|
+
Spider is a Model-View-Controller application framework, mainly intended for Web applications.
|
4
|
+
|
5
|
+
Its main features are:
|
6
|
+
|
7
|
+
* A model layer that is easy to use, yet flexible enough to adapt to legacy schemas.
|
8
|
+
Models are defined in Ruby: schemas are autogenerated, but can be specified manually when needed.
|
9
|
+
|
10
|
+
Deep loading and deep querying are supported, through a simple pure-Ruby query language:
|
11
|
+
|
12
|
+
planets = Planet.find{ |planet| planet.stars.diameter <= 1500000 }
|
13
|
+
p planets[0].atmosphere.type
|
14
|
+
|
15
|
+
No N+1 queries are performed: deep data is loaded in batches, using windows if needed.
|
16
|
+
|
17
|
+
A Unit of Work is available for complex saves. An Identity Mapper mantains
|
18
|
+
the integrity of an object's tree; but it isn't forced globally, so data replication and
|
19
|
+
translation is still possible.
|
20
|
+
|
21
|
+
* A view layer based on HTML, with reusable widgets that are easy to extend in both their controller and their view.
|
22
|
+
|
23
|
+
Templates are mostly valid HTML, but implement variable substitution, widget instantiation, conditional and iteration logic.
|
24
|
+
|
25
|
+
<div id="my-view">
|
26
|
+
<h1>Hello, { @my_name }</h1>
|
27
|
+
<core:table model="Music" />
|
28
|
+
</div>
|
29
|
+
|
30
|
+
A view can be extended by another one like
|
31
|
+
|
32
|
+
<tpl:extend src="view.shtml">
|
33
|
+
<tpl:append search="#my-view h1">, and welcome to the show.</tpl:append>
|
34
|
+
</tpl:extend>
|
35
|
+
|
36
|
+
Or, a widget can be customized directly when instantiated:
|
37
|
+
|
38
|
+
<div id="my-view">
|
39
|
+
<core:table model="Music">
|
40
|
+
<tpl:after search="tbody td">
|
41
|
+
<a sp:if="element == :title" href="#listen">Listen</a>
|
42
|
+
</tpl:after>
|
43
|
+
</core:table>
|
44
|
+
</div>
|
45
|
+
|
46
|
+
Views are compiled, so they are fast; so are the _overrides_, which always work on the source template.
|
47
|
+
|
48
|
+
No code blocks are available in the views. This forces clear separation of the logic from the presentation and allows
|
49
|
+
the programmatic manipulation of the templates.
|
50
|
+
Still, full Ruby is available on the "scene" variables, so formatting logic can be applied inline.
|
51
|
+
|
52
|
+
* A light controller with flexible redirection, before and after stacks, and an annotation system to expose methods
|
53
|
+
with different content types.
|
54
|
+
|
55
|
+
class MyController < Spider::PageController
|
56
|
+
route 'admin', MyAdminController
|
57
|
+
route /books/(.+)/, :book
|
58
|
+
|
59
|
+
__.html
|
60
|
+
def book(id)
|
61
|
+
@scene.book = Book.new(id)
|
62
|
+
render 'book'
|
63
|
+
end
|
64
|
+
|
65
|
+
end
|
66
|
+
|
67
|
+
Widgets are controllers, too, so they get the same funtionality; still, it's the main controller that manages them,
|
68
|
+
and can easily customize the widgets' behaviour.
|
69
|
+
|
70
|
+
* A JavaScript library (jQuery based) that interacts with the backend.
|
71
|
+
Controllers have their corresponding Javascript objects that allow actions on the server; so do widgets
|
72
|
+
instances, which have a direct line with the same instance on the server.
|
73
|
+
|
74
|
+
$W('my_gallery').reload({place: 'Barcelona'});
|
75
|
+
|
76
|
+
Widgets instances can be rendered individually, allowing a zero-work ajax which can be refined later while
|
77
|
+
still preserving the non-javascript functionality.
|
78
|
+
|
79
|
+
Javascript and CSS dependencies are managed by requiring them in controller and widget templates; they are
|
80
|
+
automatically resolved and compressed when running in production mode.
|
81
|
+
|
82
|
+
In addition, Spider provides:
|
83
|
+
|
84
|
+
* Thread safety (if you play it safe, of course): Spider runs equally well in a single process mode as in a multiple
|
85
|
+
process mode. A threaded single process can be faster and less resource intensive; multiple threaded or single-threaded
|
86
|
+
processes can be pooled.
|
87
|
+
Thread safety has been a requirement since the beginning of the project.
|
88
|
+
|
89
|
+
* CRUD administration but no scaffolding: administration is composed with widgets that can be extended and customized,
|
90
|
+
ensuring that common functionality can be globally modified when needed.
|
91
|
+
|
92
|
+
* Multiple app support: apps are packages which can be distributed and activated as needed.
|
93
|
+
|
94
|
+
* A (writable) YAML-based configuration system with smart defaults and includable sets.
|
95
|
+
|
96
|
+
* No pollution: monkey-patching is kept to the minimum. ActiveSupport is definitely not needed,
|
97
|
+
but is grudgingly accepted if it is loaded.
|
98
|
+
|
99
|
+
* Full internazionalization using gettext and CLDR, even for Javascript files.
|
100
|
+
|
101
|
+
You can dive in by running
|
102
|
+
|
103
|
+
$ spider create home test
|
104
|
+
$ cd test
|
105
|
+
$ spider create app my_app
|
106
|
+
|
107
|
+
Add 'my_app' to the apps in config/config.yml, run
|
108
|
+
|
109
|
+
$ spider webserver start
|
110
|
+
|
111
|
+
and point your browser to http://localhost:8080/my_app/
|
112
|
+
|
113
|
+
Have fun!
|
114
|
+
|
115
|
+
|
116
|
+
|
117
|
+
|
118
|
+
|
119
|
+
|
120
|
+
|
121
|
+
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.6.
|
1
|
+
0.6.10
|
data/apps/app_server/lib/app.rb
CHANGED
data/apps/messenger/messenger.rb
CHANGED
@@ -38,6 +38,7 @@ module Spider
|
|
38
38
|
return false if mutex.locked?
|
39
39
|
model = Spider::Messenger.const_get(self.queues[queue][:model])
|
40
40
|
lock_file = "#{self.lock_file}_#{queue}"
|
41
|
+
now = DateTime.now
|
41
42
|
mutex.synchronize do
|
42
43
|
FileUtils.touch(lock_file)
|
43
44
|
File.open(lock_file, 'r'){ |f| return false unless f.flock File::LOCK_EX | File::LOCK_NB }
|
@@ -46,7 +47,6 @@ module Spider
|
|
46
47
|
if tickets
|
47
48
|
list = model.where(:ticket => tickets)
|
48
49
|
else
|
49
|
-
now = DateTime.now
|
50
50
|
list = model.where{ (sent == nil) & (next_try <= now) }
|
51
51
|
end
|
52
52
|
list.each do |msg|
|
data/apps/worker/cmd.rb
CHANGED
@@ -12,7 +12,11 @@ module Spider; module Worker
|
|
12
12
|
status.set_execution_block do
|
13
13
|
pid = Worker.running?
|
14
14
|
if (pid)
|
15
|
-
|
15
|
+
str = "Worker running (#{pid})"
|
16
|
+
if Worker.pid_file && ::File.exists?(Worker.pid_file)
|
17
|
+
str += " since #{::File::Stat.new(Worker.pid_file).mtime}"
|
18
|
+
end
|
19
|
+
puts str
|
16
20
|
else
|
17
21
|
puts "Worker not running"
|
18
22
|
end
|
@@ -67,4 +71,4 @@ module Spider; module Worker
|
|
67
71
|
|
68
72
|
|
69
73
|
|
70
|
-
end; end
|
74
|
+
end; end
|
@@ -6,5 +6,7 @@ module Spider
|
|
6
6
|
}
|
7
7
|
config_option 'worker.detach', :type => Spider::DataTypes::Bool, :default => true
|
8
8
|
config_option 'worker.jobs_interval', _("Seconds between each jobs run"), :type => Fixnum, :default => 60
|
9
|
+
config_option 'worker.keep_running', _('Keep the worker running after the main process shuts down'), :type => Spider::Bool,
|
10
|
+
:default => lambda{ Object.const_defined?(:PhusionPassenger) ? true : false }
|
9
11
|
|
10
12
|
end
|
data/apps/worker/worker.rb
CHANGED
@@ -33,7 +33,7 @@ module Spider
|
|
33
33
|
end
|
34
34
|
|
35
35
|
def self.app_shutdown
|
36
|
-
return unless @runner || Spider.conf.get('worker.enable')
|
36
|
+
return unless @runner || (Spider.conf.get('worker.enable') && !Spider.conf.get('worker.keep_running'))
|
37
37
|
@mutex.try_lock || return
|
38
38
|
Spider::Logger.info("Shutting down worker in #{Process.pid}")
|
39
39
|
if @runner
|
data/lib/spiderfw/app.rb
CHANGED
@@ -106,7 +106,7 @@ class AppCommand < CmdParse::Command
|
|
106
106
|
install = CmdParse::Command.new( 'install', false )
|
107
107
|
install.short_desc = _("Install an app")
|
108
108
|
install.options = CmdParse::OptionParserWrapper.new do |opt|
|
109
|
-
opt.on("--
|
109
|
+
opt.on("--git", _("Use git for installing apps"), "-g"){ |r| @git = true }
|
110
110
|
opt.on("--no-dependencies", _("Don't install other apps this one depends on"), "-d"){ |d|
|
111
111
|
@no_deps = true
|
112
112
|
}
|
@@ -125,7 +125,7 @@ class AppCommand < CmdParse::Command
|
|
125
125
|
end
|
126
126
|
require 'spiderfw/setup/app_manager'
|
127
127
|
options = {
|
128
|
-
:
|
128
|
+
:git => @git, :all => @all, :no_deps => @no_deps, :no_optional => @no_optional,
|
129
129
|
:no_gems => @no_gems, :no_optional_gems => @no_optional_gems, :no_activate => @no_activate
|
130
130
|
}
|
131
131
|
options[:url] = @server_url if @server_url
|
@@ -24,7 +24,7 @@ class CreateCommand < CmdParse::Command
|
|
24
24
|
@path ||= Dir.pwd+'/apps'
|
25
25
|
names.each do |name|
|
26
26
|
Spider::Create.app(name, @path, @module_name)
|
27
|
-
|
27
|
+
Spider.output "Created app #{name} at #{@path}/#{name}" if ($verbose)
|
28
28
|
end
|
29
29
|
end
|
30
30
|
self.add_command(app, false)
|
@@ -305,10 +305,12 @@ module Spider; module ControllerMixins
|
|
305
305
|
end
|
306
306
|
@rendering_error = true
|
307
307
|
@scene.__is_error_page = true
|
308
|
-
|
309
|
-
|
310
|
-
|
311
|
-
|
308
|
+
unless exc.is_a?(Spider::Controller::NotFound)
|
309
|
+
if Spider.const_defined?(:Messenger) && Spider.conf.get('errors.send_email') && Spider.conf.get('site.tech_admin.email')
|
310
|
+
begin
|
311
|
+
send_error_email(exc)
|
312
|
+
rescue => exc2
|
313
|
+
end
|
312
314
|
end
|
313
315
|
end
|
314
316
|
render_error "#{error_page}", :layout => layout
|
data/lib/spiderfw/create.rb
CHANGED
@@ -44,11 +44,12 @@ module Spider
|
|
44
44
|
repo.add('.gitignore')
|
45
45
|
repo.commit(_("Created repository"))
|
46
46
|
rescue => exc
|
47
|
-
|
47
|
+
Spider.output exc.message, :ERROR
|
48
|
+
Spider.output "Unable to init Git repo, please init manually", :ERROR
|
48
49
|
end
|
49
50
|
Dir.chdir(cwd)
|
50
51
|
rescue LoadError
|
51
|
-
|
52
|
+
Spider.output _("git gem not installed, cannot init repo"), :NOTICE
|
52
53
|
end
|
53
54
|
end
|
54
55
|
|
@@ -3,6 +3,7 @@ require 'locale'
|
|
3
3
|
include FastGettext::Translation
|
4
4
|
FastGettext.add_text_domain('spider', :path => File.join($SPIDER_PATH, 'data', 'locale'))
|
5
5
|
FastGettext.text_domain = 'spider'
|
6
|
+
FastGettext.default_text_domain = 'spider'
|
6
7
|
l = Locale.current[0].to_s
|
7
8
|
l = $1 if l =~ /(\w\w)_+/
|
8
9
|
FastGettext.locale = l
|
@@ -149,7 +149,7 @@ module Spider; module Model
|
|
149
149
|
prev_text_domain = nil
|
150
150
|
if @definer_model && @definer_model != Spider::Model::Managed
|
151
151
|
prev_text_domain = FastGettext.text_domain
|
152
|
-
FastGettext.text_domain = @definer_model.app.short_name
|
152
|
+
FastGettext.text_domain = @definer_model.app.short_name if FastGettext.translation_repositories.key?(@definer_model.app.short_name)
|
153
153
|
end
|
154
154
|
l = self.attributes[:label] ? _(self.attributes[:label]) : Inflector.underscore_to_upcasefirst(@name.to_s)
|
155
155
|
if prev_text_domain
|
data/lib/spiderfw/requires.rb
CHANGED
@@ -24,6 +24,7 @@ require 'spiderfw/utils/ordered_hash'
|
|
24
24
|
require 'spiderfw/utils/memory'
|
25
25
|
require 'spiderfw/utils/secure_random'
|
26
26
|
require 'spiderfw/utils/http_client'
|
27
|
+
require 'spiderfw/utils/gems'
|
27
28
|
require 'spiderfw/config/configuration'
|
28
29
|
require 'spiderfw/config/configurable'
|
29
30
|
require 'spiderfw/model/model'
|
@@ -32,20 +32,20 @@ module Spider
|
|
32
32
|
return if apps.empty?
|
33
33
|
require 'spiderfw/setup/app_server_client'
|
34
34
|
use_git = false
|
35
|
-
|
35
|
+
if options[:git]
|
36
36
|
begin
|
37
37
|
require 'git'
|
38
38
|
use_git = true
|
39
39
|
rescue => exc
|
40
|
-
|
41
|
-
|
40
|
+
Spider.output exc.message, :ERROR
|
41
|
+
Spider.output "git gem not available; install git gem for Git support", :ERROR
|
42
42
|
end
|
43
43
|
end
|
44
44
|
|
45
45
|
existent = []
|
46
46
|
apps.each do |app|
|
47
47
|
if File.exist?("apps/#{app}")
|
48
|
-
|
48
|
+
Spider.output _("%s already exists, skipping") % app
|
49
49
|
existent << app
|
50
50
|
end
|
51
51
|
end
|
@@ -65,8 +65,8 @@ module Spider
|
|
65
65
|
end
|
66
66
|
deps = specs.map{ |s| s.app_id }
|
67
67
|
unless (deps - apps).empty?
|
68
|
-
|
69
|
-
|
68
|
+
Spider.output _("The following apps will be installed as a dependency:")
|
69
|
+
Spider.output((deps - apps).inspect)
|
70
70
|
end
|
71
71
|
i_options = {
|
72
72
|
:use_git => use_git,
|
@@ -96,7 +96,7 @@ module Spider
|
|
96
96
|
require 'git'
|
97
97
|
use_git = true
|
98
98
|
rescue
|
99
|
-
|
99
|
+
Spider.output "git gem not available; install git gem for Git support"
|
100
100
|
end
|
101
101
|
end
|
102
102
|
if options[:all]
|
@@ -105,10 +105,9 @@ module Spider
|
|
105
105
|
apps = home.list_apps
|
106
106
|
end
|
107
107
|
if apps.empty?
|
108
|
-
|
108
|
+
Spider.output _("No app to update")
|
109
109
|
exit
|
110
110
|
end
|
111
|
-
require 'spiderfw/setup/app_manager'
|
112
111
|
specs = []
|
113
112
|
client = Spider::AppServerClient.new(url)
|
114
113
|
if options[:no_deps]
|
@@ -118,8 +117,8 @@ module Spider
|
|
118
117
|
end
|
119
118
|
deps = specs.map{ |s| s.app_id }
|
120
119
|
unless (deps - apps).empty?
|
121
|
-
|
122
|
-
|
120
|
+
Spider.output _("The following apps will be updated as a dependency:")
|
121
|
+
Spider.output((deps - apps).inspect)
|
123
122
|
end
|
124
123
|
Spider::AppManager.update(specs, Dir.pwd, {
|
125
124
|
:use_git => use_git,
|
@@ -129,7 +128,6 @@ module Spider
|
|
129
128
|
end
|
130
129
|
|
131
130
|
def self.install(specs, home_path, options)
|
132
|
-
options[:use_git] = true unless options[:use_git] == false
|
133
131
|
options[:home_path] = home_path
|
134
132
|
specs = [specs] if specs && !specs.is_a?(Array)
|
135
133
|
specs ||= []
|
@@ -147,16 +145,17 @@ module Spider
|
|
147
145
|
def self.git_install(spec, home_path, options={})
|
148
146
|
require 'git'
|
149
147
|
if ::File.exist?("apps/#{spec.id}")
|
150
|
-
|
148
|
+
Spider.output _("%s already installed, skipping") % spec.id
|
151
149
|
return
|
152
150
|
end
|
153
151
|
repo = Git.open(home_path)
|
154
|
-
|
152
|
+
Spider.output _("Fetching %s from %s") % [spec.app_id, spec.git_repo]
|
155
153
|
repo_url = spec.git_repo
|
156
154
|
if options[:ssh_user] && repo_url =~ /ssh:\/\/([^@]+@)?(.+)/
|
157
155
|
repo_url = "ssh://#{options[:ssh_user]}@#{$2}"
|
158
156
|
end
|
159
157
|
|
158
|
+
ENV['GIT_WORK_TREE'] = nil
|
160
159
|
Dir.chdir(home_path) do
|
161
160
|
`git submodule add #{repo_url} apps/#{spec.id}`
|
162
161
|
`git submodule init`
|
@@ -171,7 +170,7 @@ module Spider
|
|
171
170
|
client = AppServerClient.new(spec.app_server)
|
172
171
|
print _("Fetching %s from server... ") % spec.app_id
|
173
172
|
tmp_path = client.fetch_app(spec.app_id)
|
174
|
-
|
173
|
+
Spider.output _("Fetched.")
|
175
174
|
dest = File.join(home_path, "apps/#{spec.app_id}")
|
176
175
|
FileUtils.mkdir_p(dest)
|
177
176
|
open tmp_path, Gem.binary_mode do |io|
|
@@ -188,7 +187,15 @@ module Spider
|
|
188
187
|
end
|
189
188
|
end
|
190
189
|
end
|
191
|
-
|
190
|
+
if File.directory?(File.join(home_path, '.git'))
|
191
|
+
begin
|
192
|
+
require 'git'
|
193
|
+
repo = Git.open(home_path)
|
194
|
+
repo.add("apps/#{spec.id}")
|
195
|
+
repo.commit(_("Added app %s") % spec.id)
|
196
|
+
rescue
|
197
|
+
end
|
198
|
+
end
|
192
199
|
end
|
193
200
|
|
194
201
|
def self.pre_setup(specs, options={})
|
@@ -196,11 +203,25 @@ module Spider
|
|
196
203
|
require 'rubygems/command.rb'
|
197
204
|
require 'rubygems/dependency_installer.rb'
|
198
205
|
unless options[:no_gems]
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
206
|
+
gems = specs.map{ |s| s.gems }
|
207
|
+
# unless options[:no_optional_gems]
|
208
|
+
# gems += specs.map{ |s| s.gems_optional }
|
209
|
+
# end
|
210
|
+
gems = gems.flatten.uniq
|
211
|
+
gems.reject!{ |g| Spider.gem_available?(g) }
|
212
|
+
unless gems.empty?
|
213
|
+
Spider.output _("Installing the following needed gems:")
|
214
|
+
Spider.output gems.inspect
|
215
|
+
inst = Gem::DependencyInstaller.new
|
216
|
+
gems.each do |g|
|
217
|
+
inst.install g
|
218
|
+
end
|
203
219
|
end
|
220
|
+
# unless Spider.gem_available?('bundler')
|
221
|
+
# puts _("Installing bundler gem")
|
222
|
+
# inst = Gem::DependencyInstaller.new
|
223
|
+
# inst.install 'bundler'
|
224
|
+
# end
|
204
225
|
end
|
205
226
|
end
|
206
227
|
|
@@ -213,12 +234,16 @@ module Spider
|
|
213
234
|
end
|
214
235
|
|
215
236
|
def self.update(specs, home_path, options)
|
216
|
-
options[:use_git] = true unless options[:use_git] == false
|
217
237
|
specs = [specs] unless specs.is_a?(Array)
|
218
238
|
pre_setup(specs, options)
|
219
239
|
pre_update(specs, options)
|
220
240
|
specs.each do |spec|
|
241
|
+
use_git = false
|
221
242
|
if spec.git_repo && options[:use_git]
|
243
|
+
app_path = File.join(home_path, "apps/#{spec.id}")
|
244
|
+
use_git = true if File.directory?(File.join(app_path, '.git'))
|
245
|
+
end
|
246
|
+
if use_git
|
222
247
|
git_update(spec, home_path, options)
|
223
248
|
else
|
224
249
|
pack_update(spec, home_path, options)
|
@@ -231,7 +256,7 @@ module Spider
|
|
231
256
|
home_repo = Git.open(home_path)
|
232
257
|
app_path = File.join(home_path, "apps/#{spec.id}")
|
233
258
|
app_repo = Git.open(app_path)
|
234
|
-
|
259
|
+
Spider.output _("Updating %s from %s") % [spec.app_id, spec.git_repo]
|
235
260
|
Dir.chdir(app_path) do
|
236
261
|
app_repo.branch('master').checkout
|
237
262
|
end
|
@@ -240,7 +265,7 @@ module Spider
|
|
240
265
|
`git --git-dir='#{app_path}/.git' pull origin master`
|
241
266
|
end
|
242
267
|
if response =~ /Aborting/
|
243
|
-
|
268
|
+
Spider.output err, :ERROR
|
244
269
|
return
|
245
270
|
end
|
246
271
|
Dir.chdir(app_path) do
|
@@ -265,12 +290,11 @@ module Spider
|
|
265
290
|
pack_install(spec, home_path)
|
266
291
|
FileUtils.rmdir(tmp_app_path)
|
267
292
|
rescue
|
268
|
-
|
293
|
+
Spider.output _("Update of %s failed") % spec.id, :ERROR
|
269
294
|
FileUtils.mv(tmp_app_path, app_path)
|
270
295
|
end
|
271
296
|
end
|
272
297
|
|
273
|
-
|
274
298
|
|
275
299
|
end
|
276
300
|
|
@@ -227,7 +227,7 @@ module Spider
|
|
227
227
|
require 'rubygems'
|
228
228
|
require 'rubygems/command.rb'
|
229
229
|
require 'rubygems/dependency_installer.rb'
|
230
|
-
unless
|
230
|
+
unless Spider.gem_available?('mysql') || Spider.gem_available?('ruby-mysql')
|
231
231
|
if ask? _("Mysql gem is not available. Install?")
|
232
232
|
ok = false
|
233
233
|
while !ok
|
data/lib/spiderfw/spider.rb
CHANGED
@@ -85,7 +85,6 @@ module Spider
|
|
85
85
|
def init_apps
|
86
86
|
@apps.each do |name, mod|
|
87
87
|
if File.directory?(File.join(mod.path, 'po'))
|
88
|
-
Spider.logger.debug("Adding text domain #{mod.short_name}")
|
89
88
|
FastGettext.add_text_domain(mod.short_name, :path => File.join(mod.path, 'data', 'locale'))
|
90
89
|
end
|
91
90
|
end
|
@@ -119,6 +118,7 @@ module Spider
|
|
119
118
|
load_configuration File.join(@root, 'config')
|
120
119
|
Locale.default = Spider.conf.get('i18n.default_locale')
|
121
120
|
setup_env
|
121
|
+
@logger = Spider::Logger
|
122
122
|
@init_base_done = true
|
123
123
|
end
|
124
124
|
|
@@ -179,9 +179,9 @@ module Spider
|
|
179
179
|
@fssm_thread = Thread.new do
|
180
180
|
monitor.run
|
181
181
|
end
|
182
|
-
Spider.
|
182
|
+
Spider.output("Monitoring restart.txt")
|
183
183
|
else
|
184
|
-
Spider.
|
184
|
+
Spider.output("FSSM not installed, unable to monitor restart.txt")
|
185
185
|
end
|
186
186
|
trap('TERM'){ Spider.main_process_shutdown; exit }
|
187
187
|
trap('INT'){ Spider.main_process_shutdown; exit }
|
@@ -274,8 +274,7 @@ module Spider
|
|
274
274
|
|
275
275
|
# Closes any open loggers, and opens new ones based on configured settings.
|
276
276
|
def start_loggers(force=false)
|
277
|
-
return if @
|
278
|
-
@logger ||= Spider::Logger
|
277
|
+
return if @logger_started && !force
|
279
278
|
@logger.close_all
|
280
279
|
@logger.open(STDERR, Spider.conf.get('log.console')) if Spider.conf.get('log.console')
|
281
280
|
begin
|
@@ -300,6 +299,7 @@ module Spider
|
|
300
299
|
end
|
301
300
|
$LOG = @logger
|
302
301
|
Object.const_set(:LOGGER, @logger)
|
302
|
+
@logger_started = true
|
303
303
|
end
|
304
304
|
|
305
305
|
def start_loggers!
|
@@ -482,13 +482,9 @@ module Spider
|
|
482
482
|
begin
|
483
483
|
@configuration.load_yaml(File.join(path, f))
|
484
484
|
rescue ConfigurationException => exc
|
485
|
-
if
|
485
|
+
if exc.type == :yaml
|
486
486
|
err = "Configuration file #{path+f} is not valid YAML"
|
487
|
-
|
488
|
-
@logger.error(err)
|
489
|
-
else
|
490
|
-
puts err
|
491
|
-
end
|
487
|
+
Spider.output(err, :ERROR)
|
492
488
|
else
|
493
489
|
raise
|
494
490
|
end
|
@@ -772,12 +768,8 @@ module Spider
|
|
772
768
|
end
|
773
769
|
rescue LoadError, RuntimeError => exc
|
774
770
|
msg = _('Unable to start debugger. Ensure ruby-debug is installed (or set debugger.start to false).')
|
775
|
-
|
776
|
-
|
777
|
-
Spider.logger.warn(msg)
|
778
|
-
else
|
779
|
-
puts msg
|
780
|
-
end
|
771
|
+
Spider.output(exc.message)
|
772
|
+
Spider.output(msg)
|
781
773
|
end
|
782
774
|
end
|
783
775
|
|
@@ -807,6 +799,15 @@ module Spider
|
|
807
799
|
end
|
808
800
|
end
|
809
801
|
|
802
|
+
def output(str, level=:INFO)
|
803
|
+
if @logger_started
|
804
|
+
@logger.log(level, str)
|
805
|
+
else
|
806
|
+
str = "#{level}: #{str}" if level == :ERROR
|
807
|
+
puts str
|
808
|
+
end
|
809
|
+
end
|
810
|
+
|
810
811
|
end
|
811
812
|
|
812
813
|
end
|
@@ -0,0 +1,12 @@
|
|
1
|
+
module Spider
|
2
|
+
|
3
|
+
# TODO: remove when old RubyGems versions are not a problem anymore
|
4
|
+
def self.gem_available?(name, *requirements)
|
5
|
+
if Gem::Specification.respond_to?(:find_by_name)
|
6
|
+
Gem::Specification.find_by_name(name, *requirements)
|
7
|
+
else
|
8
|
+
Gem.available?(name, requirements.first)
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
end
|
@@ -68,6 +68,10 @@ module Spider
|
|
68
68
|
return false
|
69
69
|
end
|
70
70
|
|
71
|
+
def log(*args, &proc)
|
72
|
+
send_to_loggers(:add, *args, &proc)
|
73
|
+
end
|
74
|
+
|
71
75
|
def debug(*args, &proc)
|
72
76
|
send_to_loggers(:debug, *args, &proc)
|
73
77
|
end
|
@@ -131,6 +135,10 @@ module Spider
|
|
131
135
|
|
132
136
|
end
|
133
137
|
|
138
|
+
def log(*args, &proc)
|
139
|
+
Spider::Logger.add(*args, &proc)
|
140
|
+
end
|
141
|
+
|
134
142
|
def debug(*args, &proc)
|
135
143
|
Spider::Logger.debug(*args, &proc)
|
136
144
|
end
|
data/spider.gemspec
CHANGED
@@ -11,7 +11,7 @@ Gem::Specification.new do |s|
|
|
11
11
|
s.has_rdoc = true
|
12
12
|
s.authors = ["Ivan Pirlik"]
|
13
13
|
s.files = [
|
14
|
-
'README',
|
14
|
+
'README.rdoc',
|
15
15
|
'VERSION',
|
16
16
|
'CHANGELOG',
|
17
17
|
'Rakefile',
|
@@ -23,8 +23,8 @@ Gem::Specification.new do |s|
|
|
23
23
|
+ Dir.glob('lib/**/*.rb') \
|
24
24
|
+ Dir.glob('views/**/*')
|
25
25
|
# s.test_files = []
|
26
|
-
# s.rdoc_options = ["--main", "README.
|
27
|
-
# s.extra_rdoc_files = ["
|
26
|
+
# s.rdoc_options = ["--main", "README.rdoc"]
|
27
|
+
# s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README.rdoc"]
|
28
28
|
s.executables = ['spider']
|
29
29
|
s.default_executable = 'spider'
|
30
30
|
s.add_dependency("cmdparse", ["> 2.0.0"])
|
metadata
CHANGED
@@ -1,13 +1,12 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: spiderfw
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
|
5
|
-
prerelease:
|
4
|
+
prerelease: false
|
6
5
|
segments:
|
7
6
|
- 0
|
8
7
|
- 6
|
9
|
-
-
|
10
|
-
version: 0.6.
|
8
|
+
- 10
|
9
|
+
version: 0.6.10
|
11
10
|
platform: ruby
|
12
11
|
authors:
|
13
12
|
- Ivan Pirlik
|
@@ -15,18 +14,16 @@ autorequire:
|
|
15
14
|
bindir: bin
|
16
15
|
cert_chain: []
|
17
16
|
|
18
|
-
date: 2011-
|
17
|
+
date: 2011-08-02 00:00:00 +02:00
|
19
18
|
default_executable: spider
|
20
19
|
dependencies:
|
21
20
|
- !ruby/object:Gem::Dependency
|
22
21
|
name: cmdparse
|
23
22
|
prerelease: false
|
24
23
|
requirement: &id001 !ruby/object:Gem::Requirement
|
25
|
-
none: false
|
26
24
|
requirements:
|
27
25
|
- - ">"
|
28
26
|
- !ruby/object:Gem::Version
|
29
|
-
hash: 15
|
30
27
|
segments:
|
31
28
|
- 2
|
32
29
|
- 0
|
@@ -38,11 +35,9 @@ dependencies:
|
|
38
35
|
name: fast_gettext
|
39
36
|
prerelease: false
|
40
37
|
requirement: &id002 !ruby/object:Gem::Requirement
|
41
|
-
none: false
|
42
38
|
requirements:
|
43
39
|
- - ">="
|
44
40
|
- !ruby/object:Gem::Version
|
45
|
-
hash: 17
|
46
41
|
segments:
|
47
42
|
- 0
|
48
43
|
- 5
|
@@ -54,11 +49,9 @@ dependencies:
|
|
54
49
|
name: hpricot
|
55
50
|
prerelease: false
|
56
51
|
requirement: &id003 !ruby/object:Gem::Requirement
|
57
|
-
none: false
|
58
52
|
requirements:
|
59
53
|
- - ">"
|
60
54
|
- !ruby/object:Gem::Version
|
61
|
-
hash: 27
|
62
55
|
segments:
|
63
56
|
- 0
|
64
57
|
- 8
|
@@ -69,11 +62,9 @@ dependencies:
|
|
69
62
|
name: json_pure
|
70
63
|
prerelease: false
|
71
64
|
requirement: &id004 !ruby/object:Gem::Requirement
|
72
|
-
none: false
|
73
65
|
requirements:
|
74
66
|
- - ">"
|
75
67
|
- !ruby/object:Gem::Version
|
76
|
-
hash: 13
|
77
68
|
segments:
|
78
69
|
- 1
|
79
70
|
- 1
|
@@ -84,11 +75,9 @@ dependencies:
|
|
84
75
|
name: uuidtools
|
85
76
|
prerelease: false
|
86
77
|
requirement: &id005 !ruby/object:Gem::Requirement
|
87
|
-
none: false
|
88
78
|
requirements:
|
89
79
|
- - ">"
|
90
80
|
- !ruby/object:Gem::Version
|
91
|
-
hash: 1
|
92
81
|
segments:
|
93
82
|
- 2
|
94
83
|
- 1
|
@@ -99,11 +88,9 @@ dependencies:
|
|
99
88
|
name: rufus-scheduler
|
100
89
|
prerelease: false
|
101
90
|
requirement: &id006 !ruby/object:Gem::Requirement
|
102
|
-
none: false
|
103
91
|
requirements:
|
104
92
|
- - ">"
|
105
93
|
- !ruby/object:Gem::Version
|
106
|
-
hash: 15
|
107
94
|
segments:
|
108
95
|
- 1
|
109
96
|
- 0
|
@@ -114,11 +101,9 @@ dependencies:
|
|
114
101
|
name: mime-types
|
115
102
|
prerelease: false
|
116
103
|
requirement: &id007 !ruby/object:Gem::Requirement
|
117
|
-
none: false
|
118
104
|
requirements:
|
119
105
|
- - ">"
|
120
106
|
- !ruby/object:Gem::Version
|
121
|
-
hash: 15
|
122
107
|
segments:
|
123
108
|
- 1
|
124
109
|
- 0
|
@@ -129,11 +114,9 @@ dependencies:
|
|
129
114
|
name: locale
|
130
115
|
prerelease: false
|
131
116
|
requirement: &id008 !ruby/object:Gem::Requirement
|
132
|
-
none: false
|
133
117
|
requirements:
|
134
118
|
- - ">"
|
135
119
|
- !ruby/object:Gem::Version
|
136
|
-
hash: 3
|
137
120
|
segments:
|
138
121
|
- 2
|
139
122
|
- 0
|
@@ -144,11 +127,9 @@ dependencies:
|
|
144
127
|
name: builder
|
145
128
|
prerelease: false
|
146
129
|
requirement: &id009 !ruby/object:Gem::Requirement
|
147
|
-
none: false
|
148
130
|
requirements:
|
149
131
|
- - ">"
|
150
132
|
- !ruby/object:Gem::Version
|
151
|
-
hash: 1
|
152
133
|
segments:
|
153
134
|
- 2
|
154
135
|
- 1
|
@@ -159,11 +140,9 @@ dependencies:
|
|
159
140
|
name: macaddr
|
160
141
|
prerelease: false
|
161
142
|
requirement: &id010 !ruby/object:Gem::Requirement
|
162
|
-
none: false
|
163
143
|
requirements:
|
164
144
|
- - ">="
|
165
145
|
- !ruby/object:Gem::Version
|
166
|
-
hash: 23
|
167
146
|
segments:
|
168
147
|
- 1
|
169
148
|
- 0
|
@@ -175,11 +154,9 @@ dependencies:
|
|
175
154
|
name: bundler
|
176
155
|
prerelease: false
|
177
156
|
requirement: &id011 !ruby/object:Gem::Requirement
|
178
|
-
none: false
|
179
157
|
requirements:
|
180
158
|
- - ">="
|
181
159
|
- !ruby/object:Gem::Version
|
182
|
-
hash: 3
|
183
160
|
segments:
|
184
161
|
- 0
|
185
162
|
version: "0"
|
@@ -189,11 +166,9 @@ dependencies:
|
|
189
166
|
name: rake
|
190
167
|
prerelease: false
|
191
168
|
requirement: &id012 !ruby/object:Gem::Requirement
|
192
|
-
none: false
|
193
169
|
requirements:
|
194
170
|
- - ">"
|
195
171
|
- !ruby/object:Gem::Version
|
196
|
-
hash: 5
|
197
172
|
segments:
|
198
173
|
- 0
|
199
174
|
- 7
|
@@ -205,11 +180,9 @@ dependencies:
|
|
205
180
|
name: ruby-debug
|
206
181
|
prerelease: false
|
207
182
|
requirement: &id013 !ruby/object:Gem::Requirement
|
208
|
-
none: false
|
209
183
|
requirements:
|
210
184
|
- - ">"
|
211
185
|
- !ruby/object:Gem::Version
|
212
|
-
hash: 61
|
213
186
|
segments:
|
214
187
|
- 0
|
215
188
|
- 9
|
@@ -226,7 +199,7 @@ extensions: []
|
|
226
199
|
extra_rdoc_files: []
|
227
200
|
|
228
201
|
files:
|
229
|
-
- README
|
202
|
+
- README.rdoc
|
230
203
|
- VERSION
|
231
204
|
- CHANGELOG
|
232
205
|
- Rakefile
|
@@ -1174,6 +1147,7 @@ files:
|
|
1174
1147
|
- lib/spiderfw/utils/annotations.rb
|
1175
1148
|
- lib/spiderfw/utils/events/event_source.rb
|
1176
1149
|
- lib/spiderfw/utils/fork.rb
|
1150
|
+
- lib/spiderfw/utils/gems.rb
|
1177
1151
|
- lib/spiderfw/utils/hash_comparison.rb
|
1178
1152
|
- lib/spiderfw/utils/http_client.rb
|
1179
1153
|
- lib/spiderfw/utils/inflector.rb
|
@@ -1226,27 +1200,23 @@ rdoc_options: []
|
|
1226
1200
|
require_paths:
|
1227
1201
|
- lib
|
1228
1202
|
required_ruby_version: !ruby/object:Gem::Requirement
|
1229
|
-
none: false
|
1230
1203
|
requirements:
|
1231
1204
|
- - ">="
|
1232
1205
|
- !ruby/object:Gem::Version
|
1233
|
-
hash: 3
|
1234
1206
|
segments:
|
1235
1207
|
- 0
|
1236
1208
|
version: "0"
|
1237
1209
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
1238
|
-
none: false
|
1239
1210
|
requirements:
|
1240
1211
|
- - ">="
|
1241
1212
|
- !ruby/object:Gem::Version
|
1242
|
-
hash: 3
|
1243
1213
|
segments:
|
1244
1214
|
- 0
|
1245
1215
|
version: "0"
|
1246
1216
|
requirements:
|
1247
1217
|
- "Optional dependencies: ripl, ripl-irb, ripl-multi_line, json, openssl, sqlite3, webrick, mongrel, ruby-oci8 >2.0, mysql, yui-compressor, home_run, cldr"
|
1248
1218
|
rubyforge_project:
|
1249
|
-
rubygems_version: 1.
|
1219
|
+
rubygems_version: 1.3.6
|
1250
1220
|
signing_key:
|
1251
1221
|
specification_version: 3
|
1252
1222
|
summary: A (web) framework
|
data/README
DELETED