spiderfw 0.6.9 → 0.6.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG +5 -1
- data/README.rdoc +121 -0
- data/VERSION +1 -1
- data/apps/app_server/lib/app.rb +2 -2
- data/apps/messenger/messenger.rb +1 -1
- data/apps/servant/lib/commands_processor.rb +1 -1
- data/apps/worker/cmd.rb +6 -2
- data/apps/worker/config/options.rb +2 -0
- data/apps/worker/worker.rb +1 -1
- data/lib/spiderfw/app.rb +1 -0
- data/lib/spiderfw/cmd/commands/app.rb +2 -2
- data/lib/spiderfw/cmd/commands/create.rb +1 -1
- data/lib/spiderfw/controller/mixins/visual.rb +6 -4
- data/lib/spiderfw/create.rb +3 -2
- data/lib/spiderfw/i18n/gettext.rb +1 -0
- data/lib/spiderfw/model/element.rb +1 -1
- data/lib/spiderfw/requires.rb +1 -0
- data/lib/spiderfw/setup/app_manager.rb +49 -25
- data/lib/spiderfw/setup/spider_setup_wizard.rb +1 -1
- data/lib/spiderfw/spider.rb +18 -17
- data/lib/spiderfw/utils/gems.rb +12 -0
- data/lib/spiderfw/utils/logger.rb +8 -0
- data/spider.gemspec +3 -3
- metadata +7 -37
- data/README +0 -4
data/CHANGELOG
CHANGED
@@ -1,7 +1,11 @@
|
|
1
|
+
= 0.6.10
|
2
|
+
== 2 August, 2011
|
3
|
+
* Bugfixes
|
4
|
+
|
1
5
|
= 0.6.9
|
2
6
|
== 28 July, 2011
|
3
7
|
* App installer now uses Git gem instead of Grit for better Windows compatibility
|
4
|
-
* Switched from GetText to FastGettext
|
8
|
+
* Switched from GetText to FastGettext; text domains are now used for each app
|
5
9
|
|
6
10
|
= 0.6.8
|
7
11
|
== 26 July, 2011
|
data/README.rdoc
ADDED
@@ -0,0 +1,121 @@
|
|
1
|
+
== Welcome to Spider
|
2
|
+
|
3
|
+
Spider is a Model-View-Controller application framework, mainly intended for Web applications.
|
4
|
+
|
5
|
+
Its main features are:
|
6
|
+
|
7
|
+
* A model layer that is easy to use, yet flexible enough to adapt to legacy schemas.
|
8
|
+
Models are defined in Ruby: schemas are autogenerated, but can be specified manually when needed.
|
9
|
+
|
10
|
+
Deep loading and deep querying are supported, through a simple pure-Ruby query language:
|
11
|
+
|
12
|
+
planets = Planet.find{ |planet| planet.stars.diameter <= 1500000 }
|
13
|
+
p planets[0].atmosphere.type
|
14
|
+
|
15
|
+
No N+1 queries are performed: deep data is loaded in batches, using windows if needed.
|
16
|
+
|
17
|
+
A Unit of Work is available for complex saves. An Identity Mapper mantains
|
18
|
+
the integrity of an object's tree; but it isn't forced globally, so data replication and
|
19
|
+
translation is still possible.
|
20
|
+
|
21
|
+
* A view layer based on HTML, with reusable widgets that are easy to extend in both their controller and their view.
|
22
|
+
|
23
|
+
Templates are mostly valid HTML, but implement variable substitution, widget instantiation, conditional and iteration logic.
|
24
|
+
|
25
|
+
<div id="my-view">
|
26
|
+
<h1>Hello, { @my_name }</h1>
|
27
|
+
<core:table model="Music" />
|
28
|
+
</div>
|
29
|
+
|
30
|
+
A view can be extended by another one like
|
31
|
+
|
32
|
+
<tpl:extend src="view.shtml">
|
33
|
+
<tpl:append search="#my-view h1">, and welcome to the show.</tpl:append>
|
34
|
+
</tpl:extend>
|
35
|
+
|
36
|
+
Or, a widget can be customized directly when instantiated:
|
37
|
+
|
38
|
+
<div id="my-view">
|
39
|
+
<core:table model="Music">
|
40
|
+
<tpl:after search="tbody td">
|
41
|
+
<a sp:if="element == :title" href="#listen">Listen</a>
|
42
|
+
</tpl:after>
|
43
|
+
</core:table>
|
44
|
+
</div>
|
45
|
+
|
46
|
+
Views are compiled, so they are fast; so are the _overrides_, which always work on the source template.
|
47
|
+
|
48
|
+
No code blocks are available in the views. This forces clear separation of the logic from the presentation and allows
|
49
|
+
the programmatic manipulation of the templates.
|
50
|
+
Still, full Ruby is available on the "scene" variables, so formatting logic can be applied inline.
|
51
|
+
|
52
|
+
* A light controller with flexible redirection, before and after stacks, and an annotation system to expose methods
|
53
|
+
with different content types.
|
54
|
+
|
55
|
+
class MyController < Spider::PageController
|
56
|
+
route 'admin', MyAdminController
|
57
|
+
route /books/(.+)/, :book
|
58
|
+
|
59
|
+
__.html
|
60
|
+
def book(id)
|
61
|
+
@scene.book = Book.new(id)
|
62
|
+
render 'book'
|
63
|
+
end
|
64
|
+
|
65
|
+
end
|
66
|
+
|
67
|
+
Widgets are controllers, too, so they get the same funtionality; still, it's the main controller that manages them,
|
68
|
+
and can easily customize the widgets' behaviour.
|
69
|
+
|
70
|
+
* A JavaScript library (jQuery based) that interacts with the backend.
|
71
|
+
Controllers have their corresponding Javascript objects that allow actions on the server; so do widgets
|
72
|
+
instances, which have a direct line with the same instance on the server.
|
73
|
+
|
74
|
+
$W('my_gallery').reload({place: 'Barcelona'});
|
75
|
+
|
76
|
+
Widgets instances can be rendered individually, allowing a zero-work ajax which can be refined later while
|
77
|
+
still preserving the non-javascript functionality.
|
78
|
+
|
79
|
+
Javascript and CSS dependencies are managed by requiring them in controller and widget templates; they are
|
80
|
+
automatically resolved and compressed when running in production mode.
|
81
|
+
|
82
|
+
In addition, Spider provides:
|
83
|
+
|
84
|
+
* Thread safety (if you play it safe, of course): Spider runs equally well in a single process mode as in a multiple
|
85
|
+
process mode. A threaded single process can be faster and less resource intensive; multiple threaded or single-threaded
|
86
|
+
processes can be pooled.
|
87
|
+
Thread safety has been a requirement since the beginning of the project.
|
88
|
+
|
89
|
+
* CRUD administration but no scaffolding: administration is composed with widgets that can be extended and customized,
|
90
|
+
ensuring that common functionality can be globally modified when needed.
|
91
|
+
|
92
|
+
* Multiple app support: apps are packages which can be distributed and activated as needed.
|
93
|
+
|
94
|
+
* A (writable) YAML-based configuration system with smart defaults and includable sets.
|
95
|
+
|
96
|
+
* No pollution: monkey-patching is kept to the minimum. ActiveSupport is definitely not needed,
|
97
|
+
but is grudgingly accepted if it is loaded.
|
98
|
+
|
99
|
+
* Full internazionalization using gettext and CLDR, even for Javascript files.
|
100
|
+
|
101
|
+
You can dive in by running
|
102
|
+
|
103
|
+
$ spider create home test
|
104
|
+
$ cd test
|
105
|
+
$ spider create app my_app
|
106
|
+
|
107
|
+
Add 'my_app' to the apps in config/config.yml, run
|
108
|
+
|
109
|
+
$ spider webserver start
|
110
|
+
|
111
|
+
and point your browser to http://localhost:8080/my_app/
|
112
|
+
|
113
|
+
Have fun!
|
114
|
+
|
115
|
+
|
116
|
+
|
117
|
+
|
118
|
+
|
119
|
+
|
120
|
+
|
121
|
+
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.6.
|
1
|
+
0.6.10
|
data/apps/app_server/lib/app.rb
CHANGED
data/apps/messenger/messenger.rb
CHANGED
@@ -38,6 +38,7 @@ module Spider
|
|
38
38
|
return false if mutex.locked?
|
39
39
|
model = Spider::Messenger.const_get(self.queues[queue][:model])
|
40
40
|
lock_file = "#{self.lock_file}_#{queue}"
|
41
|
+
now = DateTime.now
|
41
42
|
mutex.synchronize do
|
42
43
|
FileUtils.touch(lock_file)
|
43
44
|
File.open(lock_file, 'r'){ |f| return false unless f.flock File::LOCK_EX | File::LOCK_NB }
|
@@ -46,7 +47,6 @@ module Spider
|
|
46
47
|
if tickets
|
47
48
|
list = model.where(:ticket => tickets)
|
48
49
|
else
|
49
|
-
now = DateTime.now
|
50
50
|
list = model.where{ (sent == nil) & (next_try <= now) }
|
51
51
|
end
|
52
52
|
list.each do |msg|
|
data/apps/worker/cmd.rb
CHANGED
@@ -12,7 +12,11 @@ module Spider; module Worker
|
|
12
12
|
status.set_execution_block do
|
13
13
|
pid = Worker.running?
|
14
14
|
if (pid)
|
15
|
-
|
15
|
+
str = "Worker running (#{pid})"
|
16
|
+
if Worker.pid_file && ::File.exists?(Worker.pid_file)
|
17
|
+
str += " since #{::File::Stat.new(Worker.pid_file).mtime}"
|
18
|
+
end
|
19
|
+
puts str
|
16
20
|
else
|
17
21
|
puts "Worker not running"
|
18
22
|
end
|
@@ -67,4 +71,4 @@ module Spider; module Worker
|
|
67
71
|
|
68
72
|
|
69
73
|
|
70
|
-
end; end
|
74
|
+
end; end
|
@@ -6,5 +6,7 @@ module Spider
|
|
6
6
|
}
|
7
7
|
config_option 'worker.detach', :type => Spider::DataTypes::Bool, :default => true
|
8
8
|
config_option 'worker.jobs_interval', _("Seconds between each jobs run"), :type => Fixnum, :default => 60
|
9
|
+
config_option 'worker.keep_running', _('Keep the worker running after the main process shuts down'), :type => Spider::Bool,
|
10
|
+
:default => lambda{ Object.const_defined?(:PhusionPassenger) ? true : false }
|
9
11
|
|
10
12
|
end
|
data/apps/worker/worker.rb
CHANGED
@@ -33,7 +33,7 @@ module Spider
|
|
33
33
|
end
|
34
34
|
|
35
35
|
def self.app_shutdown
|
36
|
-
return unless @runner || Spider.conf.get('worker.enable')
|
36
|
+
return unless @runner || (Spider.conf.get('worker.enable') && !Spider.conf.get('worker.keep_running'))
|
37
37
|
@mutex.try_lock || return
|
38
38
|
Spider::Logger.info("Shutting down worker in #{Process.pid}")
|
39
39
|
if @runner
|
data/lib/spiderfw/app.rb
CHANGED
@@ -106,7 +106,7 @@ class AppCommand < CmdParse::Command
|
|
106
106
|
install = CmdParse::Command.new( 'install', false )
|
107
107
|
install.short_desc = _("Install an app")
|
108
108
|
install.options = CmdParse::OptionParserWrapper.new do |opt|
|
109
|
-
opt.on("--
|
109
|
+
opt.on("--git", _("Use git for installing apps"), "-g"){ |r| @git = true }
|
110
110
|
opt.on("--no-dependencies", _("Don't install other apps this one depends on"), "-d"){ |d|
|
111
111
|
@no_deps = true
|
112
112
|
}
|
@@ -125,7 +125,7 @@ class AppCommand < CmdParse::Command
|
|
125
125
|
end
|
126
126
|
require 'spiderfw/setup/app_manager'
|
127
127
|
options = {
|
128
|
-
:
|
128
|
+
:git => @git, :all => @all, :no_deps => @no_deps, :no_optional => @no_optional,
|
129
129
|
:no_gems => @no_gems, :no_optional_gems => @no_optional_gems, :no_activate => @no_activate
|
130
130
|
}
|
131
131
|
options[:url] = @server_url if @server_url
|
@@ -24,7 +24,7 @@ class CreateCommand < CmdParse::Command
|
|
24
24
|
@path ||= Dir.pwd+'/apps'
|
25
25
|
names.each do |name|
|
26
26
|
Spider::Create.app(name, @path, @module_name)
|
27
|
-
|
27
|
+
Spider.output "Created app #{name} at #{@path}/#{name}" if ($verbose)
|
28
28
|
end
|
29
29
|
end
|
30
30
|
self.add_command(app, false)
|
@@ -305,10 +305,12 @@ module Spider; module ControllerMixins
|
|
305
305
|
end
|
306
306
|
@rendering_error = true
|
307
307
|
@scene.__is_error_page = true
|
308
|
-
|
309
|
-
|
310
|
-
|
311
|
-
|
308
|
+
unless exc.is_a?(Spider::Controller::NotFound)
|
309
|
+
if Spider.const_defined?(:Messenger) && Spider.conf.get('errors.send_email') && Spider.conf.get('site.tech_admin.email')
|
310
|
+
begin
|
311
|
+
send_error_email(exc)
|
312
|
+
rescue => exc2
|
313
|
+
end
|
312
314
|
end
|
313
315
|
end
|
314
316
|
render_error "#{error_page}", :layout => layout
|
data/lib/spiderfw/create.rb
CHANGED
@@ -44,11 +44,12 @@ module Spider
|
|
44
44
|
repo.add('.gitignore')
|
45
45
|
repo.commit(_("Created repository"))
|
46
46
|
rescue => exc
|
47
|
-
|
47
|
+
Spider.output exc.message, :ERROR
|
48
|
+
Spider.output "Unable to init Git repo, please init manually", :ERROR
|
48
49
|
end
|
49
50
|
Dir.chdir(cwd)
|
50
51
|
rescue LoadError
|
51
|
-
|
52
|
+
Spider.output _("git gem not installed, cannot init repo"), :NOTICE
|
52
53
|
end
|
53
54
|
end
|
54
55
|
|
@@ -3,6 +3,7 @@ require 'locale'
|
|
3
3
|
include FastGettext::Translation
|
4
4
|
FastGettext.add_text_domain('spider', :path => File.join($SPIDER_PATH, 'data', 'locale'))
|
5
5
|
FastGettext.text_domain = 'spider'
|
6
|
+
FastGettext.default_text_domain = 'spider'
|
6
7
|
l = Locale.current[0].to_s
|
7
8
|
l = $1 if l =~ /(\w\w)_+/
|
8
9
|
FastGettext.locale = l
|
@@ -149,7 +149,7 @@ module Spider; module Model
|
|
149
149
|
prev_text_domain = nil
|
150
150
|
if @definer_model && @definer_model != Spider::Model::Managed
|
151
151
|
prev_text_domain = FastGettext.text_domain
|
152
|
-
FastGettext.text_domain = @definer_model.app.short_name
|
152
|
+
FastGettext.text_domain = @definer_model.app.short_name if FastGettext.translation_repositories.key?(@definer_model.app.short_name)
|
153
153
|
end
|
154
154
|
l = self.attributes[:label] ? _(self.attributes[:label]) : Inflector.underscore_to_upcasefirst(@name.to_s)
|
155
155
|
if prev_text_domain
|
data/lib/spiderfw/requires.rb
CHANGED
@@ -24,6 +24,7 @@ require 'spiderfw/utils/ordered_hash'
|
|
24
24
|
require 'spiderfw/utils/memory'
|
25
25
|
require 'spiderfw/utils/secure_random'
|
26
26
|
require 'spiderfw/utils/http_client'
|
27
|
+
require 'spiderfw/utils/gems'
|
27
28
|
require 'spiderfw/config/configuration'
|
28
29
|
require 'spiderfw/config/configurable'
|
29
30
|
require 'spiderfw/model/model'
|
@@ -32,20 +32,20 @@ module Spider
|
|
32
32
|
return if apps.empty?
|
33
33
|
require 'spiderfw/setup/app_server_client'
|
34
34
|
use_git = false
|
35
|
-
|
35
|
+
if options[:git]
|
36
36
|
begin
|
37
37
|
require 'git'
|
38
38
|
use_git = true
|
39
39
|
rescue => exc
|
40
|
-
|
41
|
-
|
40
|
+
Spider.output exc.message, :ERROR
|
41
|
+
Spider.output "git gem not available; install git gem for Git support", :ERROR
|
42
42
|
end
|
43
43
|
end
|
44
44
|
|
45
45
|
existent = []
|
46
46
|
apps.each do |app|
|
47
47
|
if File.exist?("apps/#{app}")
|
48
|
-
|
48
|
+
Spider.output _("%s already exists, skipping") % app
|
49
49
|
existent << app
|
50
50
|
end
|
51
51
|
end
|
@@ -65,8 +65,8 @@ module Spider
|
|
65
65
|
end
|
66
66
|
deps = specs.map{ |s| s.app_id }
|
67
67
|
unless (deps - apps).empty?
|
68
|
-
|
69
|
-
|
68
|
+
Spider.output _("The following apps will be installed as a dependency:")
|
69
|
+
Spider.output((deps - apps).inspect)
|
70
70
|
end
|
71
71
|
i_options = {
|
72
72
|
:use_git => use_git,
|
@@ -96,7 +96,7 @@ module Spider
|
|
96
96
|
require 'git'
|
97
97
|
use_git = true
|
98
98
|
rescue
|
99
|
-
|
99
|
+
Spider.output "git gem not available; install git gem for Git support"
|
100
100
|
end
|
101
101
|
end
|
102
102
|
if options[:all]
|
@@ -105,10 +105,9 @@ module Spider
|
|
105
105
|
apps = home.list_apps
|
106
106
|
end
|
107
107
|
if apps.empty?
|
108
|
-
|
108
|
+
Spider.output _("No app to update")
|
109
109
|
exit
|
110
110
|
end
|
111
|
-
require 'spiderfw/setup/app_manager'
|
112
111
|
specs = []
|
113
112
|
client = Spider::AppServerClient.new(url)
|
114
113
|
if options[:no_deps]
|
@@ -118,8 +117,8 @@ module Spider
|
|
118
117
|
end
|
119
118
|
deps = specs.map{ |s| s.app_id }
|
120
119
|
unless (deps - apps).empty?
|
121
|
-
|
122
|
-
|
120
|
+
Spider.output _("The following apps will be updated as a dependency:")
|
121
|
+
Spider.output((deps - apps).inspect)
|
123
122
|
end
|
124
123
|
Spider::AppManager.update(specs, Dir.pwd, {
|
125
124
|
:use_git => use_git,
|
@@ -129,7 +128,6 @@ module Spider
|
|
129
128
|
end
|
130
129
|
|
131
130
|
def self.install(specs, home_path, options)
|
132
|
-
options[:use_git] = true unless options[:use_git] == false
|
133
131
|
options[:home_path] = home_path
|
134
132
|
specs = [specs] if specs && !specs.is_a?(Array)
|
135
133
|
specs ||= []
|
@@ -147,16 +145,17 @@ module Spider
|
|
147
145
|
def self.git_install(spec, home_path, options={})
|
148
146
|
require 'git'
|
149
147
|
if ::File.exist?("apps/#{spec.id}")
|
150
|
-
|
148
|
+
Spider.output _("%s already installed, skipping") % spec.id
|
151
149
|
return
|
152
150
|
end
|
153
151
|
repo = Git.open(home_path)
|
154
|
-
|
152
|
+
Spider.output _("Fetching %s from %s") % [spec.app_id, spec.git_repo]
|
155
153
|
repo_url = spec.git_repo
|
156
154
|
if options[:ssh_user] && repo_url =~ /ssh:\/\/([^@]+@)?(.+)/
|
157
155
|
repo_url = "ssh://#{options[:ssh_user]}@#{$2}"
|
158
156
|
end
|
159
157
|
|
158
|
+
ENV['GIT_WORK_TREE'] = nil
|
160
159
|
Dir.chdir(home_path) do
|
161
160
|
`git submodule add #{repo_url} apps/#{spec.id}`
|
162
161
|
`git submodule init`
|
@@ -171,7 +170,7 @@ module Spider
|
|
171
170
|
client = AppServerClient.new(spec.app_server)
|
172
171
|
print _("Fetching %s from server... ") % spec.app_id
|
173
172
|
tmp_path = client.fetch_app(spec.app_id)
|
174
|
-
|
173
|
+
Spider.output _("Fetched.")
|
175
174
|
dest = File.join(home_path, "apps/#{spec.app_id}")
|
176
175
|
FileUtils.mkdir_p(dest)
|
177
176
|
open tmp_path, Gem.binary_mode do |io|
|
@@ -188,7 +187,15 @@ module Spider
|
|
188
187
|
end
|
189
188
|
end
|
190
189
|
end
|
191
|
-
|
190
|
+
if File.directory?(File.join(home_path, '.git'))
|
191
|
+
begin
|
192
|
+
require 'git'
|
193
|
+
repo = Git.open(home_path)
|
194
|
+
repo.add("apps/#{spec.id}")
|
195
|
+
repo.commit(_("Added app %s") % spec.id)
|
196
|
+
rescue
|
197
|
+
end
|
198
|
+
end
|
192
199
|
end
|
193
200
|
|
194
201
|
def self.pre_setup(specs, options={})
|
@@ -196,11 +203,25 @@ module Spider
|
|
196
203
|
require 'rubygems/command.rb'
|
197
204
|
require 'rubygems/dependency_installer.rb'
|
198
205
|
unless options[:no_gems]
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
206
|
+
gems = specs.map{ |s| s.gems }
|
207
|
+
# unless options[:no_optional_gems]
|
208
|
+
# gems += specs.map{ |s| s.gems_optional }
|
209
|
+
# end
|
210
|
+
gems = gems.flatten.uniq
|
211
|
+
gems.reject!{ |g| Spider.gem_available?(g) }
|
212
|
+
unless gems.empty?
|
213
|
+
Spider.output _("Installing the following needed gems:")
|
214
|
+
Spider.output gems.inspect
|
215
|
+
inst = Gem::DependencyInstaller.new
|
216
|
+
gems.each do |g|
|
217
|
+
inst.install g
|
218
|
+
end
|
203
219
|
end
|
220
|
+
# unless Spider.gem_available?('bundler')
|
221
|
+
# puts _("Installing bundler gem")
|
222
|
+
# inst = Gem::DependencyInstaller.new
|
223
|
+
# inst.install 'bundler'
|
224
|
+
# end
|
204
225
|
end
|
205
226
|
end
|
206
227
|
|
@@ -213,12 +234,16 @@ module Spider
|
|
213
234
|
end
|
214
235
|
|
215
236
|
def self.update(specs, home_path, options)
|
216
|
-
options[:use_git] = true unless options[:use_git] == false
|
217
237
|
specs = [specs] unless specs.is_a?(Array)
|
218
238
|
pre_setup(specs, options)
|
219
239
|
pre_update(specs, options)
|
220
240
|
specs.each do |spec|
|
241
|
+
use_git = false
|
221
242
|
if spec.git_repo && options[:use_git]
|
243
|
+
app_path = File.join(home_path, "apps/#{spec.id}")
|
244
|
+
use_git = true if File.directory?(File.join(app_path, '.git'))
|
245
|
+
end
|
246
|
+
if use_git
|
222
247
|
git_update(spec, home_path, options)
|
223
248
|
else
|
224
249
|
pack_update(spec, home_path, options)
|
@@ -231,7 +256,7 @@ module Spider
|
|
231
256
|
home_repo = Git.open(home_path)
|
232
257
|
app_path = File.join(home_path, "apps/#{spec.id}")
|
233
258
|
app_repo = Git.open(app_path)
|
234
|
-
|
259
|
+
Spider.output _("Updating %s from %s") % [spec.app_id, spec.git_repo]
|
235
260
|
Dir.chdir(app_path) do
|
236
261
|
app_repo.branch('master').checkout
|
237
262
|
end
|
@@ -240,7 +265,7 @@ module Spider
|
|
240
265
|
`git --git-dir='#{app_path}/.git' pull origin master`
|
241
266
|
end
|
242
267
|
if response =~ /Aborting/
|
243
|
-
|
268
|
+
Spider.output err, :ERROR
|
244
269
|
return
|
245
270
|
end
|
246
271
|
Dir.chdir(app_path) do
|
@@ -265,12 +290,11 @@ module Spider
|
|
265
290
|
pack_install(spec, home_path)
|
266
291
|
FileUtils.rmdir(tmp_app_path)
|
267
292
|
rescue
|
268
|
-
|
293
|
+
Spider.output _("Update of %s failed") % spec.id, :ERROR
|
269
294
|
FileUtils.mv(tmp_app_path, app_path)
|
270
295
|
end
|
271
296
|
end
|
272
297
|
|
273
|
-
|
274
298
|
|
275
299
|
end
|
276
300
|
|
@@ -227,7 +227,7 @@ module Spider
|
|
227
227
|
require 'rubygems'
|
228
228
|
require 'rubygems/command.rb'
|
229
229
|
require 'rubygems/dependency_installer.rb'
|
230
|
-
unless
|
230
|
+
unless Spider.gem_available?('mysql') || Spider.gem_available?('ruby-mysql')
|
231
231
|
if ask? _("Mysql gem is not available. Install?")
|
232
232
|
ok = false
|
233
233
|
while !ok
|
data/lib/spiderfw/spider.rb
CHANGED
@@ -85,7 +85,6 @@ module Spider
|
|
85
85
|
def init_apps
|
86
86
|
@apps.each do |name, mod|
|
87
87
|
if File.directory?(File.join(mod.path, 'po'))
|
88
|
-
Spider.logger.debug("Adding text domain #{mod.short_name}")
|
89
88
|
FastGettext.add_text_domain(mod.short_name, :path => File.join(mod.path, 'data', 'locale'))
|
90
89
|
end
|
91
90
|
end
|
@@ -119,6 +118,7 @@ module Spider
|
|
119
118
|
load_configuration File.join(@root, 'config')
|
120
119
|
Locale.default = Spider.conf.get('i18n.default_locale')
|
121
120
|
setup_env
|
121
|
+
@logger = Spider::Logger
|
122
122
|
@init_base_done = true
|
123
123
|
end
|
124
124
|
|
@@ -179,9 +179,9 @@ module Spider
|
|
179
179
|
@fssm_thread = Thread.new do
|
180
180
|
monitor.run
|
181
181
|
end
|
182
|
-
Spider.
|
182
|
+
Spider.output("Monitoring restart.txt")
|
183
183
|
else
|
184
|
-
Spider.
|
184
|
+
Spider.output("FSSM not installed, unable to monitor restart.txt")
|
185
185
|
end
|
186
186
|
trap('TERM'){ Spider.main_process_shutdown; exit }
|
187
187
|
trap('INT'){ Spider.main_process_shutdown; exit }
|
@@ -274,8 +274,7 @@ module Spider
|
|
274
274
|
|
275
275
|
# Closes any open loggers, and opens new ones based on configured settings.
|
276
276
|
def start_loggers(force=false)
|
277
|
-
return if @
|
278
|
-
@logger ||= Spider::Logger
|
277
|
+
return if @logger_started && !force
|
279
278
|
@logger.close_all
|
280
279
|
@logger.open(STDERR, Spider.conf.get('log.console')) if Spider.conf.get('log.console')
|
281
280
|
begin
|
@@ -300,6 +299,7 @@ module Spider
|
|
300
299
|
end
|
301
300
|
$LOG = @logger
|
302
301
|
Object.const_set(:LOGGER, @logger)
|
302
|
+
@logger_started = true
|
303
303
|
end
|
304
304
|
|
305
305
|
def start_loggers!
|
@@ -482,13 +482,9 @@ module Spider
|
|
482
482
|
begin
|
483
483
|
@configuration.load_yaml(File.join(path, f))
|
484
484
|
rescue ConfigurationException => exc
|
485
|
-
if
|
485
|
+
if exc.type == :yaml
|
486
486
|
err = "Configuration file #{path+f} is not valid YAML"
|
487
|
-
|
488
|
-
@logger.error(err)
|
489
|
-
else
|
490
|
-
puts err
|
491
|
-
end
|
487
|
+
Spider.output(err, :ERROR)
|
492
488
|
else
|
493
489
|
raise
|
494
490
|
end
|
@@ -772,12 +768,8 @@ module Spider
|
|
772
768
|
end
|
773
769
|
rescue LoadError, RuntimeError => exc
|
774
770
|
msg = _('Unable to start debugger. Ensure ruby-debug is installed (or set debugger.start to false).')
|
775
|
-
|
776
|
-
|
777
|
-
Spider.logger.warn(msg)
|
778
|
-
else
|
779
|
-
puts msg
|
780
|
-
end
|
771
|
+
Spider.output(exc.message)
|
772
|
+
Spider.output(msg)
|
781
773
|
end
|
782
774
|
end
|
783
775
|
|
@@ -807,6 +799,15 @@ module Spider
|
|
807
799
|
end
|
808
800
|
end
|
809
801
|
|
802
|
+
def output(str, level=:INFO)
|
803
|
+
if @logger_started
|
804
|
+
@logger.log(level, str)
|
805
|
+
else
|
806
|
+
str = "#{level}: #{str}" if level == :ERROR
|
807
|
+
puts str
|
808
|
+
end
|
809
|
+
end
|
810
|
+
|
810
811
|
end
|
811
812
|
|
812
813
|
end
|
@@ -0,0 +1,12 @@
|
|
1
|
+
module Spider
|
2
|
+
|
3
|
+
# TODO: remove when old RubyGems versions are not a problem anymore
|
4
|
+
def self.gem_available?(name, *requirements)
|
5
|
+
if Gem::Specification.respond_to?(:find_by_name)
|
6
|
+
Gem::Specification.find_by_name(name, *requirements)
|
7
|
+
else
|
8
|
+
Gem.available?(name, requirements.first)
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
end
|
@@ -68,6 +68,10 @@ module Spider
|
|
68
68
|
return false
|
69
69
|
end
|
70
70
|
|
71
|
+
def log(*args, &proc)
|
72
|
+
send_to_loggers(:add, *args, &proc)
|
73
|
+
end
|
74
|
+
|
71
75
|
def debug(*args, &proc)
|
72
76
|
send_to_loggers(:debug, *args, &proc)
|
73
77
|
end
|
@@ -131,6 +135,10 @@ module Spider
|
|
131
135
|
|
132
136
|
end
|
133
137
|
|
138
|
+
def log(*args, &proc)
|
139
|
+
Spider::Logger.add(*args, &proc)
|
140
|
+
end
|
141
|
+
|
134
142
|
def debug(*args, &proc)
|
135
143
|
Spider::Logger.debug(*args, &proc)
|
136
144
|
end
|
data/spider.gemspec
CHANGED
@@ -11,7 +11,7 @@ Gem::Specification.new do |s|
|
|
11
11
|
s.has_rdoc = true
|
12
12
|
s.authors = ["Ivan Pirlik"]
|
13
13
|
s.files = [
|
14
|
-
'README',
|
14
|
+
'README.rdoc',
|
15
15
|
'VERSION',
|
16
16
|
'CHANGELOG',
|
17
17
|
'Rakefile',
|
@@ -23,8 +23,8 @@ Gem::Specification.new do |s|
|
|
23
23
|
+ Dir.glob('lib/**/*.rb') \
|
24
24
|
+ Dir.glob('views/**/*')
|
25
25
|
# s.test_files = []
|
26
|
-
# s.rdoc_options = ["--main", "README.
|
27
|
-
# s.extra_rdoc_files = ["
|
26
|
+
# s.rdoc_options = ["--main", "README.rdoc"]
|
27
|
+
# s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README.rdoc"]
|
28
28
|
s.executables = ['spider']
|
29
29
|
s.default_executable = 'spider'
|
30
30
|
s.add_dependency("cmdparse", ["> 2.0.0"])
|
metadata
CHANGED
@@ -1,13 +1,12 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: spiderfw
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
|
5
|
-
prerelease:
|
4
|
+
prerelease: false
|
6
5
|
segments:
|
7
6
|
- 0
|
8
7
|
- 6
|
9
|
-
-
|
10
|
-
version: 0.6.
|
8
|
+
- 10
|
9
|
+
version: 0.6.10
|
11
10
|
platform: ruby
|
12
11
|
authors:
|
13
12
|
- Ivan Pirlik
|
@@ -15,18 +14,16 @@ autorequire:
|
|
15
14
|
bindir: bin
|
16
15
|
cert_chain: []
|
17
16
|
|
18
|
-
date: 2011-
|
17
|
+
date: 2011-08-02 00:00:00 +02:00
|
19
18
|
default_executable: spider
|
20
19
|
dependencies:
|
21
20
|
- !ruby/object:Gem::Dependency
|
22
21
|
name: cmdparse
|
23
22
|
prerelease: false
|
24
23
|
requirement: &id001 !ruby/object:Gem::Requirement
|
25
|
-
none: false
|
26
24
|
requirements:
|
27
25
|
- - ">"
|
28
26
|
- !ruby/object:Gem::Version
|
29
|
-
hash: 15
|
30
27
|
segments:
|
31
28
|
- 2
|
32
29
|
- 0
|
@@ -38,11 +35,9 @@ dependencies:
|
|
38
35
|
name: fast_gettext
|
39
36
|
prerelease: false
|
40
37
|
requirement: &id002 !ruby/object:Gem::Requirement
|
41
|
-
none: false
|
42
38
|
requirements:
|
43
39
|
- - ">="
|
44
40
|
- !ruby/object:Gem::Version
|
45
|
-
hash: 17
|
46
41
|
segments:
|
47
42
|
- 0
|
48
43
|
- 5
|
@@ -54,11 +49,9 @@ dependencies:
|
|
54
49
|
name: hpricot
|
55
50
|
prerelease: false
|
56
51
|
requirement: &id003 !ruby/object:Gem::Requirement
|
57
|
-
none: false
|
58
52
|
requirements:
|
59
53
|
- - ">"
|
60
54
|
- !ruby/object:Gem::Version
|
61
|
-
hash: 27
|
62
55
|
segments:
|
63
56
|
- 0
|
64
57
|
- 8
|
@@ -69,11 +62,9 @@ dependencies:
|
|
69
62
|
name: json_pure
|
70
63
|
prerelease: false
|
71
64
|
requirement: &id004 !ruby/object:Gem::Requirement
|
72
|
-
none: false
|
73
65
|
requirements:
|
74
66
|
- - ">"
|
75
67
|
- !ruby/object:Gem::Version
|
76
|
-
hash: 13
|
77
68
|
segments:
|
78
69
|
- 1
|
79
70
|
- 1
|
@@ -84,11 +75,9 @@ dependencies:
|
|
84
75
|
name: uuidtools
|
85
76
|
prerelease: false
|
86
77
|
requirement: &id005 !ruby/object:Gem::Requirement
|
87
|
-
none: false
|
88
78
|
requirements:
|
89
79
|
- - ">"
|
90
80
|
- !ruby/object:Gem::Version
|
91
|
-
hash: 1
|
92
81
|
segments:
|
93
82
|
- 2
|
94
83
|
- 1
|
@@ -99,11 +88,9 @@ dependencies:
|
|
99
88
|
name: rufus-scheduler
|
100
89
|
prerelease: false
|
101
90
|
requirement: &id006 !ruby/object:Gem::Requirement
|
102
|
-
none: false
|
103
91
|
requirements:
|
104
92
|
- - ">"
|
105
93
|
- !ruby/object:Gem::Version
|
106
|
-
hash: 15
|
107
94
|
segments:
|
108
95
|
- 1
|
109
96
|
- 0
|
@@ -114,11 +101,9 @@ dependencies:
|
|
114
101
|
name: mime-types
|
115
102
|
prerelease: false
|
116
103
|
requirement: &id007 !ruby/object:Gem::Requirement
|
117
|
-
none: false
|
118
104
|
requirements:
|
119
105
|
- - ">"
|
120
106
|
- !ruby/object:Gem::Version
|
121
|
-
hash: 15
|
122
107
|
segments:
|
123
108
|
- 1
|
124
109
|
- 0
|
@@ -129,11 +114,9 @@ dependencies:
|
|
129
114
|
name: locale
|
130
115
|
prerelease: false
|
131
116
|
requirement: &id008 !ruby/object:Gem::Requirement
|
132
|
-
none: false
|
133
117
|
requirements:
|
134
118
|
- - ">"
|
135
119
|
- !ruby/object:Gem::Version
|
136
|
-
hash: 3
|
137
120
|
segments:
|
138
121
|
- 2
|
139
122
|
- 0
|
@@ -144,11 +127,9 @@ dependencies:
|
|
144
127
|
name: builder
|
145
128
|
prerelease: false
|
146
129
|
requirement: &id009 !ruby/object:Gem::Requirement
|
147
|
-
none: false
|
148
130
|
requirements:
|
149
131
|
- - ">"
|
150
132
|
- !ruby/object:Gem::Version
|
151
|
-
hash: 1
|
152
133
|
segments:
|
153
134
|
- 2
|
154
135
|
- 1
|
@@ -159,11 +140,9 @@ dependencies:
|
|
159
140
|
name: macaddr
|
160
141
|
prerelease: false
|
161
142
|
requirement: &id010 !ruby/object:Gem::Requirement
|
162
|
-
none: false
|
163
143
|
requirements:
|
164
144
|
- - ">="
|
165
145
|
- !ruby/object:Gem::Version
|
166
|
-
hash: 23
|
167
146
|
segments:
|
168
147
|
- 1
|
169
148
|
- 0
|
@@ -175,11 +154,9 @@ dependencies:
|
|
175
154
|
name: bundler
|
176
155
|
prerelease: false
|
177
156
|
requirement: &id011 !ruby/object:Gem::Requirement
|
178
|
-
none: false
|
179
157
|
requirements:
|
180
158
|
- - ">="
|
181
159
|
- !ruby/object:Gem::Version
|
182
|
-
hash: 3
|
183
160
|
segments:
|
184
161
|
- 0
|
185
162
|
version: "0"
|
@@ -189,11 +166,9 @@ dependencies:
|
|
189
166
|
name: rake
|
190
167
|
prerelease: false
|
191
168
|
requirement: &id012 !ruby/object:Gem::Requirement
|
192
|
-
none: false
|
193
169
|
requirements:
|
194
170
|
- - ">"
|
195
171
|
- !ruby/object:Gem::Version
|
196
|
-
hash: 5
|
197
172
|
segments:
|
198
173
|
- 0
|
199
174
|
- 7
|
@@ -205,11 +180,9 @@ dependencies:
|
|
205
180
|
name: ruby-debug
|
206
181
|
prerelease: false
|
207
182
|
requirement: &id013 !ruby/object:Gem::Requirement
|
208
|
-
none: false
|
209
183
|
requirements:
|
210
184
|
- - ">"
|
211
185
|
- !ruby/object:Gem::Version
|
212
|
-
hash: 61
|
213
186
|
segments:
|
214
187
|
- 0
|
215
188
|
- 9
|
@@ -226,7 +199,7 @@ extensions: []
|
|
226
199
|
extra_rdoc_files: []
|
227
200
|
|
228
201
|
files:
|
229
|
-
- README
|
202
|
+
- README.rdoc
|
230
203
|
- VERSION
|
231
204
|
- CHANGELOG
|
232
205
|
- Rakefile
|
@@ -1174,6 +1147,7 @@ files:
|
|
1174
1147
|
- lib/spiderfw/utils/annotations.rb
|
1175
1148
|
- lib/spiderfw/utils/events/event_source.rb
|
1176
1149
|
- lib/spiderfw/utils/fork.rb
|
1150
|
+
- lib/spiderfw/utils/gems.rb
|
1177
1151
|
- lib/spiderfw/utils/hash_comparison.rb
|
1178
1152
|
- lib/spiderfw/utils/http_client.rb
|
1179
1153
|
- lib/spiderfw/utils/inflector.rb
|
@@ -1226,27 +1200,23 @@ rdoc_options: []
|
|
1226
1200
|
require_paths:
|
1227
1201
|
- lib
|
1228
1202
|
required_ruby_version: !ruby/object:Gem::Requirement
|
1229
|
-
none: false
|
1230
1203
|
requirements:
|
1231
1204
|
- - ">="
|
1232
1205
|
- !ruby/object:Gem::Version
|
1233
|
-
hash: 3
|
1234
1206
|
segments:
|
1235
1207
|
- 0
|
1236
1208
|
version: "0"
|
1237
1209
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
1238
|
-
none: false
|
1239
1210
|
requirements:
|
1240
1211
|
- - ">="
|
1241
1212
|
- !ruby/object:Gem::Version
|
1242
|
-
hash: 3
|
1243
1213
|
segments:
|
1244
1214
|
- 0
|
1245
1215
|
version: "0"
|
1246
1216
|
requirements:
|
1247
1217
|
- "Optional dependencies: ripl, ripl-irb, ripl-multi_line, json, openssl, sqlite3, webrick, mongrel, ruby-oci8 >2.0, mysql, yui-compressor, home_run, cldr"
|
1248
1218
|
rubyforge_project:
|
1249
|
-
rubygems_version: 1.
|
1219
|
+
rubygems_version: 1.3.6
|
1250
1220
|
signing_key:
|
1251
1221
|
specification_version: 3
|
1252
1222
|
summary: A (web) framework
|
data/README
DELETED