rails_build 1.0.0 → 1.2.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (7) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +8 -5
  3. data/bin/rails_build +730 -642
  4. data/lib/rails_build.rb +2 -0
  5. data/lib/rails_build/engine.rb +2 -2
  6. data/lib/rails_build/version.rb +1 -1
  7. metadata +46 -9
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: cd82bac7610eed4089c5be9667a685dea9ebd327
4
- data.tar.gz: 99110d59847285fad099e0769f333f49aded632f
3
+ metadata.gz: 6e8b28d514b5404bbcadaca8e7d8ade5021265e8
4
+ data.tar.gz: a36bf4c19682e86162b5579122ee776fdd9b20bc
5
5
  SHA512:
6
- metadata.gz: 4c4ddd3929c28ada377db4c57811e3934aaeeaae6e30e6fa920f158e5cd7c9bce2276111fbda6f15b23685658fe88cb107593d981e8bf4d0dd9788036eb9f8be
7
- data.tar.gz: e1203289ae526130f34645705cefc37edf2409065671bd591991084755eabe7e0aeb1a37838dc9d94bde6a9793d370163f24cce9ccac9843cf443322c2a55642
6
+ metadata.gz: be11ffbdd737215266b85ca7de68a873ecc23b1b59e9dc668045f82bdffd769862c36c0e0cd29d16e3fe57ce564abb08a7c53303f66edcd7d3a785a9c9cb54ec
7
+ data.tar.gz: 2e2a593592f58735a55989dbd6af9849b0b74f9b09870708b62a05bbed33b18a1d1d5523735dc4adbe369acf1d00a7492a596295802250a922539776b1ddd2a2
data/README.md CHANGED
@@ -8,7 +8,7 @@ built as a Rails 5 engine.
8
8
  ## How It Works
9
9
 
10
10
  RailsBuild bundles all your assets, public directory, and configured urls into
11
- a static site suitable for deployment to Netlify, Amazon S3, or your favorite
11
+ a static site suitable for deployment to [Netlify](https://www.netlify.com/), Amazon S3, or your favorite
12
12
  static website hosting solution. It does this by:
13
13
 
14
14
  - Booting your application in *production* mode
@@ -92,14 +92,17 @@ like './about' vs. '../about'
92
92
 
93
93
  ## Optimization and Notes
94
94
 
95
-
96
95
  RailsBuild is fast. Very fast. [DOJO4](http://dojo4.com) has seen optimized [Middleman](https://middlemanapp.com/) builds of > 30 minutes dropped to *60 seconds* by simply making basic use of Rails' built-in caching facilites.
97
96
 
97
+ You app has to run in production mode to build! Don't forget to setup
98
+ secrets, or anything else generally required in production mode.
99
+
98
100
  When trying to squeeze out performance just remember that RailsBuild runs in
99
101
  production mode and, therefore, making a build go fast follows the *exact same
100
102
  rules* as making anything other Rails' application fast. The first place to
101
103
  reach is typically fragment caching of partials used in your app.
102
104
 
105
+
103
106
  Finally, don't forget about *./config/initializers/assets.rb* - RailsBuild
104
107
  doesn't do anything special to the asset pipeline and only those assets
105
108
  normally built when
@@ -157,14 +160,14 @@ After installation and configuration simply run
157
160
 
158
161
 
159
162
 
160
- ## Netlfiy
163
+ ## Netlify
161
164
 
162
- We love Netlify at [DOJO4](http://dojo4.com). RailsBuild works with netlify
165
+ We love [Netlify](https://www.netlify.com/) at [DOJO4](http://dojo4.com). RailsBuild works with netlify
163
166
  out of the box and simply requires
164
167
 
165
168
  ```yaml
166
169
 
167
- build_command : rails_build
170
+ build_command : ./bin/rails_build
168
171
 
169
172
  build_directory: build
170
173
 
data/bin/rails_build CHANGED
@@ -10,13 +10,13 @@ END{
10
10
 
11
11
 
12
12
  module RailsBuild
13
- class CLI
14
- def CLI.usage
15
- <<-__
16
- NAME
17
- rails_build
13
+ class CLI
14
+ def CLI.usage
15
+ <<-__
16
+ NAME
17
+ rails_build
18
18
 
19
- SYNOPSIS
19
+ SYNOPSIS
20
20
  a small, simple, bullet proof, and very fast static site generator built on rails 5
21
21
 
22
22
  USAGE
@@ -27,168 +27,171 @@ module RailsBuild
27
27
  --rails_root,--r : specifiy the RAILS_ROOT, default=./
28
28
  --parallel,--p : how many requests to make in parallel, default=n_cpus/2
29
29
  --env,--e : speciify the RAILS_ENV, default=production
30
- --server, -s : provide the url of the build server, do *not* start separate one
30
+ --url, -u : provide the url of the build server, do *not* start separate one
31
+ --server, -s : passenger | puma, default=passenger
31
32
  --log, -l : specify the logfile, default=STDERR
32
33
  --verbose, -v : turn on verbose logging
33
- __
34
- end
34
+ __
35
+ end
35
36
 
36
- def CLI.opts
37
- GetoptLong.new(
38
- [ '--help' , '-h' , GetoptLong::NO_ARGUMENT ] ,
39
- [ '--server' , '-s' , GetoptLong::REQUIRED_ARGUMENT ] ,
40
- [ '--parallel' , '-p' , GetoptLong::REQUIRED_ARGUMENT ] ,
41
- [ '--rails_root' , '-r' , GetoptLong::REQUIRED_ARGUMENT ] ,
42
- [ '--env' , '-e' , GetoptLong::REQUIRED_ARGUMENT ] ,
43
- [ '--log' , '-l' , GetoptLong::REQUIRED_ARGUMENT ] ,
44
- [ '--verbose' , '-v' , GetoptLong::NO_ARGUMENT ] ,
45
- )
46
- end
37
+ def CLI.opts
38
+ GetoptLong.new(
39
+ [ '--help' , '-h' , GetoptLong::NO_ARGUMENT ] ,
40
+ [ '--url' , '-u' , GetoptLong::REQUIRED_ARGUMENT ] ,
41
+ [ '--server' , '-s' , GetoptLong::REQUIRED_ARGUMENT ] ,
42
+ [ '--parallel' , '-p' , GetoptLong::REQUIRED_ARGUMENT ] ,
43
+ [ '--rails_root' , '-r' , GetoptLong::REQUIRED_ARGUMENT ] ,
44
+ [ '--env' , '-e' , GetoptLong::REQUIRED_ARGUMENT ] ,
45
+ [ '--log' , '-l' , GetoptLong::REQUIRED_ARGUMENT ] ,
46
+ [ '--verbose' , '-v' , GetoptLong::NO_ARGUMENT ] ,
47
+ )
48
+ end
47
49
 
48
- def build!
49
- prepare!
50
+ def build!
51
+ prepare!
50
52
 
51
- mkdir!
53
+ mkdir!
52
54
 
53
- start_server! unless server
55
+ start_server! unless url
54
56
 
55
- clear_cache!
57
+ clear_cache!
56
58
 
57
- extract_urls! url_for('/rails_build/configuration.json')
59
+ extract_urls! url_for('/rails_build/configuration.json')
58
60
 
59
- precompile_assets!
61
+ precompile_assets!
60
62
 
61
- rsync_public!
63
+ rsync_public!
62
64
 
63
- parallel_build!
65
+ parallel_build!
64
66
 
65
- finalize!
66
- end
67
+ finalize!
68
+ end
67
69
 
68
- #
69
- def CLI.build!(*args, &block)
70
- new(*args, &block).build!
71
- end
70
+ #
71
+ def CLI.build!(*args, &block)
72
+ new(*args, &block).build!
73
+ end
72
74
 
73
- #
74
- attr_accessor :rails_root
75
- attr_accessor :server
76
- attr_accessor :directory
77
- attr_accessor :uuid
78
- attr_accessor :id
79
- attr_accessor :env
80
- attr_accessor :puma
81
- attr_accessor :passenger
82
- attr_accessor :url
75
+ #
76
+ attr_accessor :rails_root
77
+ attr_accessor :url
78
+ attr_accessor :server
79
+ attr_accessor :directory
80
+ attr_accessor :uuid
81
+ attr_accessor :id
82
+ attr_accessor :env
83
+ attr_accessor :parallel
83
84
 
84
- #
85
- def initialize(*args, &block)
85
+ #
86
+ def initialize(*args, &block)
86
87
  setup!
87
88
 
88
- @args = parse_args!
89
+ @args = parse_args!
89
90
 
90
- @opts = parse_opts!
91
+ @opts = parse_opts!
91
92
 
92
- if @opts[:help]
93
- usage!
94
- exit(42)
95
- end
96
- end
93
+ if @opts[:help]
94
+ usage!
95
+ exit(42)
96
+ end
97
+ end
97
98
 
98
99
  #
99
100
  def setup!
100
- #
101
- STDOUT.sync = true
102
- STDERR.sync = true
101
+ #
102
+ STDOUT.sync = true
103
+ STDERR.sync = true
103
104
 
104
- #
105
- ENV['SPRING_DISABLE'] = 'true'
106
- ENV['DISABLE_SPRING'] = 'true'
105
+ #
106
+ ENV['SPRING_DISABLE'] = 'true'
107
+ ENV['DISABLE_SPRING'] = 'true'
107
108
 
108
- #
109
- %w[
110
- fileutils pathname thread socket timeout time uri etc open-uri securerandom logger getoptlong rubygems json
111
- ].each do |stdlib|
112
- require(stdlib)
113
- end
109
+ #
110
+ %w[
111
+ fileutils pathname thread socket timeout time uri etc open-uri securerandom logger getoptlong rubygems json
112
+ ].each do |stdlib|
113
+ require(stdlib)
114
+ end
114
115
  end
115
116
 
116
- #
117
- def prepare!
117
+ #
118
+ def prepare!
118
119
  #
119
- @rails_root = find_rails_root!(@args[0] || '.')
120
+ @rails_root = find_rails_root!(@args[0] || '.')
120
121
 
121
122
  #
122
- Dir.chdir(@rails_root)
123
+ Dir.chdir(@rails_root)
123
124
 
124
125
  #
125
- if File.exists?('./Gemfile')
126
- require 'bundler/setup'
127
- Bundler.setup(:require => false)
128
- end
129
-
130
- #
131
- %w[
132
- threadify persistent_http phusion_passenger
133
- ].each do |gem|
126
+ if File.exists?('./Gemfile')
127
+ require 'bundler/setup'
128
+ Bundler.setup(:require => false)
129
+ end
130
+
131
+ #
132
+ %w[
133
+ threadify persistent_http
134
+ ].each do |gem|
134
135
  begin
135
136
  require(gem)
136
137
  rescue LoadError
137
- abort "add gem 'rails_build' to your Gemfile"
138
+ abort "add gem '#{ gem }' to your Gemfile"
138
139
  end
139
- end
140
+ end
140
141
 
141
142
  #
142
- begin
143
- require('pry')
144
- rescue LoadError
145
- nil
146
- end
147
-
148
- #
149
- @logger = Logger.new(@opts[:log] || STDERR)
150
-
151
- @uuid = ENV['RAILS_BUILD']
152
- @time = ENV['RAILS_BUILD_TIME']
153
- @server = @opts[:server] || ENV['RAILS_BUILD_SERVER']
154
- @env = @opts[:env] || ENV['RAILS_BUILD_ENV'] || ENV['RAILS_ENV']
155
- @parallel = @opts[:parallel] || ENV['RAILS_BUILD_PARALLEL']
156
- @verbose = @opts[:verbose] || ENV['RAILS_BUILD_VERBOSE']
157
-
158
- @uuid ||= SecureRandom.uuid
159
- @time ||= Time.now.utc
160
- @env ||= 'production'
161
- @parallel ||= (Etc.nprocessors/2)
162
-
163
- unless @time.is_a?(Time)
164
- @time = Time.parse(@time.to_s).utc
165
- end
166
-
167
- @parallel = @parallel.to_i
143
+ begin
144
+ require('pry')
145
+ rescue LoadError
146
+ nil
147
+ end
168
148
 
149
+ #
150
+ @logger = Logger.new(@opts[:log] || STDERR)
151
+
152
+ @uuid = ENV['RAILS_BUILD']
153
+ @time = ENV['RAILS_BUILD_TIME']
154
+ @url = @opts[:url] || ENV['RAILS_BUILD_URL']
155
+ @server = @opts[:server] || ENV['RAILS_BUILD_SERVER']
156
+ @env = @opts[:env] || ENV['RAILS_BUILD_ENV'] || ENV['RAILS_ENV']
157
+ @parallel = @opts[:parallel] || ENV['RAILS_BUILD_PARALLEL']
158
+ @verbose = @opts[:verbose] || ENV['RAILS_BUILD_VERBOSE']
159
+
160
+ @uuid ||= SecureRandom.uuid
161
+ @time ||= Time.now.utc
162
+ @server ||= 'passenger'
163
+ @env ||= 'production'
164
+ @parallel ||= (Etc.nprocessors/2)
165
+
166
+ unless @time.is_a?(Time)
167
+ @time = Time.parse(@time.to_s).utc
168
+ end
169
169
 
170
- @puma = 'bundle exec puma'
171
- @pumactl = 'bundle exec pumactl'
170
+ @parallel = @parallel.to_i
172
171
 
173
- @passenger = 'bundle exec passenger'
172
+ if ENV['RAILS_BUILD_DIRECTORY']
173
+ @build_directory = File.expand_path(ENV['RAILS_BUILD_DIRECTORY'])
174
+ else
175
+ @build_directory = File.join(@rails_root, 'builds')
176
+ end
174
177
 
175
- if ENV['RAILS_BUILD_DIRECTORY']
176
- @build_directory = File.expand_path(ENV['RAILS_BUILD_DIRECTORY'])
177
- else
178
- @build_directory = File.join(@rails_root, 'builds')
179
- end
178
+ @directory = File.join(@build_directory, @uuid)
180
179
 
181
- @directory = File.join(@build_directory, @uuid)
180
+ ENV['RAILS_ENV'] = @env
181
+ ENV['RAILS_BUILD'] = @uuid
182
+ ENV['RAILS_BUILD_ENV'] = @env
183
+ ENV['RAILS_BUILD_TIME'] = @time.httpdate
182
184
 
183
- ENV['RAILS_ENV'] = @env
184
- ENV['RAILS_BUILD'] = @uuid
185
- ENV['RAILS_BUILD_ENV'] = @env
186
- ENV['RAILS_BUILD_TIME'] = @time.httpdate
185
+ @urls = []
187
186
 
188
- @urls = []
187
+ @started_at = Time.now
189
188
 
190
- @started_at = Time.now
191
- end
189
+ unless(( server = Server.for(@server, self) ))
190
+ abort "no server found for server=#{ @server.inspect }"
191
+ else
192
+ @server = server
193
+ end
194
+ end
192
195
 
193
196
  #
194
197
  def find_rails_root!(path)
@@ -208,523 +211,608 @@ module RailsBuild
208
211
  abort("could not find a rails_root in or above #{ path }!?")
209
212
  end
210
213
 
211
- #
212
- def parse_args!
213
- @args = ARGV.map{|arg| "#{ arg }"}
214
- end
215
-
216
- #
217
- def parse_opts!
218
- @opts = Hash.new
219
-
220
- CLI.opts.each do |opt, arg|
221
- key, val = opt.split('--').last, arg
222
- @opts[key] = (val == '' ? true : val)
223
- end
224
-
225
- @opts
226
- end
227
-
228
- #
229
- def usage!
230
- lines = CLI.usage.strip.split(/\n/)
231
- n = lines[1].to_s.scan(/^\s+/).size
232
- indent = ' ' * n
233
- re = /^#{ Regexp.escape(indent) }/
234
- usage = lines.map{|line| line.gsub(re, '')}.join("\n")
235
- STDERR.puts(usage)
236
- end
237
-
238
- #
239
- def mkdir!
240
- FileUtils.rm_rf(@directory)
241
- FileUtils.mkdir_p(@directory)
242
- log(:info, "directory: #{ @directory }")
243
- end
244
-
245
- #
246
- def start_server!
247
- @server =
248
- nil
249
-
250
- @port =
251
- nil
252
-
253
- ports =
254
- (2000 .. 9000).to_a
255
-
256
- # TODO - get puma working...
257
- =begin
258
- pidfile = File.join(@directory, "puma.pid")
259
- statefile = File.join(@directory, "puma-state.txt")
260
- configfile = File.join(@directory, "puma-config.rb")
261
- FileUtils.touch(configfile)
262
- =end
263
-
264
- start_server, stop_server = nil
265
-
266
- ports.each do |port|
267
- next unless port_open?(port)
268
-
269
- # TODO - get puma working...
270
- =begin
271
- start_server =
272
- "puma --port=#{ port } --environment=#{ @env } --pidfile=#{ pidfile } --state=#{ statefile } --config=#{ configfile } --daemon --preload config.ru"
273
- stop_server =
274
- "#{ @pumactl } --pidfile=#{ pidfile } --state=#{ statefile } stop"
275
- =end
276
- start_server =
277
- "#{ @passenger } start --daemonize --environment #{ @env } --port #{ port } --max-pool-size 16"
278
- stop_server =
279
- "#{ @passenger } stop --port #{ port }"
280
-
281
- `#{ stop_server } 2>&1`.strip
282
-
283
- log(:info, "cmd: #{ start_server }")
284
- server_output = `#{ start_server } 2>&1`.strip
285
-
286
- log(:info, "status: #{ $?.exitstatus }")
287
-
288
- t = Time.now.to_f
289
- timeout = 10
290
- i = 0
291
-
292
- loop do
293
- i += 1
294
-
295
- begin
296
- url = "http://localhost:#{ port }"
297
- open(url){|socket| socket.read}
298
- @server = url
299
- @port = port
300
- break
301
- rescue Object => e
302
- if i > 2
303
- log :error, "#{ e.message }(#{ e.class })\n"
304
- log :error, "#{ server_output }\n\n"
305
- end
306
-
307
- if((Time.now.to_f - t) > timeout)
308
- abort("could not start server inside of #{ timeout } seconds ;-/")
309
- else
310
- sleep(rand(0.42))
311
- end
312
- end
313
- end
314
-
315
- break if @server
316
- end
317
-
318
- # barf if server could not be started
319
- #
320
- unless @server
321
- abort("could not start server on any of ports #{ ports.first } .. #{ ports.last }")
322
- else
323
- log(:info, "started server on #{ @server }")
324
- end
325
-
326
- # set assassins to ensure the server daemon never outlives the build script
327
- # no matter how it is killed (even -9)
328
- #
329
- at_exit{
330
- log(:info, "cmd: #{ stop_server }")
331
- `#{ stop_server } >/dev/null 2>&1`
332
- log(:info, "status: #{ $?.exitstatus }")
333
- log(:info, "stopped server #{ @server }")
334
- }
335
-
336
- assassin = <<-__
337
- pid = #{ Process.pid }
338
-
339
- 4242.times do
340
- begin
341
- Process.kill(0, pid)
342
- rescue Object => e
343
- if e.is_a?(Errno::ESRCH)
344
- `#{ stop_server } >/dev/null 2>&1`
345
- Process.kill(-15, pid) rescue nil
346
- sleep(rand + rand)
347
- Process.kill(-9, pid) rescue nil
348
- end
349
- exit
350
- end
351
- sleep(1 + rand)
352
- end
353
- __
354
- IO.binwrite('tmp/build-assassin.rb', assassin)
355
- cmd = "nohup ruby tmp/build-assassin.rb >/dev/null 2>&1 &"
356
- system cmd
357
-
358
- #
359
- @started_at = Time.now
360
- @server
361
- end
362
-
363
- #
364
- def extract_urls!(build_url)
365
- urls = []
366
-
367
- code, body = http_get(build_url)
368
-
369
- unless code == 200
370
- raise "failed to get build urls from #{ build_url }"
371
- end
372
-
373
- @_build = JSON.parse(body)
374
-
375
- unless @_build['urls'].is_a?(Array)
376
- raise "failed to find any build urls at #{ build_url }"
377
- end
378
-
379
- urls = @_build['urls']
380
-
381
- urls.map!{|url| url_for(url)}
382
-
383
- log(:info, "extracted #{ urls.size } urls to build from #{ build_url }")
384
-
385
- @urls = urls
386
- end
387
-
388
- #
389
- def clear_cache!
390
- #spawn "rake tmp:cache:clear"
391
- spawn "rails runner 'Rails.cache.clear'"
392
- end
393
-
394
- #
395
- def precompile_assets!
396
- @asset_dir = File.join(@rails_root, "public/assets")
397
- @asset_tmp = false
398
-
399
- if test(?d, @asset_dir)
400
- @asset_tmp = File.join(@rails_root, "tmp/assets-build-#{ @uuid }")
401
- FileUtils.mv(@asset_dir, @asset_tmp)
402
- end
403
-
404
- spawn "RAILS_ENV=production rake assets:precompile"
405
-
406
- assets = Dir.glob(File.join(@rails_root, 'public/assets/**/**'))
407
-
408
- log(:info, "precompiled #{ assets.size } assets")
409
-
410
- ensure_non_digested_assets_also_exist!(assets)
411
- end
412
-
413
- #
414
- def rsync_public!
415
- spawn "rsync -avz ./public/ #{ @directory }"
416
-
417
- count = 0
418
-
419
- Dir.glob(File.join(@directory, '**/**')).each{ count += 1 }
420
-
421
- log(:info, "rsync'd #{ count } files")
422
-
423
- if @asset_tmp
424
- FileUtils.rm_rf(@asset_dir)
425
- FileUtils.mv(@asset_tmp, @asset_dir)
426
- end
427
- end
428
-
429
- #
430
- def parallel_build!(n = nil)
431
- n ||= @parallel
432
-
433
- stats = {
434
- :success => [], :missing => [], :failure => [],
435
- }
436
-
437
- times = Queue.new
438
-
439
- avg = nil
440
-
441
- Thread.new do
442
- Thread.current.abort_on_exception = true
443
- total = 0.0
444
- n = 0
445
- loop do
446
- while(time = times.pop)
447
- total += time
448
- n += 1
449
- avg = (total / n).round(2)
450
- end
451
- end
452
- end
453
-
454
- a = Time.now.to_f
455
-
456
- _stats =
457
- @urls.threadify(n) do |url|
458
- uri = uri_for(url)
459
- path = path_for(uri)
460
- upath = uri.path
461
- rpath = relative_path(path, :from => @directory)
462
-
463
- _a = Time.now.to_f
464
-
465
- code, body = http_get(uri)
466
-
467
- _b = Time.now.to_f
468
- _e = (_b - _a).round(2)
469
-
470
- times.push(_e)
471
-
472
- label = "#{ code } @ (t̄:#{ avg }s)"
473
-
474
- case code
475
- when 200
476
- write_path(path, body)
477
- log(:info, "#{ label }: #{ upath } -> #{ rpath } (t:#{ _e }s)")
478
- [:success, url]
479
- when 404
480
- log(:error, "#{ label }: #{ upath }")
481
- [:missing, url]
482
- when 500
483
- log(:error, "#{ label }: #{ upath }")
484
- [:failure, url]
485
- else
486
- log(:error, "#{ label }: #{ upath }")
487
- [:failure, url]
488
- end
489
- end
490
-
491
- _stats.each{|stat, url| stats[stat].push(url)}
492
-
493
- b = Time.now.to_f
494
-
495
- borked = false
496
-
497
- if stats[:missing].size > 0
498
- borked = true
499
- log(:error, "missing on #{ stats[:missing].size } urls")
500
- end
501
-
502
- if stats[:failure].size > 0
503
- borked = true
504
- log(:error, "failure on #{ stats[:failure].size } urls")
505
- end
506
-
507
- if borked
508
- exit(1)
509
- end
510
-
511
- elapsed = b - a
512
- n = @urls.size
513
- rps = (n / elapsed).round(2)
514
-
515
- log(:info, "downloaded #{ n } urls at ~#{ rps }/s")
516
-
517
- @urls
518
- end
519
-
520
- #
521
- def finalize!
522
- @finished_at = Time.now
523
- elapsed = (@finished_at.to_f - @started_at.to_f)
524
- log(:info, "build time - #{ hms(elapsed) }")
525
- on_netlify = ENV['DEPLOY_PRIME_URL'].to_s =~ /netlify/
526
- strategy = on_netlify ? 'cp_r' : 'ln_s' # netlify refuses to deploy from a symlink ;-/ # FIXME
527
- build = File.join(@rails_root, 'build')
528
- FileUtils.rm_rf(build)
529
- FileUtils.send(strategy, @directory, build)
530
- log(:info, "preview with `static ./build/` # brew install node-static")
531
- end
532
-
533
- #
534
- def http_get(url)
535
- response = HTTP.get(url.to_s, :follow => true)
536
- body = response.body.to_s rescue ''
537
- code = response.status.code rescue 500
538
- [code, body]
539
- end
540
-
541
- def http_connection
542
- @http_connection ||= (
543
- PersistentHTTP.new({
544
- :url => @server,
545
- :pool_size => (@parallel + 1),
546
- :logger => (@verbose ? @logger : nil),
547
- :pool_timeout => 10,
548
- :warn_timeout => 1,
549
- :force_retry => true,
550
- })
551
- )
552
- end
553
-
554
- def http_get(url)
555
- request = Net::HTTP::Get.new(url)
556
- response = http_connection.request(request)
557
-
558
- if response.is_a?(Net::HTTPRedirection)
559
- location = response['Location']
560
- if location.to_s == url.to_s
561
- log(:fatal, "circular redirection on #{ url }")
562
- exit(1)
563
- end
564
- return http_get(location)
565
- end
566
-
567
- code = response.code.to_i rescue 500
568
- body = response.body.to_s rescue ''
569
-
570
- [code, body]
571
- end
572
-
573
- #
574
- def path_for(url)
575
- uri = uri_for(url)
576
-
577
- case
578
- when uri.path=='/' || uri.path=='.'
579
- path = File.join(@directory, 'index.html')
580
- else
581
- path = File.join(@directory, uri.path)
582
- dirname, basename = File.split(path)
583
- base, ext = basename.split('.', 2)
584
- if ext.nil?
585
- path = File.join(path, 'index.html')
586
- end
587
- end
588
- path
589
- end
590
-
591
- #
592
- def write_path(path, body)
593
- FileUtils.mkdir_p(File.dirname(path))
594
- IO.binwrite(path, body)
595
- end
596
-
597
- #
598
- def ensure_non_digested_assets_also_exist!(assets)
599
- re = /(-{1}[a-z0-9]{32}*\.{1}){1}/
600
-
601
- assets.each do |file|
602
- next if File.directory?(file) || file !~ re
603
- source = file.split('/')
604
- source.push(source.pop.gsub(re, '.'))
605
- non_digested = File.join(source)
606
- #log(:debug, "asset: #{ file } -> #{ non_digested }")
607
- FileUtils.ln(file, non_digested)
608
- end
609
- end
610
-
611
- #
612
- def url_for(url)
613
- uri = URI.parse(url.to_s)
614
-
615
- if uri.absolute?
616
- uri.to_s
617
- else
618
- relative_uri = URI.parse(@server)
619
- relative_uri.path = absolute_path_for(uri.path)
620
- relative_uri.query = uri.query
621
- relative_uri.fragment = uri.fragment
622
- relative_uri.to_s
623
- end
624
- end
625
-
626
- #
627
- def uri_for(url)
628
- uri = url.is_a?(URI) ? url : URI.parse(url.to_s)
629
- end
630
-
631
- #
632
- def to_s
633
- @directory.to_s
634
- end
635
-
636
- #
637
- def log(level, *args, &block)
638
- @logger.send(level, *args, &block)
639
- end
640
-
641
- #
642
- def hms(seconds)
643
- return unless seconds
644
- "%02d:%02d:%02d" % hours_minutes_seconds(seconds)
645
- end
646
-
647
- #
648
- def hours_minutes_seconds(seconds)
649
- return unless seconds
650
- seconds = Float(seconds).to_i
651
- hours, seconds = seconds.divmod(3600)
652
- minutes, seconds = seconds.divmod(60)
653
- [hours.to_i, minutes.to_s, seconds]
654
- end
655
-
656
- #
657
- def stopwatch(&block)
658
- a = Time.now
659
- result = block.call
660
- b = Time.now
661
- [result, b.to_f - a.to_f]
662
- end
663
-
664
- #
665
- def port_open?(port, options = {})
666
- seconds = options[:timeout] || 1
667
- ip = options[:ip] || '0.0.0.0'
668
-
669
- Timeout::timeout(seconds) do
670
- begin
671
- TCPSocket.new(ip, port).close
672
- false
673
- rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
674
- true
675
- rescue Object
676
- false
677
- end
678
- end
679
- rescue Timeout::Error
680
- false
681
- end
682
-
683
- #
684
- def paths_for(*args)
685
- path = args.flatten.compact.join('/')
686
- path.gsub!(%r|[.]+/|, '/')
687
- path.squeeze!('/')
688
- path.sub!(%r|^/|, '')
689
- path.sub!(%r|/$|, '')
690
- paths = path.split('/')
691
- end
692
-
693
- #
694
- def absolute_path_for(*args)
695
- path = ('/' + paths_for(*args).join('/')).squeeze('/')
696
- path unless path.strip.empty?
697
- end
698
-
699
- #
700
- def relative_path_for(*args)
701
- path = absolute_path_for(*args).sub(%r{^/+}, '')
702
- path unless path.strip.empty?
703
- end
704
-
705
- #
706
- def relative_path(path, *args)
707
- options = args.last.is_a?(Hash) ? args.pop : {}
708
- path = path.to_s
709
- relative = args.shift || options[:relative] || options[:to] || options[:from]
710
- if relative
711
- Pathname.new(path).relative_path_from(Pathname.new(relative.to_s)).to_s
712
- else
713
- relative_path_for(path)
714
- end
715
- end
716
-
717
- #
718
- def spawn(command)
719
- oe = `#{ command } 2>&1`
720
-
721
- unless $? == 0
722
- msg = "command(#{ command }) failed with (#{ $? })"
723
- log(:error, msg)
724
- raise(msg)
725
- end
726
-
727
- oe
728
- end
729
- end
214
+ #
215
+ def parse_args!
216
+ @args = ARGV.map{|arg| "#{ arg }"}
217
+ end
218
+
219
+ #
220
+ def parse_opts!
221
+ @opts = Hash.new
222
+
223
+ CLI.opts.each do |opt, arg|
224
+ key, val = opt.split('--').last, arg
225
+ @opts[key.to_s.to_sym] = (val == '' ? true : val)
226
+ end
227
+
228
+ @opts
229
+ end
230
+
231
+ #
232
+ def usage!
233
+ lines = CLI.usage.strip.split(/\n/)
234
+ n = lines[1].to_s.scan(/^\s+/).size
235
+ indent = ' ' * n
236
+ re = /^#{ Regexp.escape(indent) }/
237
+ usage = lines.map{|line| line.gsub(re, '')}.join("\n")
238
+ STDERR.puts(usage)
239
+ end
240
+
241
+ #
242
+ def mkdir!
243
+ FileUtils.rm_rf(@directory)
244
+ FileUtils.mkdir_p(@directory)
245
+ log(:info, "directory: #{ @directory }")
246
+ end
247
+
248
+ #
249
+ def start_server!
250
+ @url =
251
+ nil
252
+
253
+ @port =
254
+ nil
255
+
256
+ ports =
257
+ (2000 .. 9000).to_a
258
+
259
+ start_server, stop_server = nil
260
+
261
+ @server.setup!
262
+
263
+ ports.each do |port|
264
+ next unless port_open?(port)
265
+
266
+ start_server = @server.start_command_for(port)
267
+ stop_server = @server.stop_command_for(port)
268
+
269
+ `#{ stop_server } 2>&1`.strip
270
+
271
+ log(:info, "cmd: #{ start_server }")
272
+ server_output = `#{ start_server } 2>&1`.strip
273
+
274
+ log(:info, "status: #{ $?.exitstatus }")
275
+
276
+ t = Time.now.to_f
277
+ timeout = 10
278
+ i = 0
279
+ url = nil
280
+
281
+ loop do
282
+ i += 1
283
+
284
+ begin
285
+ url = "http://localhost:#{ port }"
286
+ open(url){|socket| socket.read}
287
+ @url = url
288
+ @port = port
289
+ break
290
+ rescue Object => e
291
+ if i > 2
292
+ log :error, "url: #{ url } ->"
293
+ log :error, "#{ e.message }(#{ e.class })\n"
294
+ log :error, "#{ server_output }\n\n"
295
+ end
296
+
297
+ if((Time.now.to_f - t) > timeout)
298
+ abort("could not start server inside of #{ timeout } seconds via\n\n#{ start_server }\n\n;-/")
299
+ else
300
+ sleep(rand(0.42))
301
+ end
302
+ end
303
+ end
304
+
305
+ break if @url
306
+ end
307
+
308
+ # barf if server could not be started
309
+ #
310
+ unless @url
311
+ abort("could not start server on any of ports #{ ports.first } .. #{ ports.last }")
312
+ else
313
+ log(:info, "started server on #{ @url }")
314
+ end
315
+
316
+ # set assassins to ensure the server daemon never outlives the build script
317
+ # no matter how it is killed (even -9)
318
+ #
319
+ at_exit{
320
+ log(:info, "cmd: #{ stop_server }")
321
+ `#{ stop_server } >/dev/null 2>&1`
322
+ log(:info, "status: #{ $?.exitstatus }")
323
+ log(:info, "stopped server #{ @url }")
324
+ @server.cleanup!(:port => @port)
325
+ }
326
+
327
+ assassin = <<-__
328
+ pid = #{ Process.pid }
329
+
330
+ 4242.times do
331
+ begin
332
+ Process.kill(0, pid)
333
+ rescue Object => e
334
+ if e.is_a?(Errno::ESRCH)
335
+ `#{ stop_server } >/dev/null 2>&1`
336
+ Process.kill(-15, pid) rescue nil
337
+ sleep(rand + rand)
338
+ Process.kill(-9, pid) rescue nil
339
+ end
340
+ exit
341
+ end
342
+ sleep(1 + rand)
343
+ end
344
+ __
345
+ IO.binwrite('tmp/build-assassin.rb', assassin)
346
+ cmd = "nohup ruby tmp/build-assassin.rb >/dev/null 2>&1 &"
347
+ system cmd
348
+
349
+ #
350
+ @started_at = Time.now
351
+ @url
352
+ end
353
+
354
+ #
355
+ def extract_urls!(build_url)
356
+ urls = []
357
+
358
+ code, body = http_get(build_url)
359
+
360
+ unless code == 200
361
+ raise "failed to get build urls from #{ build_url }"
362
+ end
363
+
364
+ @_build = JSON.parse(body)
365
+
366
+ unless @_build['urls'].is_a?(Array)
367
+ raise "failed to find any build urls at #{ build_url } - edit ./config/rails_build.rb # see README"
368
+ end
369
+
370
+ urls = @_build['urls']
371
+
372
+ urls.map!{|url| url_for(url)}
373
+
374
+ log(:info, "extracted #{ urls.size } url(s) to build from #{ build_url }")
375
+
376
+ @urls = urls
377
+ end
378
+
379
+ #
380
+ def clear_cache!
381
+ #spawn "rake tmp:cache:clear"
382
+ spawn "rails runner 'Rails.cache.clear'"
383
+ end
384
+
385
+ #
386
+ def precompile_assets!
387
+ @asset_dir = File.join(@rails_root, "public/assets")
388
+ @asset_tmp = false
389
+
390
+ if test(?d, @asset_dir)
391
+ @asset_tmp = File.join(@rails_root, "tmp/assets-build-#{ @uuid }")
392
+ FileUtils.mv(@asset_dir, @asset_tmp)
393
+ end
394
+
395
+ spawn "RAILS_ENV=production rake assets:precompile"
396
+
397
+ assets = Dir.glob(File.join(@rails_root, 'public/assets/**/**'))
398
+
399
+ log(:info, "precompiled #{ assets.size } assets")
400
+
401
+ ensure_non_digested_assets_also_exist!(assets)
402
+ end
403
+
404
+ #
405
+ def rsync_public!
406
+ spawn "rsync -avz ./public/ #{ @directory }"
407
+
408
+ count = 0
409
+
410
+ Dir.glob(File.join(@directory, '**/**')).each{ count += 1 }
411
+
412
+ log(:info, "rsync'd #{ count } files")
413
+
414
+ if @asset_tmp
415
+ FileUtils.rm_rf(@asset_dir)
416
+ FileUtils.mv(@asset_tmp, @asset_dir)
417
+ end
418
+ end
419
+
420
+ #
421
+ def parallel_build!(n = nil)
422
+ n ||= @parallel
423
+
424
+ stats = {
425
+ :success => [], :missing => [], :failure => [],
426
+ }
427
+
428
+ times = Queue.new
429
+
430
+ avg = nil
431
+
432
+ Thread.new do
433
+ Thread.current.abort_on_exception = true
434
+ total = 0.0
435
+ n = 0
436
+ loop do
437
+ while(time = times.pop)
438
+ total += time
439
+ n += 1
440
+ avg = (total / n).round(2)
441
+ end
442
+ end
443
+ end
444
+
445
+ a = Time.now.to_f
446
+
447
+ _stats =
448
+ @urls.threadify(n) do |url|
449
+ uri = uri_for(url)
450
+ path = path_for(uri)
451
+ upath = uri.path
452
+ rpath = relative_path(path, :from => @directory)
453
+
454
+ _a = Time.now.to_f
455
+
456
+ code, body = http_get(uri)
457
+
458
+ _b = Time.now.to_f
459
+ _e = (_b - _a).round(2)
460
+
461
+ times.push(_e)
462
+
463
+ label = "#{ code } @ (t̄:#{ avg }s)"
464
+
465
+ case code
466
+ when 200
467
+ write_path(path, body)
468
+ log(:info, "#{ label }: #{ upath } -> #{ rpath } (t:#{ _e }s)")
469
+ [:success, url]
470
+ when 404
471
+ log(:error, "#{ label }: #{ upath }")
472
+ [:missing, url]
473
+ when 500
474
+ log(:error, "#{ label }: #{ upath }")
475
+ [:failure, url]
476
+ else
477
+ log(:error, "#{ label }: #{ upath }")
478
+ [:failure, url]
479
+ end
480
+ end
481
+
482
+ _stats.each{|stat, url| stats[stat].push(url)}
483
+
484
+ b = Time.now.to_f
485
+
486
+ borked = false
487
+
488
+ if stats[:missing].size > 0
489
+ borked = true
490
+ log(:error, "missing on #{ stats[:missing].size } urls")
491
+ end
492
+
493
+ if stats[:failure].size > 0
494
+ borked = true
495
+ log(:error, "failure on #{ stats[:failure].size } urls")
496
+ end
497
+
498
+ if borked
499
+ exit(1)
500
+ end
501
+
502
+ elapsed = b - a
503
+ n = @urls.size
504
+ rps = (n / elapsed).round(2)
505
+
506
+ log(:info, "downloaded #{ n } urls at ~#{ rps }/s")
507
+
508
+ @urls
509
+ end
510
+
511
+ #
512
+ def finalize!
513
+ @finished_at = Time.now
514
+ elapsed = (@finished_at.to_f - @started_at.to_f)
515
+ log(:info, "build time - #{ hms(elapsed) }")
516
+ on_netlify = ENV['DEPLOY_PRIME_URL'].to_s =~ /netlify/
517
+ strategy = on_netlify ? 'cp_r' : 'ln_s' # netlify refuses to deploy from a symlink ;-/ # FIXME
518
+ build = File.join(@rails_root, 'build')
519
+ FileUtils.rm_rf(build)
520
+ FileUtils.send(strategy, @directory, build)
521
+ log(:info, "preview with `static ./build/` # brew install node-static")
522
+ end
523
+
524
+ def http_connection
525
+ @http_connection ||= (
526
+ PersistentHTTP.new({
527
+ :url => @url,
528
+ :pool_size => (@parallel + 1),
529
+ :logger => (@verbose ? @logger : nil),
530
+ :pool_timeout => 10,
531
+ :warn_timeout => 1,
532
+ :force_retry => true,
533
+ })
534
+ )
535
+ end
536
+
537
+ def http_get(url)
538
+ request = Net::HTTP::Get.new(url)
539
+ response = http_connection.request(request)
540
+
541
+ if response.is_a?(Net::HTTPRedirection)
542
+ location = response['Location']
543
+ if location.to_s == url.to_s
544
+ log(:fatal, "circular redirection on #{ url }")
545
+ exit(1)
546
+ end
547
+ return http_get(location)
548
+ end
549
+
550
+ code = response.code.to_i rescue 500
551
+ body = response.body.to_s rescue ''
552
+
553
+ [code, body]
554
+ end
555
+
556
+ #
557
+ def to_s
558
+ @directory.to_s
559
+ end
560
+
561
+ #
562
+ def log(level, *args, &block)
563
+ @logger.send(level, *args, &block)
564
+ end
565
+
566
+ #
567
+ def path_for(url)
568
+ uri = uri_for(url)
569
+
570
+ case
571
+ when uri.path=='/' || uri.path=='.'
572
+ path = File.join(@directory, 'index.html')
573
+ else
574
+ path = File.join(@directory, uri.path)
575
+ dirname, basename = File.split(path)
576
+ base, ext = basename.split('.', 2)
577
+ if ext.nil?
578
+ path = File.join(path, 'index.html')
579
+ end
580
+ end
581
+ path
582
+ end
583
+
584
+ #
585
+ def write_path(path, body)
586
+ FileUtils.mkdir_p(File.dirname(path))
587
+ IO.binwrite(path, body)
588
+ end
589
+
590
+ #
591
+ def ensure_non_digested_assets_also_exist!(assets)
592
+ re = /(-{1}[a-z0-9]{32}*\.{1}){1}/
593
+
594
+ assets.each do |file|
595
+ next if File.directory?(file) || file !~ re
596
+ source = file.split('/')
597
+ source.push(source.pop.gsub(re, '.'))
598
+ non_digested = File.join(source)
599
+ #log(:debug, "asset: #{ file } -> #{ non_digested }")
600
+ FileUtils.ln(file, non_digested)
601
+ end
602
+ end
603
+
604
+ #
605
+ def url_for(url)
606
+ uri = URI.parse(url.to_s)
607
+
608
+ if uri.absolute?
609
+ uri.to_s
610
+ else
611
+ relative_uri = URI.parse(@url)
612
+ relative_uri.path = absolute_path_for(uri.path)
613
+ relative_uri.query = uri.query
614
+ relative_uri.fragment = uri.fragment
615
+ relative_uri.to_s
616
+ end
617
+ end
618
+
619
+ #
620
+ def uri_for(url)
621
+ uri = url.is_a?(URI) ? url : URI.parse(url.to_s)
622
+ end
623
+
624
+ #
625
+ def hms(seconds)
626
+ return unless seconds
627
+ "%02d:%02d:%02d" % hours_minutes_seconds(seconds)
628
+ end
629
+
630
+ #
631
+ def hours_minutes_seconds(seconds)
632
+ return unless seconds
633
+ seconds = Float(seconds).to_i
634
+ hours, seconds = seconds.divmod(3600)
635
+ minutes, seconds = seconds.divmod(60)
636
+ [hours.to_i, minutes.to_s, seconds]
637
+ end
638
+
639
+ #
640
+ def stopwatch(&block)
641
+ a = Time.now
642
+ result = block.call
643
+ b = Time.now
644
+ [result, b.to_f - a.to_f]
645
+ end
646
+
647
+ #
648
+ def port_open?(port, options = {})
649
+ seconds = options[:timeout] || 1
650
+ ip = options[:ip] || '0.0.0.0'
651
+
652
+ Timeout::timeout(seconds) do
653
+ begin
654
+ TCPSocket.new(ip, port).close
655
+ false
656
+ rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
657
+ true
658
+ rescue Object
659
+ false
660
+ end
661
+ end
662
+ rescue Timeout::Error
663
+ false
664
+ end
665
+
666
+ #
667
+ def paths_for(*args)
668
+ path = args.flatten.compact.join('/')
669
+ path.gsub!(%r|[.]+/|, '/')
670
+ path.squeeze!('/')
671
+ path.sub!(%r|^/|, '')
672
+ path.sub!(%r|/$|, '')
673
+ paths = path.split('/')
674
+ end
675
+
676
+ #
677
+ def absolute_path_for(*args)
678
+ path = ('/' + paths_for(*args).join('/')).squeeze('/')
679
+ path unless path.strip.empty?
680
+ end
681
+
682
+ #
683
+ def relative_path_for(*args)
684
+ path = absolute_path_for(*args).sub(%r{^/+}, '')
685
+ path unless path.strip.empty?
686
+ end
687
+
688
+ #
689
+ def relative_path(path, *args)
690
+ options = args.last.is_a?(Hash) ? args.pop : {}
691
+ path = path.to_s
692
+ relative = args.shift || options[:relative] || options[:to] || options[:from]
693
+ if relative
694
+ Pathname.new(path).relative_path_from(Pathname.new(relative.to_s)).to_s
695
+ else
696
+ relative_path_for(path)
697
+ end
698
+ end
699
+
700
+ #
701
+ def spawn(command)
702
+ oe = `#{ command } 2>&1`
703
+
704
+ unless $? == 0
705
+ msg = "command(#{ command }) failed with (#{ $? })"
706
+ log(:error, msg)
707
+ raise(msg)
708
+ end
709
+
710
+ oe
711
+ end
712
+ end
713
+
714
+ #
715
+ class Server
716
+ def Server.for(type, cli, *args, &block)
717
+ type = type.to_s.strip.downcase
718
+
719
+ case type
720
+ when /puma/
721
+ Puma.new(cli, *args, &block)
722
+ when /passenger/
723
+ Passenger.new(cli, *args, &block)
724
+ else
725
+ nil
726
+ end
727
+ end
728
+
729
+ def initialize(cli)
730
+ @cli = cli
731
+ end
732
+
733
+ def setup!
734
+ nil
735
+ end
736
+
737
+ class Puma < Server
738
+ def initialize(cli)
739
+ @cli = cli
740
+
741
+ @env = @cli.env
742
+ @directory = @cli.directory
743
+ @rails_root = @cli.rails_root
744
+ @parallel = @cli.parallel
745
+
746
+ @puma = "bundle exec puma"
747
+ @pumactl = "bundle exec pumactl"
748
+
749
+ @pidfile = @cli.relative_path(File.join(@directory, ".puma-pid.txt"), :from => @cli.rails_root)
750
+ @statefile = @cli.relative_path(File.join(@directory, ".puma-state.txt"), :from => @cli.rails_root)
751
+ end
752
+
753
+ def start_command_for(port)
754
+ %W[
755
+ #{ @puma }
756
+ --pidfile=#{ @pidfile }
757
+ --state=#{ @statefile }
758
+ --port=#{ port }
759
+ --environment=#{ @env }
760
+ --workers=#{ @parallel }
761
+ --config=/dev/null
762
+ --threads=1:1
763
+ --preload
764
+ --daemon
765
+ ./config.ru
766
+ ].join(' ').strip
767
+ end
768
+
769
+ def stop_command_for(port)
770
+ %W[
771
+ #{ @pumactl }
772
+ --pidfile=#{ @pidfile }
773
+ --state=#{ @statefile }
774
+ stop
775
+ ].join(' ').strip
776
+ end
777
+
778
+ def cleanup!(*args)
779
+ nil
780
+ end
781
+ end
782
+
783
+ class Passenger < Server
784
+ def initialize(cli)
785
+ @cli = cli
786
+
787
+ @env = @cli.env
788
+ @directory = @cli.directory
789
+ @rails_root = @cli.rails_root
790
+ @parallel = @cli.parallel
791
+
792
+ @passenger = "bundle exec passenger"
793
+ end
794
+
795
+ def start_command_for(port)
796
+ "#{ @passenger } start --daemonize --environment #{ @env } --port #{ port } --max-pool-size #{ @parallel }"
797
+ end
798
+
799
+ def stop_command_for(port)
800
+ "#{ @passenger } stop --port #{ port }"
801
+ end
802
+
803
+ def cleanup!(*args)
804
+ options = args.last.is_a?(Hash) ? args.pop : {}
805
+ port = options[:port]
806
+ FileUtils.rm_rf("#{ @rails_root }/tmp/pids/passenger.#{ port }.pid")
807
+ end
808
+
809
+ def setup!
810
+ begin
811
+ require 'phusion_passenger'
812
+ rescue LoadError => le
813
+ abort "please add `gem 'passenger'` to your Gemfile"
814
+ end
815
+ end
816
+ end
817
+ end
730
818
  end