rails_build 1.0.0 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +8 -5
  3. data/bin/rails_build +730 -642
  4. data/lib/rails_build.rb +2 -0
  5. data/lib/rails_build/engine.rb +2 -2
  6. data/lib/rails_build/version.rb +1 -1
  7. metadata +46 -9
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: cd82bac7610eed4089c5be9667a685dea9ebd327
4
- data.tar.gz: 99110d59847285fad099e0769f333f49aded632f
3
+ metadata.gz: 6e8b28d514b5404bbcadaca8e7d8ade5021265e8
4
+ data.tar.gz: a36bf4c19682e86162b5579122ee776fdd9b20bc
5
5
  SHA512:
6
- metadata.gz: 4c4ddd3929c28ada377db4c57811e3934aaeeaae6e30e6fa920f158e5cd7c9bce2276111fbda6f15b23685658fe88cb107593d981e8bf4d0dd9788036eb9f8be
7
- data.tar.gz: e1203289ae526130f34645705cefc37edf2409065671bd591991084755eabe7e0aeb1a37838dc9d94bde6a9793d370163f24cce9ccac9843cf443322c2a55642
6
+ metadata.gz: be11ffbdd737215266b85ca7de68a873ecc23b1b59e9dc668045f82bdffd769862c36c0e0cd29d16e3fe57ce564abb08a7c53303f66edcd7d3a785a9c9cb54ec
7
+ data.tar.gz: 2e2a593592f58735a55989dbd6af9849b0b74f9b09870708b62a05bbed33b18a1d1d5523735dc4adbe369acf1d00a7492a596295802250a922539776b1ddd2a2
data/README.md CHANGED
@@ -8,7 +8,7 @@ built as a Rails 5 engine.
8
8
  ## How It Works
9
9
 
10
10
  RailsBuild bundles all your assets, public directory, and configured urls into
11
- a static site suitable for deployment to Netlify, Amazon S3, or your favorite
11
+ a static site suitable for deployment to [Netlify](https://www.netlify.com/), Amazon S3, or your favorite
12
12
  static website hosting solution. It does this by:
13
13
 
14
14
  - Booting your application in *production* mode
@@ -92,14 +92,17 @@ like './about' vs. '../about'
92
92
 
93
93
  ## Optimization and Notes
94
94
 
95
-
96
95
  RailsBuild is fast. Very fast. [DOJO4](http://dojo4.com) has seen optimized [Middleman](https://middlemanapp.com/) builds of > 30 minutes dropped to *60 seconds* by simply making basic use of Rails' built-in caching facilites.
97
96
 
97
+ You app has to run in production mode to build! Don't forget to setup
98
+ secrets, or anything else generally required in production mode.
99
+
98
100
  When trying to squeeze out performance just remember that RailsBuild runs in
99
101
  production mode and, therefore, making a build go fast follows the *exact same
100
102
  rules* as making anything other Rails' application fast. The first place to
101
103
  reach is typically fragment caching of partials used in your app.
102
104
 
105
+
103
106
  Finally, don't forget about *./config/initializers/assets.rb* - RailsBuild
104
107
  doesn't do anything special to the asset pipeline and only those assets
105
108
  normally built when
@@ -157,14 +160,14 @@ After installation and configuration simply run
157
160
 
158
161
 
159
162
 
160
- ## Netlfiy
163
+ ## Netlify
161
164
 
162
- We love Netlify at [DOJO4](http://dojo4.com). RailsBuild works with netlify
165
+ We love [Netlify](https://www.netlify.com/) at [DOJO4](http://dojo4.com). RailsBuild works with netlify
163
166
  out of the box and simply requires
164
167
 
165
168
  ```yaml
166
169
 
167
- build_command : rails_build
170
+ build_command : ./bin/rails_build
168
171
 
169
172
  build_directory: build
170
173
 
data/bin/rails_build CHANGED
@@ -10,13 +10,13 @@ END{
10
10
 
11
11
 
12
12
  module RailsBuild
13
- class CLI
14
- def CLI.usage
15
- <<-__
16
- NAME
17
- rails_build
13
+ class CLI
14
+ def CLI.usage
15
+ <<-__
16
+ NAME
17
+ rails_build
18
18
 
19
- SYNOPSIS
19
+ SYNOPSIS
20
20
  a small, simple, bullet proof, and very fast static site generator built on rails 5
21
21
 
22
22
  USAGE
@@ -27,168 +27,171 @@ module RailsBuild
27
27
  --rails_root,--r : specifiy the RAILS_ROOT, default=./
28
28
  --parallel,--p : how many requests to make in parallel, default=n_cpus/2
29
29
  --env,--e : speciify the RAILS_ENV, default=production
30
- --server, -s : provide the url of the build server, do *not* start separate one
30
+ --url, -u : provide the url of the build server, do *not* start separate one
31
+ --server, -s : passenger | puma, default=passenger
31
32
  --log, -l : specify the logfile, default=STDERR
32
33
  --verbose, -v : turn on verbose logging
33
- __
34
- end
34
+ __
35
+ end
35
36
 
36
- def CLI.opts
37
- GetoptLong.new(
38
- [ '--help' , '-h' , GetoptLong::NO_ARGUMENT ] ,
39
- [ '--server' , '-s' , GetoptLong::REQUIRED_ARGUMENT ] ,
40
- [ '--parallel' , '-p' , GetoptLong::REQUIRED_ARGUMENT ] ,
41
- [ '--rails_root' , '-r' , GetoptLong::REQUIRED_ARGUMENT ] ,
42
- [ '--env' , '-e' , GetoptLong::REQUIRED_ARGUMENT ] ,
43
- [ '--log' , '-l' , GetoptLong::REQUIRED_ARGUMENT ] ,
44
- [ '--verbose' , '-v' , GetoptLong::NO_ARGUMENT ] ,
45
- )
46
- end
37
+ def CLI.opts
38
+ GetoptLong.new(
39
+ [ '--help' , '-h' , GetoptLong::NO_ARGUMENT ] ,
40
+ [ '--url' , '-u' , GetoptLong::REQUIRED_ARGUMENT ] ,
41
+ [ '--server' , '-s' , GetoptLong::REQUIRED_ARGUMENT ] ,
42
+ [ '--parallel' , '-p' , GetoptLong::REQUIRED_ARGUMENT ] ,
43
+ [ '--rails_root' , '-r' , GetoptLong::REQUIRED_ARGUMENT ] ,
44
+ [ '--env' , '-e' , GetoptLong::REQUIRED_ARGUMENT ] ,
45
+ [ '--log' , '-l' , GetoptLong::REQUIRED_ARGUMENT ] ,
46
+ [ '--verbose' , '-v' , GetoptLong::NO_ARGUMENT ] ,
47
+ )
48
+ end
47
49
 
48
- def build!
49
- prepare!
50
+ def build!
51
+ prepare!
50
52
 
51
- mkdir!
53
+ mkdir!
52
54
 
53
- start_server! unless server
55
+ start_server! unless url
54
56
 
55
- clear_cache!
57
+ clear_cache!
56
58
 
57
- extract_urls! url_for('/rails_build/configuration.json')
59
+ extract_urls! url_for('/rails_build/configuration.json')
58
60
 
59
- precompile_assets!
61
+ precompile_assets!
60
62
 
61
- rsync_public!
63
+ rsync_public!
62
64
 
63
- parallel_build!
65
+ parallel_build!
64
66
 
65
- finalize!
66
- end
67
+ finalize!
68
+ end
67
69
 
68
- #
69
- def CLI.build!(*args, &block)
70
- new(*args, &block).build!
71
- end
70
+ #
71
+ def CLI.build!(*args, &block)
72
+ new(*args, &block).build!
73
+ end
72
74
 
73
- #
74
- attr_accessor :rails_root
75
- attr_accessor :server
76
- attr_accessor :directory
77
- attr_accessor :uuid
78
- attr_accessor :id
79
- attr_accessor :env
80
- attr_accessor :puma
81
- attr_accessor :passenger
82
- attr_accessor :url
75
+ #
76
+ attr_accessor :rails_root
77
+ attr_accessor :url
78
+ attr_accessor :server
79
+ attr_accessor :directory
80
+ attr_accessor :uuid
81
+ attr_accessor :id
82
+ attr_accessor :env
83
+ attr_accessor :parallel
83
84
 
84
- #
85
- def initialize(*args, &block)
85
+ #
86
+ def initialize(*args, &block)
86
87
  setup!
87
88
 
88
- @args = parse_args!
89
+ @args = parse_args!
89
90
 
90
- @opts = parse_opts!
91
+ @opts = parse_opts!
91
92
 
92
- if @opts[:help]
93
- usage!
94
- exit(42)
95
- end
96
- end
93
+ if @opts[:help]
94
+ usage!
95
+ exit(42)
96
+ end
97
+ end
97
98
 
98
99
  #
99
100
  def setup!
100
- #
101
- STDOUT.sync = true
102
- STDERR.sync = true
101
+ #
102
+ STDOUT.sync = true
103
+ STDERR.sync = true
103
104
 
104
- #
105
- ENV['SPRING_DISABLE'] = 'true'
106
- ENV['DISABLE_SPRING'] = 'true'
105
+ #
106
+ ENV['SPRING_DISABLE'] = 'true'
107
+ ENV['DISABLE_SPRING'] = 'true'
107
108
 
108
- #
109
- %w[
110
- fileutils pathname thread socket timeout time uri etc open-uri securerandom logger getoptlong rubygems json
111
- ].each do |stdlib|
112
- require(stdlib)
113
- end
109
+ #
110
+ %w[
111
+ fileutils pathname thread socket timeout time uri etc open-uri securerandom logger getoptlong rubygems json
112
+ ].each do |stdlib|
113
+ require(stdlib)
114
+ end
114
115
  end
115
116
 
116
- #
117
- def prepare!
117
+ #
118
+ def prepare!
118
119
  #
119
- @rails_root = find_rails_root!(@args[0] || '.')
120
+ @rails_root = find_rails_root!(@args[0] || '.')
120
121
 
121
122
  #
122
- Dir.chdir(@rails_root)
123
+ Dir.chdir(@rails_root)
123
124
 
124
125
  #
125
- if File.exists?('./Gemfile')
126
- require 'bundler/setup'
127
- Bundler.setup(:require => false)
128
- end
129
-
130
- #
131
- %w[
132
- threadify persistent_http phusion_passenger
133
- ].each do |gem|
126
+ if File.exists?('./Gemfile')
127
+ require 'bundler/setup'
128
+ Bundler.setup(:require => false)
129
+ end
130
+
131
+ #
132
+ %w[
133
+ threadify persistent_http
134
+ ].each do |gem|
134
135
  begin
135
136
  require(gem)
136
137
  rescue LoadError
137
- abort "add gem 'rails_build' to your Gemfile"
138
+ abort "add gem '#{ gem }' to your Gemfile"
138
139
  end
139
- end
140
+ end
140
141
 
141
142
  #
142
- begin
143
- require('pry')
144
- rescue LoadError
145
- nil
146
- end
147
-
148
- #
149
- @logger = Logger.new(@opts[:log] || STDERR)
150
-
151
- @uuid = ENV['RAILS_BUILD']
152
- @time = ENV['RAILS_BUILD_TIME']
153
- @server = @opts[:server] || ENV['RAILS_BUILD_SERVER']
154
- @env = @opts[:env] || ENV['RAILS_BUILD_ENV'] || ENV['RAILS_ENV']
155
- @parallel = @opts[:parallel] || ENV['RAILS_BUILD_PARALLEL']
156
- @verbose = @opts[:verbose] || ENV['RAILS_BUILD_VERBOSE']
157
-
158
- @uuid ||= SecureRandom.uuid
159
- @time ||= Time.now.utc
160
- @env ||= 'production'
161
- @parallel ||= (Etc.nprocessors/2)
162
-
163
- unless @time.is_a?(Time)
164
- @time = Time.parse(@time.to_s).utc
165
- end
166
-
167
- @parallel = @parallel.to_i
143
+ begin
144
+ require('pry')
145
+ rescue LoadError
146
+ nil
147
+ end
168
148
 
149
+ #
150
+ @logger = Logger.new(@opts[:log] || STDERR)
151
+
152
+ @uuid = ENV['RAILS_BUILD']
153
+ @time = ENV['RAILS_BUILD_TIME']
154
+ @url = @opts[:url] || ENV['RAILS_BUILD_URL']
155
+ @server = @opts[:server] || ENV['RAILS_BUILD_SERVER']
156
+ @env = @opts[:env] || ENV['RAILS_BUILD_ENV'] || ENV['RAILS_ENV']
157
+ @parallel = @opts[:parallel] || ENV['RAILS_BUILD_PARALLEL']
158
+ @verbose = @opts[:verbose] || ENV['RAILS_BUILD_VERBOSE']
159
+
160
+ @uuid ||= SecureRandom.uuid
161
+ @time ||= Time.now.utc
162
+ @server ||= 'passenger'
163
+ @env ||= 'production'
164
+ @parallel ||= (Etc.nprocessors/2)
165
+
166
+ unless @time.is_a?(Time)
167
+ @time = Time.parse(@time.to_s).utc
168
+ end
169
169
 
170
- @puma = 'bundle exec puma'
171
- @pumactl = 'bundle exec pumactl'
170
+ @parallel = @parallel.to_i
172
171
 
173
- @passenger = 'bundle exec passenger'
172
+ if ENV['RAILS_BUILD_DIRECTORY']
173
+ @build_directory = File.expand_path(ENV['RAILS_BUILD_DIRECTORY'])
174
+ else
175
+ @build_directory = File.join(@rails_root, 'builds')
176
+ end
174
177
 
175
- if ENV['RAILS_BUILD_DIRECTORY']
176
- @build_directory = File.expand_path(ENV['RAILS_BUILD_DIRECTORY'])
177
- else
178
- @build_directory = File.join(@rails_root, 'builds')
179
- end
178
+ @directory = File.join(@build_directory, @uuid)
180
179
 
181
- @directory = File.join(@build_directory, @uuid)
180
+ ENV['RAILS_ENV'] = @env
181
+ ENV['RAILS_BUILD'] = @uuid
182
+ ENV['RAILS_BUILD_ENV'] = @env
183
+ ENV['RAILS_BUILD_TIME'] = @time.httpdate
182
184
 
183
- ENV['RAILS_ENV'] = @env
184
- ENV['RAILS_BUILD'] = @uuid
185
- ENV['RAILS_BUILD_ENV'] = @env
186
- ENV['RAILS_BUILD_TIME'] = @time.httpdate
185
+ @urls = []
187
186
 
188
- @urls = []
187
+ @started_at = Time.now
189
188
 
190
- @started_at = Time.now
191
- end
189
+ unless(( server = Server.for(@server, self) ))
190
+ abort "no server found for server=#{ @server.inspect }"
191
+ else
192
+ @server = server
193
+ end
194
+ end
192
195
 
193
196
  #
194
197
  def find_rails_root!(path)
@@ -208,523 +211,608 @@ module RailsBuild
208
211
  abort("could not find a rails_root in or above #{ path }!?")
209
212
  end
210
213
 
211
- #
212
- def parse_args!
213
- @args = ARGV.map{|arg| "#{ arg }"}
214
- end
215
-
216
- #
217
- def parse_opts!
218
- @opts = Hash.new
219
-
220
- CLI.opts.each do |opt, arg|
221
- key, val = opt.split('--').last, arg
222
- @opts[key] = (val == '' ? true : val)
223
- end
224
-
225
- @opts
226
- end
227
-
228
- #
229
- def usage!
230
- lines = CLI.usage.strip.split(/\n/)
231
- n = lines[1].to_s.scan(/^\s+/).size
232
- indent = ' ' * n
233
- re = /^#{ Regexp.escape(indent) }/
234
- usage = lines.map{|line| line.gsub(re, '')}.join("\n")
235
- STDERR.puts(usage)
236
- end
237
-
238
- #
239
- def mkdir!
240
- FileUtils.rm_rf(@directory)
241
- FileUtils.mkdir_p(@directory)
242
- log(:info, "directory: #{ @directory }")
243
- end
244
-
245
- #
246
- def start_server!
247
- @server =
248
- nil
249
-
250
- @port =
251
- nil
252
-
253
- ports =
254
- (2000 .. 9000).to_a
255
-
256
- # TODO - get puma working...
257
- =begin
258
- pidfile = File.join(@directory, "puma.pid")
259
- statefile = File.join(@directory, "puma-state.txt")
260
- configfile = File.join(@directory, "puma-config.rb")
261
- FileUtils.touch(configfile)
262
- =end
263
-
264
- start_server, stop_server = nil
265
-
266
- ports.each do |port|
267
- next unless port_open?(port)
268
-
269
- # TODO - get puma working...
270
- =begin
271
- start_server =
272
- "puma --port=#{ port } --environment=#{ @env } --pidfile=#{ pidfile } --state=#{ statefile } --config=#{ configfile } --daemon --preload config.ru"
273
- stop_server =
274
- "#{ @pumactl } --pidfile=#{ pidfile } --state=#{ statefile } stop"
275
- =end
276
- start_server =
277
- "#{ @passenger } start --daemonize --environment #{ @env } --port #{ port } --max-pool-size 16"
278
- stop_server =
279
- "#{ @passenger } stop --port #{ port }"
280
-
281
- `#{ stop_server } 2>&1`.strip
282
-
283
- log(:info, "cmd: #{ start_server }")
284
- server_output = `#{ start_server } 2>&1`.strip
285
-
286
- log(:info, "status: #{ $?.exitstatus }")
287
-
288
- t = Time.now.to_f
289
- timeout = 10
290
- i = 0
291
-
292
- loop do
293
- i += 1
294
-
295
- begin
296
- url = "http://localhost:#{ port }"
297
- open(url){|socket| socket.read}
298
- @server = url
299
- @port = port
300
- break
301
- rescue Object => e
302
- if i > 2
303
- log :error, "#{ e.message }(#{ e.class })\n"
304
- log :error, "#{ server_output }\n\n"
305
- end
306
-
307
- if((Time.now.to_f - t) > timeout)
308
- abort("could not start server inside of #{ timeout } seconds ;-/")
309
- else
310
- sleep(rand(0.42))
311
- end
312
- end
313
- end
314
-
315
- break if @server
316
- end
317
-
318
- # barf if server could not be started
319
- #
320
- unless @server
321
- abort("could not start server on any of ports #{ ports.first } .. #{ ports.last }")
322
- else
323
- log(:info, "started server on #{ @server }")
324
- end
325
-
326
- # set assassins to ensure the server daemon never outlives the build script
327
- # no matter how it is killed (even -9)
328
- #
329
- at_exit{
330
- log(:info, "cmd: #{ stop_server }")
331
- `#{ stop_server } >/dev/null 2>&1`
332
- log(:info, "status: #{ $?.exitstatus }")
333
- log(:info, "stopped server #{ @server }")
334
- }
335
-
336
- assassin = <<-__
337
- pid = #{ Process.pid }
338
-
339
- 4242.times do
340
- begin
341
- Process.kill(0, pid)
342
- rescue Object => e
343
- if e.is_a?(Errno::ESRCH)
344
- `#{ stop_server } >/dev/null 2>&1`
345
- Process.kill(-15, pid) rescue nil
346
- sleep(rand + rand)
347
- Process.kill(-9, pid) rescue nil
348
- end
349
- exit
350
- end
351
- sleep(1 + rand)
352
- end
353
- __
354
- IO.binwrite('tmp/build-assassin.rb', assassin)
355
- cmd = "nohup ruby tmp/build-assassin.rb >/dev/null 2>&1 &"
356
- system cmd
357
-
358
- #
359
- @started_at = Time.now
360
- @server
361
- end
362
-
363
- #
364
- def extract_urls!(build_url)
365
- urls = []
366
-
367
- code, body = http_get(build_url)
368
-
369
- unless code == 200
370
- raise "failed to get build urls from #{ build_url }"
371
- end
372
-
373
- @_build = JSON.parse(body)
374
-
375
- unless @_build['urls'].is_a?(Array)
376
- raise "failed to find any build urls at #{ build_url }"
377
- end
378
-
379
- urls = @_build['urls']
380
-
381
- urls.map!{|url| url_for(url)}
382
-
383
- log(:info, "extracted #{ urls.size } urls to build from #{ build_url }")
384
-
385
- @urls = urls
386
- end
387
-
388
- #
389
- def clear_cache!
390
- #spawn "rake tmp:cache:clear"
391
- spawn "rails runner 'Rails.cache.clear'"
392
- end
393
-
394
- #
395
- def precompile_assets!
396
- @asset_dir = File.join(@rails_root, "public/assets")
397
- @asset_tmp = false
398
-
399
- if test(?d, @asset_dir)
400
- @asset_tmp = File.join(@rails_root, "tmp/assets-build-#{ @uuid }")
401
- FileUtils.mv(@asset_dir, @asset_tmp)
402
- end
403
-
404
- spawn "RAILS_ENV=production rake assets:precompile"
405
-
406
- assets = Dir.glob(File.join(@rails_root, 'public/assets/**/**'))
407
-
408
- log(:info, "precompiled #{ assets.size } assets")
409
-
410
- ensure_non_digested_assets_also_exist!(assets)
411
- end
412
-
413
- #
414
- def rsync_public!
415
- spawn "rsync -avz ./public/ #{ @directory }"
416
-
417
- count = 0
418
-
419
- Dir.glob(File.join(@directory, '**/**')).each{ count += 1 }
420
-
421
- log(:info, "rsync'd #{ count } files")
422
-
423
- if @asset_tmp
424
- FileUtils.rm_rf(@asset_dir)
425
- FileUtils.mv(@asset_tmp, @asset_dir)
426
- end
427
- end
428
-
429
- #
430
- def parallel_build!(n = nil)
431
- n ||= @parallel
432
-
433
- stats = {
434
- :success => [], :missing => [], :failure => [],
435
- }
436
-
437
- times = Queue.new
438
-
439
- avg = nil
440
-
441
- Thread.new do
442
- Thread.current.abort_on_exception = true
443
- total = 0.0
444
- n = 0
445
- loop do
446
- while(time = times.pop)
447
- total += time
448
- n += 1
449
- avg = (total / n).round(2)
450
- end
451
- end
452
- end
453
-
454
- a = Time.now.to_f
455
-
456
- _stats =
457
- @urls.threadify(n) do |url|
458
- uri = uri_for(url)
459
- path = path_for(uri)
460
- upath = uri.path
461
- rpath = relative_path(path, :from => @directory)
462
-
463
- _a = Time.now.to_f
464
-
465
- code, body = http_get(uri)
466
-
467
- _b = Time.now.to_f
468
- _e = (_b - _a).round(2)
469
-
470
- times.push(_e)
471
-
472
- label = "#{ code } @ (t̄:#{ avg }s)"
473
-
474
- case code
475
- when 200
476
- write_path(path, body)
477
- log(:info, "#{ label }: #{ upath } -> #{ rpath } (t:#{ _e }s)")
478
- [:success, url]
479
- when 404
480
- log(:error, "#{ label }: #{ upath }")
481
- [:missing, url]
482
- when 500
483
- log(:error, "#{ label }: #{ upath }")
484
- [:failure, url]
485
- else
486
- log(:error, "#{ label }: #{ upath }")
487
- [:failure, url]
488
- end
489
- end
490
-
491
- _stats.each{|stat, url| stats[stat].push(url)}
492
-
493
- b = Time.now.to_f
494
-
495
- borked = false
496
-
497
- if stats[:missing].size > 0
498
- borked = true
499
- log(:error, "missing on #{ stats[:missing].size } urls")
500
- end
501
-
502
- if stats[:failure].size > 0
503
- borked = true
504
- log(:error, "failure on #{ stats[:failure].size } urls")
505
- end
506
-
507
- if borked
508
- exit(1)
509
- end
510
-
511
- elapsed = b - a
512
- n = @urls.size
513
- rps = (n / elapsed).round(2)
514
-
515
- log(:info, "downloaded #{ n } urls at ~#{ rps }/s")
516
-
517
- @urls
518
- end
519
-
520
- #
521
- def finalize!
522
- @finished_at = Time.now
523
- elapsed = (@finished_at.to_f - @started_at.to_f)
524
- log(:info, "build time - #{ hms(elapsed) }")
525
- on_netlify = ENV['DEPLOY_PRIME_URL'].to_s =~ /netlify/
526
- strategy = on_netlify ? 'cp_r' : 'ln_s' # netlify refuses to deploy from a symlink ;-/ # FIXME
527
- build = File.join(@rails_root, 'build')
528
- FileUtils.rm_rf(build)
529
- FileUtils.send(strategy, @directory, build)
530
- log(:info, "preview with `static ./build/` # brew install node-static")
531
- end
532
-
533
- #
534
- def http_get(url)
535
- response = HTTP.get(url.to_s, :follow => true)
536
- body = response.body.to_s rescue ''
537
- code = response.status.code rescue 500
538
- [code, body]
539
- end
540
-
541
- def http_connection
542
- @http_connection ||= (
543
- PersistentHTTP.new({
544
- :url => @server,
545
- :pool_size => (@parallel + 1),
546
- :logger => (@verbose ? @logger : nil),
547
- :pool_timeout => 10,
548
- :warn_timeout => 1,
549
- :force_retry => true,
550
- })
551
- )
552
- end
553
-
554
- def http_get(url)
555
- request = Net::HTTP::Get.new(url)
556
- response = http_connection.request(request)
557
-
558
- if response.is_a?(Net::HTTPRedirection)
559
- location = response['Location']
560
- if location.to_s == url.to_s
561
- log(:fatal, "circular redirection on #{ url }")
562
- exit(1)
563
- end
564
- return http_get(location)
565
- end
566
-
567
- code = response.code.to_i rescue 500
568
- body = response.body.to_s rescue ''
569
-
570
- [code, body]
571
- end
572
-
573
- #
574
- def path_for(url)
575
- uri = uri_for(url)
576
-
577
- case
578
- when uri.path=='/' || uri.path=='.'
579
- path = File.join(@directory, 'index.html')
580
- else
581
- path = File.join(@directory, uri.path)
582
- dirname, basename = File.split(path)
583
- base, ext = basename.split('.', 2)
584
- if ext.nil?
585
- path = File.join(path, 'index.html')
586
- end
587
- end
588
- path
589
- end
590
-
591
- #
592
- def write_path(path, body)
593
- FileUtils.mkdir_p(File.dirname(path))
594
- IO.binwrite(path, body)
595
- end
596
-
597
- #
598
- def ensure_non_digested_assets_also_exist!(assets)
599
- re = /(-{1}[a-z0-9]{32}*\.{1}){1}/
600
-
601
- assets.each do |file|
602
- next if File.directory?(file) || file !~ re
603
- source = file.split('/')
604
- source.push(source.pop.gsub(re, '.'))
605
- non_digested = File.join(source)
606
- #log(:debug, "asset: #{ file } -> #{ non_digested }")
607
- FileUtils.ln(file, non_digested)
608
- end
609
- end
610
-
611
- #
612
- def url_for(url)
613
- uri = URI.parse(url.to_s)
614
-
615
- if uri.absolute?
616
- uri.to_s
617
- else
618
- relative_uri = URI.parse(@server)
619
- relative_uri.path = absolute_path_for(uri.path)
620
- relative_uri.query = uri.query
621
- relative_uri.fragment = uri.fragment
622
- relative_uri.to_s
623
- end
624
- end
625
-
626
- #
627
- def uri_for(url)
628
- uri = url.is_a?(URI) ? url : URI.parse(url.to_s)
629
- end
630
-
631
- #
632
- def to_s
633
- @directory.to_s
634
- end
635
-
636
- #
637
- def log(level, *args, &block)
638
- @logger.send(level, *args, &block)
639
- end
640
-
641
- #
642
- def hms(seconds)
643
- return unless seconds
644
- "%02d:%02d:%02d" % hours_minutes_seconds(seconds)
645
- end
646
-
647
- #
648
- def hours_minutes_seconds(seconds)
649
- return unless seconds
650
- seconds = Float(seconds).to_i
651
- hours, seconds = seconds.divmod(3600)
652
- minutes, seconds = seconds.divmod(60)
653
- [hours.to_i, minutes.to_s, seconds]
654
- end
655
-
656
- #
657
- def stopwatch(&block)
658
- a = Time.now
659
- result = block.call
660
- b = Time.now
661
- [result, b.to_f - a.to_f]
662
- end
663
-
664
- #
665
- def port_open?(port, options = {})
666
- seconds = options[:timeout] || 1
667
- ip = options[:ip] || '0.0.0.0'
668
-
669
- Timeout::timeout(seconds) do
670
- begin
671
- TCPSocket.new(ip, port).close
672
- false
673
- rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
674
- true
675
- rescue Object
676
- false
677
- end
678
- end
679
- rescue Timeout::Error
680
- false
681
- end
682
-
683
- #
684
- def paths_for(*args)
685
- path = args.flatten.compact.join('/')
686
- path.gsub!(%r|[.]+/|, '/')
687
- path.squeeze!('/')
688
- path.sub!(%r|^/|, '')
689
- path.sub!(%r|/$|, '')
690
- paths = path.split('/')
691
- end
692
-
693
- #
694
- def absolute_path_for(*args)
695
- path = ('/' + paths_for(*args).join('/')).squeeze('/')
696
- path unless path.strip.empty?
697
- end
698
-
699
- #
700
- def relative_path_for(*args)
701
- path = absolute_path_for(*args).sub(%r{^/+}, '')
702
- path unless path.strip.empty?
703
- end
704
-
705
- #
706
- def relative_path(path, *args)
707
- options = args.last.is_a?(Hash) ? args.pop : {}
708
- path = path.to_s
709
- relative = args.shift || options[:relative] || options[:to] || options[:from]
710
- if relative
711
- Pathname.new(path).relative_path_from(Pathname.new(relative.to_s)).to_s
712
- else
713
- relative_path_for(path)
714
- end
715
- end
716
-
717
- #
718
- def spawn(command)
719
- oe = `#{ command } 2>&1`
720
-
721
- unless $? == 0
722
- msg = "command(#{ command }) failed with (#{ $? })"
723
- log(:error, msg)
724
- raise(msg)
725
- end
726
-
727
- oe
728
- end
729
- end
214
+ #
215
+ def parse_args!
216
+ @args = ARGV.map{|arg| "#{ arg }"}
217
+ end
218
+
219
+ #
220
+ def parse_opts!
221
+ @opts = Hash.new
222
+
223
+ CLI.opts.each do |opt, arg|
224
+ key, val = opt.split('--').last, arg
225
+ @opts[key.to_s.to_sym] = (val == '' ? true : val)
226
+ end
227
+
228
+ @opts
229
+ end
230
+
231
+ #
232
+ def usage!
233
+ lines = CLI.usage.strip.split(/\n/)
234
+ n = lines[1].to_s.scan(/^\s+/).size
235
+ indent = ' ' * n
236
+ re = /^#{ Regexp.escape(indent) }/
237
+ usage = lines.map{|line| line.gsub(re, '')}.join("\n")
238
+ STDERR.puts(usage)
239
+ end
240
+
241
+ #
242
+ def mkdir!
243
+ FileUtils.rm_rf(@directory)
244
+ FileUtils.mkdir_p(@directory)
245
+ log(:info, "directory: #{ @directory }")
246
+ end
247
+
248
+ #
249
+ def start_server!
250
+ @url =
251
+ nil
252
+
253
+ @port =
254
+ nil
255
+
256
+ ports =
257
+ (2000 .. 9000).to_a
258
+
259
+ start_server, stop_server = nil
260
+
261
+ @server.setup!
262
+
263
+ ports.each do |port|
264
+ next unless port_open?(port)
265
+
266
+ start_server = @server.start_command_for(port)
267
+ stop_server = @server.stop_command_for(port)
268
+
269
+ `#{ stop_server } 2>&1`.strip
270
+
271
+ log(:info, "cmd: #{ start_server }")
272
+ server_output = `#{ start_server } 2>&1`.strip
273
+
274
+ log(:info, "status: #{ $?.exitstatus }")
275
+
276
+ t = Time.now.to_f
277
+ timeout = 10
278
+ i = 0
279
+ url = nil
280
+
281
+ loop do
282
+ i += 1
283
+
284
+ begin
285
+ url = "http://localhost:#{ port }"
286
+ open(url){|socket| socket.read}
287
+ @url = url
288
+ @port = port
289
+ break
290
+ rescue Object => e
291
+ if i > 2
292
+ log :error, "url: #{ url } ->"
293
+ log :error, "#{ e.message }(#{ e.class })\n"
294
+ log :error, "#{ server_output }\n\n"
295
+ end
296
+
297
+ if((Time.now.to_f - t) > timeout)
298
+ abort("could not start server inside of #{ timeout } seconds via\n\n#{ start_server }\n\n;-/")
299
+ else
300
+ sleep(rand(0.42))
301
+ end
302
+ end
303
+ end
304
+
305
+ break if @url
306
+ end
307
+
308
+ # barf if server could not be started
309
+ #
310
+ unless @url
311
+ abort("could not start server on any of ports #{ ports.first } .. #{ ports.last }")
312
+ else
313
+ log(:info, "started server on #{ @url }")
314
+ end
315
+
316
+ # set assassins to ensure the server daemon never outlives the build script
317
+ # no matter how it is killed (even -9)
318
+ #
319
+ at_exit{
320
+ log(:info, "cmd: #{ stop_server }")
321
+ `#{ stop_server } >/dev/null 2>&1`
322
+ log(:info, "status: #{ $?.exitstatus }")
323
+ log(:info, "stopped server #{ @url }")
324
+ @server.cleanup!(:port => @port)
325
+ }
326
+
327
+ assassin = <<-__
328
+ pid = #{ Process.pid }
329
+
330
+ 4242.times do
331
+ begin
332
+ Process.kill(0, pid)
333
+ rescue Object => e
334
+ if e.is_a?(Errno::ESRCH)
335
+ `#{ stop_server } >/dev/null 2>&1`
336
+ Process.kill(-15, pid) rescue nil
337
+ sleep(rand + rand)
338
+ Process.kill(-9, pid) rescue nil
339
+ end
340
+ exit
341
+ end
342
+ sleep(1 + rand)
343
+ end
344
+ __
345
+ IO.binwrite('tmp/build-assassin.rb', assassin)
346
+ cmd = "nohup ruby tmp/build-assassin.rb >/dev/null 2>&1 &"
347
+ system cmd
348
+
349
+ #
350
+ @started_at = Time.now
351
+ @url
352
+ end
353
+
354
+ #
355
+ def extract_urls!(build_url)
356
+ urls = []
357
+
358
+ code, body = http_get(build_url)
359
+
360
+ unless code == 200
361
+ raise "failed to get build urls from #{ build_url }"
362
+ end
363
+
364
+ @_build = JSON.parse(body)
365
+
366
+ unless @_build['urls'].is_a?(Array)
367
+ raise "failed to find any build urls at #{ build_url } - edit ./config/rails_build.rb # see README"
368
+ end
369
+
370
+ urls = @_build['urls']
371
+
372
+ urls.map!{|url| url_for(url)}
373
+
374
+ log(:info, "extracted #{ urls.size } url(s) to build from #{ build_url }")
375
+
376
+ @urls = urls
377
+ end
378
+
379
+ #
380
+ def clear_cache!
381
+ #spawn "rake tmp:cache:clear"
382
+ spawn "rails runner 'Rails.cache.clear'"
383
+ end
384
+
385
+ #
386
+ def precompile_assets!
387
+ @asset_dir = File.join(@rails_root, "public/assets")
388
+ @asset_tmp = false
389
+
390
+ if test(?d, @asset_dir)
391
+ @asset_tmp = File.join(@rails_root, "tmp/assets-build-#{ @uuid }")
392
+ FileUtils.mv(@asset_dir, @asset_tmp)
393
+ end
394
+
395
+ spawn "RAILS_ENV=production rake assets:precompile"
396
+
397
+ assets = Dir.glob(File.join(@rails_root, 'public/assets/**/**'))
398
+
399
+ log(:info, "precompiled #{ assets.size } assets")
400
+
401
+ ensure_non_digested_assets_also_exist!(assets)
402
+ end
403
+
404
+ #
405
+ def rsync_public!
406
+ spawn "rsync -avz ./public/ #{ @directory }"
407
+
408
+ count = 0
409
+
410
+ Dir.glob(File.join(@directory, '**/**')).each{ count += 1 }
411
+
412
+ log(:info, "rsync'd #{ count } files")
413
+
414
+ if @asset_tmp
415
+ FileUtils.rm_rf(@asset_dir)
416
+ FileUtils.mv(@asset_tmp, @asset_dir)
417
+ end
418
+ end
419
+
420
+ #
421
+ def parallel_build!(n = nil)
422
+ n ||= @parallel
423
+
424
+ stats = {
425
+ :success => [], :missing => [], :failure => [],
426
+ }
427
+
428
+ times = Queue.new
429
+
430
+ avg = nil
431
+
432
+ Thread.new do
433
+ Thread.current.abort_on_exception = true
434
+ total = 0.0
435
+ n = 0
436
+ loop do
437
+ while(time = times.pop)
438
+ total += time
439
+ n += 1
440
+ avg = (total / n).round(2)
441
+ end
442
+ end
443
+ end
444
+
445
+ a = Time.now.to_f
446
+
447
+ _stats =
448
+ @urls.threadify(n) do |url|
449
+ uri = uri_for(url)
450
+ path = path_for(uri)
451
+ upath = uri.path
452
+ rpath = relative_path(path, :from => @directory)
453
+
454
+ _a = Time.now.to_f
455
+
456
+ code, body = http_get(uri)
457
+
458
+ _b = Time.now.to_f
459
+ _e = (_b - _a).round(2)
460
+
461
+ times.push(_e)
462
+
463
+ label = "#{ code } @ (t̄:#{ avg }s)"
464
+
465
+ case code
466
+ when 200
467
+ write_path(path, body)
468
+ log(:info, "#{ label }: #{ upath } -> #{ rpath } (t:#{ _e }s)")
469
+ [:success, url]
470
+ when 404
471
+ log(:error, "#{ label }: #{ upath }")
472
+ [:missing, url]
473
+ when 500
474
+ log(:error, "#{ label }: #{ upath }")
475
+ [:failure, url]
476
+ else
477
+ log(:error, "#{ label }: #{ upath }")
478
+ [:failure, url]
479
+ end
480
+ end
481
+
482
+ _stats.each{|stat, url| stats[stat].push(url)}
483
+
484
+ b = Time.now.to_f
485
+
486
+ borked = false
487
+
488
+ if stats[:missing].size > 0
489
+ borked = true
490
+ log(:error, "missing on #{ stats[:missing].size } urls")
491
+ end
492
+
493
+ if stats[:failure].size > 0
494
+ borked = true
495
+ log(:error, "failure on #{ stats[:failure].size } urls")
496
+ end
497
+
498
+ if borked
499
+ exit(1)
500
+ end
501
+
502
+ elapsed = b - a
503
+ n = @urls.size
504
+ rps = (n / elapsed).round(2)
505
+
506
+ log(:info, "downloaded #{ n } urls at ~#{ rps }/s")
507
+
508
+ @urls
509
+ end
510
+
511
+ #
512
+ def finalize!
513
+ @finished_at = Time.now
514
+ elapsed = (@finished_at.to_f - @started_at.to_f)
515
+ log(:info, "build time - #{ hms(elapsed) }")
516
+ on_netlify = ENV['DEPLOY_PRIME_URL'].to_s =~ /netlify/
517
+ strategy = on_netlify ? 'cp_r' : 'ln_s' # netlify refuses to deploy from a symlink ;-/ # FIXME
518
+ build = File.join(@rails_root, 'build')
519
+ FileUtils.rm_rf(build)
520
+ FileUtils.send(strategy, @directory, build)
521
+ log(:info, "preview with `static ./build/` # brew install node-static")
522
+ end
523
+
524
+ def http_connection
525
+ @http_connection ||= (
526
+ PersistentHTTP.new({
527
+ :url => @url,
528
+ :pool_size => (@parallel + 1),
529
+ :logger => (@verbose ? @logger : nil),
530
+ :pool_timeout => 10,
531
+ :warn_timeout => 1,
532
+ :force_retry => true,
533
+ })
534
+ )
535
+ end
536
+
537
+ def http_get(url)
538
+ request = Net::HTTP::Get.new(url)
539
+ response = http_connection.request(request)
540
+
541
+ if response.is_a?(Net::HTTPRedirection)
542
+ location = response['Location']
543
+ if location.to_s == url.to_s
544
+ log(:fatal, "circular redirection on #{ url }")
545
+ exit(1)
546
+ end
547
+ return http_get(location)
548
+ end
549
+
550
+ code = response.code.to_i rescue 500
551
+ body = response.body.to_s rescue ''
552
+
553
+ [code, body]
554
+ end
555
+
556
+ #
557
+ def to_s
558
+ @directory.to_s
559
+ end
560
+
561
+ #
562
+ def log(level, *args, &block)
563
+ @logger.send(level, *args, &block)
564
+ end
565
+
566
+ #
567
+ def path_for(url)
568
+ uri = uri_for(url)
569
+
570
+ case
571
+ when uri.path=='/' || uri.path=='.'
572
+ path = File.join(@directory, 'index.html')
573
+ else
574
+ path = File.join(@directory, uri.path)
575
+ dirname, basename = File.split(path)
576
+ base, ext = basename.split('.', 2)
577
+ if ext.nil?
578
+ path = File.join(path, 'index.html')
579
+ end
580
+ end
581
+ path
582
+ end
583
+
584
+ #
585
+ def write_path(path, body)
586
+ FileUtils.mkdir_p(File.dirname(path))
587
+ IO.binwrite(path, body)
588
+ end
589
+
590
+ #
591
+ def ensure_non_digested_assets_also_exist!(assets)
592
+ re = /(-{1}[a-z0-9]{32}*\.{1}){1}/
593
+
594
+ assets.each do |file|
595
+ next if File.directory?(file) || file !~ re
596
+ source = file.split('/')
597
+ source.push(source.pop.gsub(re, '.'))
598
+ non_digested = File.join(source)
599
+ #log(:debug, "asset: #{ file } -> #{ non_digested }")
600
+ FileUtils.ln(file, non_digested)
601
+ end
602
+ end
603
+
604
+ #
605
+ def url_for(url)
606
+ uri = URI.parse(url.to_s)
607
+
608
+ if uri.absolute?
609
+ uri.to_s
610
+ else
611
+ relative_uri = URI.parse(@url)
612
+ relative_uri.path = absolute_path_for(uri.path)
613
+ relative_uri.query = uri.query
614
+ relative_uri.fragment = uri.fragment
615
+ relative_uri.to_s
616
+ end
617
+ end
618
+
619
+ #
620
+ def uri_for(url)
621
+ uri = url.is_a?(URI) ? url : URI.parse(url.to_s)
622
+ end
623
+
624
+ #
625
+ def hms(seconds)
626
+ return unless seconds
627
+ "%02d:%02d:%02d" % hours_minutes_seconds(seconds)
628
+ end
629
+
630
+ #
631
+ def hours_minutes_seconds(seconds)
632
+ return unless seconds
633
+ seconds = Float(seconds).to_i
634
+ hours, seconds = seconds.divmod(3600)
635
+ minutes, seconds = seconds.divmod(60)
636
+ [hours.to_i, minutes.to_s, seconds]
637
+ end
638
+
639
+ #
640
+ def stopwatch(&block)
641
+ a = Time.now
642
+ result = block.call
643
+ b = Time.now
644
+ [result, b.to_f - a.to_f]
645
+ end
646
+
647
+ #
648
+ def port_open?(port, options = {})
649
+ seconds = options[:timeout] || 1
650
+ ip = options[:ip] || '0.0.0.0'
651
+
652
+ Timeout::timeout(seconds) do
653
+ begin
654
+ TCPSocket.new(ip, port).close
655
+ false
656
+ rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
657
+ true
658
+ rescue Object
659
+ false
660
+ end
661
+ end
662
+ rescue Timeout::Error
663
+ false
664
+ end
665
+
666
+ #
667
+ def paths_for(*args)
668
+ path = args.flatten.compact.join('/')
669
+ path.gsub!(%r|[.]+/|, '/')
670
+ path.squeeze!('/')
671
+ path.sub!(%r|^/|, '')
672
+ path.sub!(%r|/$|, '')
673
+ paths = path.split('/')
674
+ end
675
+
676
+ #
677
+ def absolute_path_for(*args)
678
+ path = ('/' + paths_for(*args).join('/')).squeeze('/')
679
+ path unless path.strip.empty?
680
+ end
681
+
682
+ #
683
+ def relative_path_for(*args)
684
+ path = absolute_path_for(*args).sub(%r{^/+}, '')
685
+ path unless path.strip.empty?
686
+ end
687
+
688
+ #
689
+ def relative_path(path, *args)
690
+ options = args.last.is_a?(Hash) ? args.pop : {}
691
+ path = path.to_s
692
+ relative = args.shift || options[:relative] || options[:to] || options[:from]
693
+ if relative
694
+ Pathname.new(path).relative_path_from(Pathname.new(relative.to_s)).to_s
695
+ else
696
+ relative_path_for(path)
697
+ end
698
+ end
699
+
700
+ #
701
+ def spawn(command)
702
+ oe = `#{ command } 2>&1`
703
+
704
+ unless $? == 0
705
+ msg = "command(#{ command }) failed with (#{ $? })"
706
+ log(:error, msg)
707
+ raise(msg)
708
+ end
709
+
710
+ oe
711
+ end
712
+ end
713
+
714
+ #
715
+ class Server
716
+ def Server.for(type, cli, *args, &block)
717
+ type = type.to_s.strip.downcase
718
+
719
+ case type
720
+ when /puma/
721
+ Puma.new(cli, *args, &block)
722
+ when /passenger/
723
+ Passenger.new(cli, *args, &block)
724
+ else
725
+ nil
726
+ end
727
+ end
728
+
729
+ def initialize(cli)
730
+ @cli = cli
731
+ end
732
+
733
+ def setup!
734
+ nil
735
+ end
736
+
737
+ class Puma < Server
738
+ def initialize(cli)
739
+ @cli = cli
740
+
741
+ @env = @cli.env
742
+ @directory = @cli.directory
743
+ @rails_root = @cli.rails_root
744
+ @parallel = @cli.parallel
745
+
746
+ @puma = "bundle exec puma"
747
+ @pumactl = "bundle exec pumactl"
748
+
749
+ @pidfile = @cli.relative_path(File.join(@directory, ".puma-pid.txt"), :from => @cli.rails_root)
750
+ @statefile = @cli.relative_path(File.join(@directory, ".puma-state.txt"), :from => @cli.rails_root)
751
+ end
752
+
753
+ def start_command_for(port)
754
+ %W[
755
+ #{ @puma }
756
+ --pidfile=#{ @pidfile }
757
+ --state=#{ @statefile }
758
+ --port=#{ port }
759
+ --environment=#{ @env }
760
+ --workers=#{ @parallel }
761
+ --config=/dev/null
762
+ --threads=1:1
763
+ --preload
764
+ --daemon
765
+ ./config.ru
766
+ ].join(' ').strip
767
+ end
768
+
769
+ def stop_command_for(port)
770
+ %W[
771
+ #{ @pumactl }
772
+ --pidfile=#{ @pidfile }
773
+ --state=#{ @statefile }
774
+ stop
775
+ ].join(' ').strip
776
+ end
777
+
778
+ def cleanup!(*args)
779
+ nil
780
+ end
781
+ end
782
+
783
+ class Passenger < Server
784
+ def initialize(cli)
785
+ @cli = cli
786
+
787
+ @env = @cli.env
788
+ @directory = @cli.directory
789
+ @rails_root = @cli.rails_root
790
+ @parallel = @cli.parallel
791
+
792
+ @passenger = "bundle exec passenger"
793
+ end
794
+
795
+ def start_command_for(port)
796
+ "#{ @passenger } start --daemonize --environment #{ @env } --port #{ port } --max-pool-size #{ @parallel }"
797
+ end
798
+
799
+ def stop_command_for(port)
800
+ "#{ @passenger } stop --port #{ port }"
801
+ end
802
+
803
+ def cleanup!(*args)
804
+ options = args.last.is_a?(Hash) ? args.pop : {}
805
+ port = options[:port]
806
+ FileUtils.rm_rf("#{ @rails_root }/tmp/pids/passenger.#{ port }.pid")
807
+ end
808
+
809
+ def setup!
810
+ begin
811
+ require 'phusion_passenger'
812
+ rescue LoadError => le
813
+ abort "please add `gem 'passenger'` to your Gemfile"
814
+ end
815
+ end
816
+ end
817
+ end
730
818
  end