shuttlebay 0.1.4-aarch64-linux → 0.2.1-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +77 -6
- data/docs/guide.md +323 -0
- data/lib/shuttlebay/cache.rb +61 -0
- data/lib/shuttlebay/engine.rb +25 -0
- data/lib/shuttlebay/handler.rb +8 -6
- data/lib/shuttlebay/logger.rb +5 -0
- data/lib/shuttlebay/railtie.rb +42 -1
- data/lib/shuttlebay/runtime.rb +45 -3
- data/lib/shuttlebay/server.rb +80 -29
- data/lib/shuttlebay/shuttlebay.so +0 -0
- data/lib/shuttlebay/version.rb +1 -1
- data/lib/shuttlebay/worker.rb +2 -2
- data/lib/shuttlebay.rb +1 -0
- metadata +21 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: fe674b008d997e54f538c829df7114f774b0b69dc981ce9b8f0a3ac646a9670f
|
|
4
|
+
data.tar.gz: 7456ad62c67d55985b15776f51f3b9568058d6cb475bf15cdd5e6c6893850c39
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 313fc004e0fc3f19cd9b2357fa89819ac73096f5199037c604df810ef4012262dc16f9f331418d16d596865548269d88ef54cda53ccf03209299ce05233a0c0d
|
|
7
|
+
data.tar.gz: 9dd9a29f436328435136750f05b6e3976d92a9b724e3329e37ec64d442aa9331410a6a37c08a96d393f1f4b2b799058b8150108a4eb0d1dc58e188aa47dd715b
|
data/README.md
CHANGED
|
@@ -15,6 +15,10 @@ flowchart TD
|
|
|
15
15
|
master --> w1["worker 1<br/>threads"]
|
|
16
16
|
```
|
|
17
17
|
|
|
18
|
+
**New here?** [docs/guide.md](docs/guide.md) walks a Rails app from install to
|
|
19
|
+
production: the manifest, coming from Puma, moving `Rack::Attack` rules,
|
|
20
|
+
caching, TLS and HTTP/2, Docker and Kamal, several hosts.
|
|
21
|
+
|
|
18
22
|
## Install
|
|
19
23
|
|
|
20
24
|
```ruby
|
|
@@ -57,9 +61,59 @@ bundle exec mothership
|
|
|
57
61
|
| Real client IP | `RemoteIp` + proxy config | `REMOTE_ADDR` resolved (Forwarded / PROXY protocol) |
|
|
58
62
|
| Load balancing to workers | kernel accept / `wait_for_less_busy_worker` | idle-thread checkout; queueing in Mothership |
|
|
59
63
|
| Request body buffering | App server | Mothership reads the whole body before claiming a thread |
|
|
64
|
+
| `ETag` / `If-None-Match` → 304 | `Rack::ETag`, `Rack::ConditionalGet` | Mothership hashes complete bodies and answers 304 |
|
|
65
|
+
| IP allowlists, rate limits | `Rack::Attack` | `ip_allowlist`, `rate_limits`, `trusted_proxies` in the manifest |
|
|
66
|
+
| Page / API caching | CDN, `Rack::Cache` | `[mothership.response_cache]`, purged by tag |
|
|
67
|
+
|
|
68
|
+
When Rails is loaded, the bundled Railtie removes the middleware Mothership
|
|
69
|
+
makes redundant:
|
|
70
|
+
|
|
71
|
+
- `Rack::Sendfile`: the engine offloads `to_path` bodies itself.
|
|
72
|
+
- `ActionDispatch::Static`: when Mothership serves the app's `public/` as a
|
|
73
|
+
static dir (`MS_STATIC_ROOTS`). `config.public_file_server.headers` it
|
|
74
|
+
does not send (per `MS_STATIC_HEADERS`) are logged at boot: move them to
|
|
75
|
+
the static dir's `headers` in the manifest.
|
|
76
|
+
- `Rack::ETag` and `Rack::ConditionalGet`: when Mothership validates
|
|
77
|
+
conditionals (`MS_CONDITIONAL_GET=1`), so bodies are not hashed twice.
|
|
78
|
+
|
|
79
|
+
Other Rack apps (Sinatra, Roda, Hanami) ask the same in `config.ru`;
|
|
80
|
+
`Shuttlebay.offloaded?` is false under any other server:
|
|
81
|
+
|
|
82
|
+
```ruby
|
|
83
|
+
use Rack::ETag unless Shuttlebay.offloaded?(:conditional_get)
|
|
84
|
+
use Rack::ConditionalGet unless Shuttlebay.offloaded?(:conditional_get)
|
|
85
|
+
use Rack::Static, urls: [""], root: "public", cascade: true unless Shuttlebay.offloaded?(:static, "public")
|
|
86
|
+
use Rack::Sendfile unless Shuttlebay.offloaded?(:sendfile)
|
|
87
|
+
run App
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Rate limiting and allowlists in `Rack::Attack` are app configuration: drop
|
|
91
|
+
them yourself once the manifest enforces the same rules.
|
|
92
|
+
|
|
93
|
+
## Caching at the edge
|
|
94
|
+
|
|
95
|
+
Mothership's response cache keeps entries until their data changes when the
|
|
96
|
+
app tags responses with what they show and purges the tags on change:
|
|
97
|
+
|
|
98
|
+
```ruby
|
|
99
|
+
class BooksController < ApplicationController
|
|
100
|
+
def show
|
|
101
|
+
@book = Book.find(params[:id])
|
|
102
|
+
Shuttlebay::Cache.tag(response.headers, "book-#{@book.id}", "books")
|
|
103
|
+
expires_in 1.hour, public: true # or a manifest rule
|
|
104
|
+
end
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
class Book < ApplicationRecord
|
|
108
|
+
after_commit { Shuttlebay::Cache.purge(tags: ["book-#{id}", "books"]) }
|
|
109
|
+
end
|
|
110
|
+
```
|
|
60
111
|
|
|
61
|
-
|
|
62
|
-
|
|
112
|
+
`purge` also takes `urls:` (path and query), `prefixes:`, `all: true`, and
|
|
113
|
+
`soft: true` (mark stale and revalidate instead of dropping). It posts to
|
|
114
|
+
`MS_ADMIN_URL`, returns the number purged, is a no-op returning `nil` where
|
|
115
|
+
there is no Mothership, and raises `Shuttlebay::Error` when Mothership refuses
|
|
116
|
+
(e.g. no cache configured) or a network error when it is unreachable.
|
|
63
117
|
|
|
64
118
|
## Hooks
|
|
65
119
|
|
|
@@ -67,7 +121,7 @@ engine offloads `to_path` bodies itself.
|
|
|
67
121
|
|
|
68
122
|
```ruby
|
|
69
123
|
Shuttlebay.before_fork { ActiveRecord::Base.connection_pool.disconnect! }
|
|
70
|
-
Shuttlebay.on_worker_boot { |index| Rails.logger.info("worker #{index} up") }
|
|
124
|
+
Shuttlebay.on_worker_boot { |index, ship| Rails.logger.info("#{ship} worker #{index} up") }
|
|
71
125
|
Shuttlebay.on_worker_shutdown { |index| Rails.logger.info("worker #{index} drained") }
|
|
72
126
|
Shuttlebay.out_of_band { GC.start } # runs when a worker has no request in flight
|
|
73
127
|
run Rails.application
|
|
@@ -152,18 +206,35 @@ For HTTPS, run Mothership with a `tls` bind and the app as a docking upstream
|
|
|
152
206
|
- **Large uploads:** bodies over 1 MiB are spooled to an unlinked temp file
|
|
153
207
|
(`rack.input` is a `Tempfile`). Memory stays bounded however large uploads
|
|
154
208
|
are allowed to be.
|
|
209
|
+
- **Warmup:** the master calls `Process.warmup` once the app is loaded, so
|
|
210
|
+
workers fork from a compacted heap. `Shuttlebay.warmup = false` in
|
|
211
|
+
`config.ru` skips it.
|
|
212
|
+
- **Forks:** a `fork_from` bay in the manifest is served by this master: it
|
|
213
|
+
loads that bay's app too and forks its workers, sharing the preloaded heap.
|
|
214
|
+
A fork app that fails to load is logged and skipped. Each worker serves one
|
|
215
|
+
bay: it holds only that bay's socket, logs and sees (`MS_SHIP`) its name,
|
|
216
|
+
and hooks get it as their second argument (`|index, ship|`; hooks taking
|
|
217
|
+
only the index keep working).
|
|
155
218
|
- **Signals:** on `TERM`, workers stop accepting, drop idle links, and finish
|
|
156
219
|
in-flight requests within `MS_BAY_DRAIN_TIMEOUT` (Mothership passes the
|
|
157
220
|
bay's `request_timeout`). A dead worker is re-forked with backoff.
|
|
158
221
|
- **Deploys:** `kill -USR1` the mothership process to roll the bay to new code
|
|
159
222
|
with no dropped requests (phased restart).
|
|
160
|
-
- **TLS:** terminated by Mothership on a `tls` bind
|
|
161
|
-
`rack.url_scheme = "https"`.
|
|
223
|
+
- **TLS:** terminated by Mothership on a `tls` bind (guardship 0.2.1 and
|
|
224
|
+
later ship with TLS); the app sees `rack.url_scheme = "https"`.
|
|
225
|
+
- **HTTP/2:** browsers get it from Mothership's `tls` binds (ALPN), or from
|
|
226
|
+
the CDN or proxy in front; cleartext binds also take HTTP/2 with prior
|
|
227
|
+
knowledge. The app is unaffected: every request reaches it the same way,
|
|
228
|
+
as a Rack env, whatever the client spoke.
|
|
162
229
|
- **Logs:** JSON lines on stdout, which Mothership passes through.
|
|
163
230
|
|
|
164
231
|
## Environment (set by Mothership)
|
|
165
232
|
|
|
166
|
-
`MS_SOCKET_PATH`, `MS_SHIP`, `MS_BAY_WORKERS`, `MS_BAY_THREADS`, `MS_BAY_DRAIN_TIMEOUT
|
|
233
|
+
`MS_SOCKET_PATH`, `MS_SHIP`, `MS_BAY_WORKERS`, `MS_BAY_THREADS`, `MS_BAY_DRAIN_TIMEOUT`,
|
|
234
|
+
`MS_STATIC_ROOTS` (directories Mothership serves at `/`, one per line),
|
|
235
|
+
`MS_STATIC_HEADERS` (the headers their files carry, JSON by path),
|
|
236
|
+
`MS_CONDITIONAL_GET=1` (Mothership answers conditionals),
|
|
237
|
+
`MS_ADMIN_URL` (admin endpoint for cache purges).
|
|
167
238
|
|
|
168
239
|
## Development
|
|
169
240
|
|
data/docs/guide.md
ADDED
|
@@ -0,0 +1,323 @@
|
|
|
1
|
+
# Running a Rails app on Shuttlebay
|
|
2
|
+
|
|
3
|
+
From `rails new` (or an app on Puma) to production behind Mothership. Any
|
|
4
|
+
Rack app works the same way; the Rails-specific parts are marked.
|
|
5
|
+
|
|
6
|
+
Mothership is the HTTP server: it terminates connections, serves `public/`,
|
|
7
|
+
compresses, limits, caches and routes. Shuttlebay runs your app on threads
|
|
8
|
+
that Mothership hands finished requests to. There is no Puma, no Thruster
|
|
9
|
+
and no nginx left in the request path.
|
|
10
|
+
|
|
11
|
+
## 1. Install
|
|
12
|
+
|
|
13
|
+
```ruby
|
|
14
|
+
# Gemfile
|
|
15
|
+
gem "shuttlebay"
|
|
16
|
+
gem "guardship" # the mothership binary, prebuilt for macOS, Linux, FreeBSD 15+
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
bundle install
|
|
21
|
+
bundle exec mothership --help
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Keep Puma in the Gemfile if you like: nothing below removes it, and
|
|
25
|
+
`rails server` without `-u` still uses it.
|
|
26
|
+
|
|
27
|
+
## 2. Development and system tests
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
bin/rails server -u shuttlebay
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
```ruby
|
|
34
|
+
# test/application_system_test_case.rb
|
|
35
|
+
require "shuttlebay/capybara"
|
|
36
|
+
Capybara.server = :shuttlebay
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Both start a Mothership attached to the app, so what you test is the stack
|
|
40
|
+
you deploy.
|
|
41
|
+
|
|
42
|
+
## 3. The manifest
|
|
43
|
+
|
|
44
|
+
Mothership reads `ship-manifest.toml` from the directory it starts in:
|
|
45
|
+
|
|
46
|
+
```toml
|
|
47
|
+
[mothership]
|
|
48
|
+
compression = true
|
|
49
|
+
|
|
50
|
+
[mothership.bind]
|
|
51
|
+
http = "0.0.0.0:3000"
|
|
52
|
+
|
|
53
|
+
[[mothership.static_dirs]]
|
|
54
|
+
path = "./public"
|
|
55
|
+
prefix = "/"
|
|
56
|
+
headers = { "cache-control" = "public, max-age=31536000" }
|
|
57
|
+
|
|
58
|
+
[[bays.http]]
|
|
59
|
+
name = "web"
|
|
60
|
+
command = "bundle"
|
|
61
|
+
args = ["exec", "shuttlebay", "config.ru"]
|
|
62
|
+
env = { RAILS_ENV = "production" }
|
|
63
|
+
workers = "auto"
|
|
64
|
+
threads = 5
|
|
65
|
+
routes = [{ bind = "http", pattern = "/.*" }]
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
bundle exec mothership clearance # validate the manifest
|
|
70
|
+
bundle exec mothership chart # what is routed where
|
|
71
|
+
bundle exec mothership # run
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
- `workers = "auto"` forks one worker per CPU the process may use, as many as
|
|
75
|
+
memory affords (512 MiB each); a 1 CPU or 1 GiB container serves from one
|
|
76
|
+
process. A number, or `"$WEB_CONCURRENCY"`, works too.
|
|
77
|
+
- `threads` is the threads per worker, each with its own docking link to
|
|
78
|
+
Mothership. A request goes only to an idle thread; when all are busy it
|
|
79
|
+
waits in Mothership (`queue_timeout`, 30 s, then `503`).
|
|
80
|
+
- The static dir serves `public/` before any request reaches Rails, `/about`
|
|
81
|
+
from `about.html` and precompressed `.br`/`.gz` files included. The
|
|
82
|
+
Railtie sees that and drops `ActionDispatch::Static`. Give it the headers
|
|
83
|
+
your `config.public_file_server.headers` sets: the Railtie logs a warning at
|
|
84
|
+
boot for any it does not see there.
|
|
85
|
+
|
|
86
|
+
## 4. Coming from Puma
|
|
87
|
+
|
|
88
|
+
| `config/puma.rb` | With Shuttlebay |
|
|
89
|
+
|---|---|
|
|
90
|
+
| `workers ENV.fetch("WEB_CONCURRENCY")` | `workers = "$WEB_CONCURRENCY"` (or `"auto"`) |
|
|
91
|
+
| `threads n, n` / `RAILS_MAX_THREADS` | `threads = "$RAILS_MAX_THREADS"` |
|
|
92
|
+
| `port` / `bind` | `[mothership.bind]` |
|
|
93
|
+
| `preload_app!` | always: `config.ru` loads in the master, then workers fork |
|
|
94
|
+
| `before_fork`, `on_worker_boot`, `on_worker_shutdown`, `out_of_band` | `Shuttlebay.before_fork { }` and friends, in `config.ru` |
|
|
95
|
+
| `worker_timeout` | the bay's `request_timeout` (`504`, then the stuck worker is recycled) |
|
|
96
|
+
| `pumactl phased-restart` / `plugin :tmp_restart` | `kill -USR1` the mothership process |
|
|
97
|
+
| `wait_for_less_busy_worker` | built in: requests only go to idle threads |
|
|
98
|
+
|
|
99
|
+
Hooks go in `config.ru`, before `run`:
|
|
100
|
+
|
|
101
|
+
```ruby
|
|
102
|
+
# config.ru
|
|
103
|
+
require_relative "config/environment"
|
|
104
|
+
|
|
105
|
+
Shuttlebay.on_worker_boot { |index| Rails.logger.info("worker #{index} up") }
|
|
106
|
+
Shuttlebay.out_of_band { GC.start } # when a worker has no request in flight
|
|
107
|
+
|
|
108
|
+
run Rails.application
|
|
109
|
+
Rails.application.load_server
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
`config/puma.rb` can stay for `rails server`; Shuttlebay never reads it.
|
|
113
|
+
|
|
114
|
+
## 5. Let Mothership do the middleware's work
|
|
115
|
+
|
|
116
|
+
Under Mothership the Railtie removes, on its own, what Mothership already does:
|
|
117
|
+
`Rack::Sendfile`, `ActionDispatch::Static` (when the static dir serves
|
|
118
|
+
`public/`) and `Rack::ETag` / `Rack::ConditionalGet` (Mothership tags pages and
|
|
119
|
+
answers revalidations itself, off the GVL). Nothing changes under Puma.
|
|
120
|
+
|
|
121
|
+
`Rack::Attack` rules are your configuration, so you move them yourself:
|
|
122
|
+
|
|
123
|
+
```ruby
|
|
124
|
+
# config/initializers/rack_attack.rb, before
|
|
125
|
+
Rack::Attack.throttle("api", limit: 60, period: 1.minute) do |req|
|
|
126
|
+
req.ip if req.path.start_with?("/api/")
|
|
127
|
+
end
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
```toml
|
|
131
|
+
# ship-manifest.toml, after
|
|
132
|
+
[[mothership.rate_limits]]
|
|
133
|
+
pattern = "^/api/"
|
|
134
|
+
limit = 60
|
|
135
|
+
period = "1m"
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Mothership keeps the counts in memory, with no cache-store write per request,
|
|
139
|
+
and answers `429` with `Retry-After`. Allowlists move to
|
|
140
|
+
`[[mothership.ip_allowlist]]`; see the Mothership README for both.
|
|
141
|
+
|
|
142
|
+
## 6. Behind a CDN or a proxy
|
|
143
|
+
|
|
144
|
+
Behind Cloudflare, Kamal proxy or a load balancer, tell Mothership which
|
|
145
|
+
proxies to see through, so allowlists and rate limits apply to the visitor,
|
|
146
|
+
not to the proxy:
|
|
147
|
+
|
|
148
|
+
```toml
|
|
149
|
+
[mothership]
|
|
150
|
+
trusted_proxies = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"] # + your CDN's ranges
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
The app's `REMOTE_ADDR` is the address that connected to Mothership (or the
|
|
154
|
+
PROXY-protocol one, with `proxy_protocol = true` on the bind), and the
|
|
155
|
+
forwarding headers pass through. Rails resolves `request.remote_ip` from them
|
|
156
|
+
as it always has: keep the configuration you need for that today (for
|
|
157
|
+
Cloudflare, `cloudflare-rails`).
|
|
158
|
+
|
|
159
|
+
## 7. Caching
|
|
160
|
+
|
|
161
|
+
Mothership can cache whole responses in memory, the way a CDN does, purged by
|
|
162
|
+
tag when the data changes:
|
|
163
|
+
|
|
164
|
+
```toml
|
|
165
|
+
[mothership]
|
|
166
|
+
metrics_port = 9090 # also serves the purge endpoint, on 127.0.0.1
|
|
167
|
+
|
|
168
|
+
[mothership.response_cache]
|
|
169
|
+
status_header = true # Cache-Status on responses
|
|
170
|
+
|
|
171
|
+
[[mothership.response_cache.rules]]
|
|
172
|
+
pattern = "^/api/"
|
|
173
|
+
ttl = "5m"
|
|
174
|
+
stale = "1m"
|
|
175
|
+
bypass_cookies = ["_myapp_session"] # signed-in users skip the cache
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
```ruby
|
|
179
|
+
class BooksController < ApplicationController
|
|
180
|
+
def show
|
|
181
|
+
@book = Book.find(params[:id])
|
|
182
|
+
Shuttlebay::Cache.tag(response.headers, "book-#{@book.id}")
|
|
183
|
+
end
|
|
184
|
+
end
|
|
185
|
+
|
|
186
|
+
class Book < ApplicationRecord
|
|
187
|
+
after_commit { Shuttlebay::Cache.purge(tags: ["book-#{id}"]) }
|
|
188
|
+
end
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
A successful `POST`/`PUT`/`PATCH`/`DELETE` through Mothership already
|
|
192
|
+
invalidates its own URL. Requests with `Authorization`, or with a cookie a rule
|
|
193
|
+
does not let through, never touch a rule's cache. The README's caching section
|
|
194
|
+
covers the rest.
|
|
195
|
+
|
|
196
|
+
## 8. TLS and HTTP/2
|
|
197
|
+
|
|
198
|
+
Behind a CDN or Kamal proxy, they terminate TLS and speak HTTP/2 to browsers;
|
|
199
|
+
Mothership listens on plain HTTP behind them, and nothing here is needed.
|
|
200
|
+
|
|
201
|
+
To terminate TLS in Mothership itself (guardship 0.2.1 and later), get a
|
|
202
|
+
certificate first, then add a TLS bind.
|
|
203
|
+
|
|
204
|
+
**The certificate.** Mothership serves `public/`, so certbot's webroot mode
|
|
205
|
+
works while the app runs on plain HTTP:
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
mkdir -p public/.well-known/acme-challenge # create it before certbot runs
|
|
209
|
+
certbot certonly --webroot -w ./public -d example.com -d www.example.com
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Create `.well-known` beforehand: Mothership skips the filesystem for
|
|
213
|
+
top-level names it has not seen in `public/` (it re-lists the directory
|
|
214
|
+
about once a second), so one certbot creates on the spot can miss the
|
|
215
|
+
validation request. One certificate can name every domain (`-d` repeated).
|
|
216
|
+
|
|
217
|
+
**The binds.**
|
|
218
|
+
|
|
219
|
+
```toml
|
|
220
|
+
[mothership.bind]
|
|
221
|
+
http = "0.0.0.0:80"
|
|
222
|
+
https = { addr = "0.0.0.0:443", tls = { cert = "/etc/letsencrypt/live/example.com/fullchain.pem", key = "/etc/letsencrypt/live/example.com/privkey.pem" } }
|
|
223
|
+
|
|
224
|
+
[[bays.http]]
|
|
225
|
+
name = "web"
|
|
226
|
+
# command, args, workers, threads as before
|
|
227
|
+
routes = [
|
|
228
|
+
{ bind = "http", pattern = "/.*" },
|
|
229
|
+
{ bind = "https", pattern = "/.*" },
|
|
230
|
+
]
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
```ruby
|
|
234
|
+
# config/environments/production.rb
|
|
235
|
+
config.force_ssl = true # redirects HTTP to HTTPS, sends HSTS, secures cookies
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
- Browsers get HTTP/2 (negotiated over ALPN). On the TLS bind the app sees
|
|
239
|
+
`rack.url_scheme = "https"` and `HTTPS=on`, so `force_ssl` and
|
|
240
|
+
`request.ssl?` work with no `assume_ssl` or forwarded headers.
|
|
241
|
+
- Renewals are picked up without a restart, within 30 s of certbot writing
|
|
242
|
+
them. A renewal that does not load is logged; the old one keeps serving.
|
|
243
|
+
- Let's Encrypt keys are readable by root only: run with access to them, or
|
|
244
|
+
have a certbot `--deploy-hook` copy the pair where the app's user can read
|
|
245
|
+
it. Ports 80 and 443 need `setcap cap_net_bind_service=+ep` on the binary,
|
|
246
|
+
`AmbientCapabilities=CAP_NET_BIND_SERVICE` under systemd, or a container.
|
|
247
|
+
|
|
248
|
+
From guardship 0.3, Mothership can also redirect HTTP to HTTPS itself
|
|
249
|
+
(`http = { addr = "0.0.0.0:80", https_redirect = true }`, ACME challenges
|
|
250
|
+
left on HTTP) and serve a certificate per domain on one bind
|
|
251
|
+
(`tls = [{ cert, key }, ...]`, picked by SNI); see the Mothership README.
|
|
252
|
+
|
|
253
|
+
## 9. Docker and Kamal
|
|
254
|
+
|
|
255
|
+
Mothership replaces `./bin/thrust ./bin/rails server` in the Dockerfile Rails
|
|
256
|
+
generates:
|
|
257
|
+
|
|
258
|
+
```dockerfile
|
|
259
|
+
EXPOSE 80
|
|
260
|
+
CMD ["bundle", "exec", "mothership"]
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
with the bind on port 80 (`http = "0.0.0.0:80"`), where kamal-proxy expects
|
|
264
|
+
the app. Rails' `bin/docker-entrypoint` prepares the database only when the
|
|
265
|
+
command is `./bin/rails server`, so run that as a prelaunch job instead:
|
|
266
|
+
|
|
267
|
+
```toml
|
|
268
|
+
[[mothership.prelaunch]]
|
|
269
|
+
name = "db-prepare"
|
|
270
|
+
command = "bin/rails"
|
|
271
|
+
args = ["db:prepare"]
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
```yaml
|
|
275
|
+
# config/deploy.yml
|
|
276
|
+
proxy:
|
|
277
|
+
host: app.example.com
|
|
278
|
+
ssl: true
|
|
279
|
+
healthcheck:
|
|
280
|
+
path: /up
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
Mothership runs fine as PID 1: on Linux it reaps and forwards signals itself
|
|
284
|
+
(`docker run --init` is still the safer default elsewhere). Kamal deploys by
|
|
285
|
+
starting a new container and moving traffic to it once `/up` answers, so
|
|
286
|
+
phased restarts (`kill -USR1`) are for servers you deploy to in place.
|
|
287
|
+
|
|
288
|
+
On several hosts:
|
|
289
|
+
|
|
290
|
+
- Run prelaunch jobs (migrations) once: tag one host and elect it.
|
|
291
|
+
|
|
292
|
+
```yaml
|
|
293
|
+
servers:
|
|
294
|
+
web:
|
|
295
|
+
- 10.0.0.5: flagship
|
|
296
|
+
- 10.0.0.6
|
|
297
|
+
env:
|
|
298
|
+
tags:
|
|
299
|
+
flagship:
|
|
300
|
+
MS_FLAGSHIP: "true"
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
```toml
|
|
304
|
+
[mothership.flagship]
|
|
305
|
+
enabled = true
|
|
306
|
+
election = "static"
|
|
307
|
+
static_flagship = "$MS_FLAGSHIP"
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
- With a response cache, let the hosts relay purges and invalidations to one
|
|
311
|
+
another (`[mothership.response_cache.peers]` in the Mothership README).
|
|
312
|
+
|
|
313
|
+
## 10. Watching it run
|
|
314
|
+
|
|
315
|
+
- `bundle exec mothership health` prints the fleet's health as JSON (needs
|
|
316
|
+
`metrics_port`); `/metrics` serves Prometheus metrics, `mothership_http_bay_*`
|
|
317
|
+
among them.
|
|
318
|
+
- Both Mothership and Shuttlebay log JSON lines to stdout.
|
|
319
|
+
- `503 Starting`: the bay is still booting (`boot_timeout`, 120 s).
|
|
320
|
+
- `503` once running: every thread was busy for `queue_timeout`; add
|
|
321
|
+
`threads` (I/O-bound apps) or `workers` (CPU-bound ones).
|
|
322
|
+
- `504`: no response head within `request_timeout` (60 s); the stuck worker
|
|
323
|
+
is recycled.
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
require 'net/http'
|
|
5
|
+
require 'uri'
|
|
6
|
+
|
|
7
|
+
module Shuttlebay
|
|
8
|
+
# Mothership's edge cache, from the app: tag responses with the data they
|
|
9
|
+
# show, and purge the tags when that data changes, so entries live until
|
|
10
|
+
# purged instead of until a guessed TTL runs out.
|
|
11
|
+
#
|
|
12
|
+
# # in a controller
|
|
13
|
+
# Shuttlebay::Cache.tag(response.headers, "book-#{@book.id}", 'books')
|
|
14
|
+
# # in a model
|
|
15
|
+
# after_commit { Shuttlebay::Cache.purge(tags: ["book-#{id}", 'books']) }
|
|
16
|
+
#
|
|
17
|
+
# Purging works from any process Mothership runs (web workers, job
|
|
18
|
+
# workers): it posts to the admin endpoint in MS_ADMIN_URL, which
|
|
19
|
+
# Mothership sets when it has a metrics port.
|
|
20
|
+
module Cache
|
|
21
|
+
HEADER = 'cache-tag'
|
|
22
|
+
|
|
23
|
+
module_function
|
|
24
|
+
|
|
25
|
+
# Add `tags` to the response's Cache-Tag (Mothership keeps it from clients).
|
|
26
|
+
def tag(headers, *tags)
|
|
27
|
+
tags = tags.flatten.map { |tag| tag.to_s.strip }.reject(&:empty?)
|
|
28
|
+
return headers if tags.empty?
|
|
29
|
+
|
|
30
|
+
headers[HEADER] = [headers[HEADER], *tags].compact.reject(&:empty?).join(', ')
|
|
31
|
+
headers
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# Purge the entries tagged with any of `tags`, for any of `urls` (path and
|
|
35
|
+
# query), under any of `prefixes`, or `all`. `soft: true` marks them stale
|
|
36
|
+
# instead: still served while one request refreshes them. Returns how many
|
|
37
|
+
# were purged; nil outside Mothership (no MS_ADMIN_URL), where there is no
|
|
38
|
+
# cache to purge. A purge Mothership refuses raises Shuttlebay::Error.
|
|
39
|
+
def purge(tags: [], urls: [], prefixes: [], all: false, soft: false, admin_url: ENV.fetch('MS_ADMIN_URL', nil),
|
|
40
|
+
timeout: 2)
|
|
41
|
+
params = Array(tags).map { |tag| ['tag', tag.to_s] } + Array(urls).map { |url| ['url', url.to_s] } +
|
|
42
|
+
Array(prefixes).map { |prefix| ['prefix', prefix.to_s] }
|
|
43
|
+
params << %w[all true] if all
|
|
44
|
+
raise ArgumentError, 'purge needs tags, urls, prefixes or all: true' if params.empty?
|
|
45
|
+
return nil if admin_url.nil? || admin_url.empty?
|
|
46
|
+
|
|
47
|
+
params << %w[soft true] if soft
|
|
48
|
+
post_purge(URI.join(admin_url, '/cache/purge').tap { |uri| uri.query = URI.encode_www_form(params) }, timeout)
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def post_purge(uri, timeout)
|
|
52
|
+
response = Net::HTTP.start(uri.host, uri.port, open_timeout: timeout, read_timeout: timeout) do |http|
|
|
53
|
+
http.post(uri.request_uri, '')
|
|
54
|
+
end
|
|
55
|
+
raise Error, "cache purge failed: #{response.code} #{response.body}" unless response.is_a?(Net::HTTPSuccess)
|
|
56
|
+
|
|
57
|
+
JSON.parse(response.body).fetch('purged')
|
|
58
|
+
end
|
|
59
|
+
private_class_method :post_purge
|
|
60
|
+
end
|
|
61
|
+
end
|
data/lib/shuttlebay/engine.rb
CHANGED
|
@@ -19,5 +19,30 @@ module Shuttlebay
|
|
|
19
19
|
def self.spool_file
|
|
20
20
|
Tempfile.new('shuttlebay-body', binmode: true).tap(&:unlink)
|
|
21
21
|
end
|
|
22
|
+
|
|
23
|
+
# The rack.input of a request without a body, like Puma's NullIO: always
|
|
24
|
+
# at EOF and stateless, so one frozen instance serves every request.
|
|
25
|
+
class NullInput
|
|
26
|
+
def gets = nil
|
|
27
|
+
def each = nil
|
|
28
|
+
def rewind = 0
|
|
29
|
+
def close = nil
|
|
30
|
+
def closed? = false
|
|
31
|
+
def eof? = true
|
|
32
|
+
def size = 0
|
|
33
|
+
def string = ''
|
|
34
|
+
|
|
35
|
+
def read(length = nil, buffer = nil)
|
|
36
|
+
raise ArgumentError, "negative length #{length} given" if length&.negative?
|
|
37
|
+
|
|
38
|
+
buffer = buffer.nil? ? +''.b : String.try_convert(buffer)
|
|
39
|
+
raise TypeError, 'buffer must be a String' unless buffer
|
|
40
|
+
|
|
41
|
+
buffer.clear
|
|
42
|
+
length&.positive? ? nil : buffer
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
NULL_INPUT = NullInput.new.freeze
|
|
22
47
|
end
|
|
23
48
|
end
|
data/lib/shuttlebay/handler.rb
CHANGED
|
@@ -74,11 +74,15 @@ module Shuttlebay
|
|
|
74
74
|
def respond(io, codec, env, status, headers, body)
|
|
75
75
|
head_sent = false
|
|
76
76
|
kind, path = reply_kind(env, status, body)
|
|
77
|
-
|
|
77
|
+
if kind == :stream && body.respond_to?(:to_ary)
|
|
78
|
+
# The whole body is in hand: head, body and FIN leave in one write.
|
|
79
|
+
codec.reply_whole(io, status, headers, body.to_ary)
|
|
80
|
+
return [true, nil]
|
|
81
|
+
end
|
|
78
82
|
|
|
79
83
|
codec.reply(io, status, headers, kind, path)
|
|
80
84
|
head_sent = true
|
|
81
|
-
write_body(io, codec, body
|
|
85
|
+
write_body(io, codec, body) if kind == :stream
|
|
82
86
|
codec.finish(io)
|
|
83
87
|
[true, nil]
|
|
84
88
|
rescue *LINK_LOST => e
|
|
@@ -100,10 +104,8 @@ module Shuttlebay
|
|
|
100
104
|
[:stream, nil]
|
|
101
105
|
end
|
|
102
106
|
|
|
103
|
-
def write_body(io, codec, body
|
|
104
|
-
if
|
|
105
|
-
chunks.each { |chunk| codec.write(io, chunk) }
|
|
106
|
-
elsif body.respond_to?(:each)
|
|
107
|
+
def write_body(io, codec, body)
|
|
108
|
+
if body.respond_to?(:each)
|
|
107
109
|
body.each do |chunk|
|
|
108
110
|
codec.write(io, chunk)
|
|
109
111
|
codec.flush(io)
|
data/lib/shuttlebay/logger.rb
CHANGED
|
@@ -16,6 +16,11 @@ module Shuttlebay
|
|
|
16
16
|
define_method(level) { |msg, **fields| log(level, msg, **fields) }
|
|
17
17
|
end
|
|
18
18
|
|
|
19
|
+
# This logger as `ship`'s: same stream, another ship name.
|
|
20
|
+
def for_ship(ship)
|
|
21
|
+
ship == @ship ? self : self.class.new(@io, ship: ship)
|
|
22
|
+
end
|
|
23
|
+
|
|
19
24
|
def log(level, msg, **fields)
|
|
20
25
|
line = JSON.generate({ level: level, msg: msg, ship: @ship, pid: Process.pid, **fields })
|
|
21
26
|
@io.write("#{line}\n")
|
data/lib/shuttlebay/railtie.rb
CHANGED
|
@@ -1,11 +1,52 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require 'json'
|
|
3
4
|
require 'rack/sendfile'
|
|
4
5
|
|
|
5
6
|
module Shuttlebay
|
|
7
|
+
# Under Mothership, some of a Rails app's middleware repeats work Mothership
|
|
8
|
+
# already does for its HTTP bays: the app's copy is removed. Other servers
|
|
9
|
+
# (rails server, Puma) keep the full stack.
|
|
10
|
+
#
|
|
11
|
+
# - Rack::Sendfile: Mothership serves `to_path` bodies itself.
|
|
12
|
+
# - ActionDispatch::Static: when Mothership serves the app's public/ at `/`
|
|
13
|
+
# for everything routed to the bay (it lists those in MS_STATIC_ROOTS).
|
|
14
|
+
# The app's `public_file_server.headers` Mothership does not send (per
|
|
15
|
+
# MS_STATIC_HEADERS) are logged: they belong in the manifest now.
|
|
16
|
+
# - Rack::ETag, Rack::ConditionalGet: when Mothership tags responses and
|
|
17
|
+
# answers revalidations (MS_CONDITIONAL_GET=1), off the app's GVL.
|
|
6
18
|
class Railtie < ::Rails::Railtie
|
|
7
19
|
initializer 'shuttlebay.mothership_serves_files' do |app|
|
|
8
|
-
|
|
20
|
+
next unless Shuttlebay.running?
|
|
21
|
+
|
|
22
|
+
middleware = app.config.middleware
|
|
23
|
+
middleware.delete(::Rack::Sendfile) if Shuttlebay.offloaded?(:sendfile)
|
|
24
|
+
public = app.paths['public'].first
|
|
25
|
+
if Shuttlebay.offloaded?(:static, public)
|
|
26
|
+
middleware.delete(::ActionDispatch::Static)
|
|
27
|
+
Shuttlebay::Railtie.warn_unsent_headers(public, app.config.public_file_server.headers)
|
|
28
|
+
end
|
|
29
|
+
if Shuttlebay.offloaded?(:conditional_get)
|
|
30
|
+
middleware.delete(::Rack::ETag)
|
|
31
|
+
middleware.delete(::Rack::ConditionalGet)
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# The app's `headers` for public/ files that Mothership, serving them
|
|
36
|
+
# now, does not send as given; logged once, at boot.
|
|
37
|
+
def self.warn_unsent_headers(public, headers, announced = ENV.fetch('MS_STATIC_HEADERS', '{}'))
|
|
38
|
+
unsent = unsent_headers(public, headers, announced)
|
|
39
|
+
return if unsent.empty?
|
|
40
|
+
|
|
41
|
+
Shuttlebay.logger.warn('public_file_server.headers are not sent by Mothership, which serves public/; ' \
|
|
42
|
+
'add them to the [[mothership.static_dirs]] headers', missing: unsent)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def self.unsent_headers(public, headers, announced)
|
|
46
|
+
sent = JSON.parse(announced).fetch(File.realpath(public), {})
|
|
47
|
+
(headers || {}).to_h { |name, value| [name.to_s.downcase, value.to_s] }.reject { |name, value| sent[name] == value }
|
|
48
|
+
rescue JSON::ParserError, SystemCallError
|
|
49
|
+
(headers || {}).to_h { |name, value| [name.to_s.downcase, value.to_s] }
|
|
9
50
|
end
|
|
10
51
|
end
|
|
11
52
|
end
|
data/lib/shuttlebay/runtime.rb
CHANGED
|
@@ -13,12 +13,16 @@ module Shuttlebay
|
|
|
13
13
|
class << self
|
|
14
14
|
attr_accessor :config
|
|
15
15
|
|
|
16
|
-
attr_writer :running, :logger
|
|
16
|
+
attr_writer :running, :logger, :warmup
|
|
17
17
|
|
|
18
18
|
def running?
|
|
19
19
|
@running == true
|
|
20
20
|
end
|
|
21
21
|
|
|
22
|
+
def warmup?
|
|
23
|
+
@warmup != false
|
|
24
|
+
end
|
|
25
|
+
|
|
22
26
|
def logger
|
|
23
27
|
@logger ||= Logger.new
|
|
24
28
|
end
|
|
@@ -51,8 +55,46 @@ module Shuttlebay
|
|
|
51
55
|
@fiber_scheduler_factory = factory
|
|
52
56
|
end
|
|
53
57
|
|
|
54
|
-
|
|
55
|
-
|
|
58
|
+
# Run `name`'s hooks with `args`, each given only as many as it takes:
|
|
59
|
+
# `on_worker_boot { |index| }` and `{ |index, ship| }` both work, lambdas
|
|
60
|
+
# and methods included.
|
|
61
|
+
def run_hooks(name, *args)
|
|
62
|
+
hooks.fetch(name).each do |hook|
|
|
63
|
+
hook.call(*(hook.arity.negative? ? args : args.first(hook.arity)))
|
|
64
|
+
end
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# Whether Mothership does `concern` for this app, so its Rack middleware
|
|
68
|
+
# for it can go (the Railtie does this for Rails; elsewhere, in
|
|
69
|
+
# config.ru):
|
|
70
|
+
#
|
|
71
|
+
# use Rack::ETag unless Shuttlebay.offloaded?(:conditional_get)
|
|
72
|
+
# use Rack::Static, urls: [""], root: "public" unless Shuttlebay.offloaded?(:static, "public")
|
|
73
|
+
#
|
|
74
|
+
# - :sendfile: `to_path` bodies are streamed by Mothership (Rack::Sendfile)
|
|
75
|
+
# - :conditional_get: ETags and 304s (Rack::ETag, Rack::ConditionalGet)
|
|
76
|
+
# - :static, dir: `dir` is served at `/`, pages by name included
|
|
77
|
+
# (Rack::Static, ActionDispatch::Static)
|
|
78
|
+
#
|
|
79
|
+
# Always false outside Mothership.
|
|
80
|
+
def offloaded?(concern, dir = nil)
|
|
81
|
+
return false unless running?
|
|
82
|
+
|
|
83
|
+
case concern
|
|
84
|
+
when :sendfile then true
|
|
85
|
+
when :conditional_get then ENV['MS_CONDITIONAL_GET'] == '1'
|
|
86
|
+
when :static then static_root?(dir)
|
|
87
|
+
else raise ArgumentError, "unknown concern #{concern.inspect} (:sendfile, :conditional_get, :static)"
|
|
88
|
+
end
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# Whether Mothership serves `dir` at `/` (it lists those in MS_STATIC_ROOTS).
|
|
92
|
+
def static_root?(dir, roots = ENV.fetch('MS_STATIC_ROOTS', ''))
|
|
93
|
+
raise ArgumentError, 'Shuttlebay.offloaded?(:static, dir) needs the directory' if dir.nil?
|
|
94
|
+
|
|
95
|
+
roots.split("\n").include?(File.realpath(dir))
|
|
96
|
+
rescue SystemCallError
|
|
97
|
+
false
|
|
56
98
|
end
|
|
57
99
|
|
|
58
100
|
def clear_hooks!
|
data/lib/shuttlebay/server.rb
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require 'json'
|
|
3
4
|
require 'socket'
|
|
4
5
|
|
|
5
6
|
module Shuttlebay
|
|
@@ -11,9 +12,10 @@ module Shuttlebay
|
|
|
11
12
|
STABLE_AFTER = 30.0
|
|
12
13
|
SIGNALS = { 'TERM' => 'T', 'INT' => 'I', 'QUIT' => 'Q', 'CHLD' => 'C' }.freeze
|
|
13
14
|
|
|
14
|
-
|
|
15
|
+
Group = Struct.new(:ship, :app_path, :socket_path, :workers, :threads, :app, :listener)
|
|
16
|
+
Slot = Struct.new(:group, :index, :pid, :started_at, :backoff, :respawn_at)
|
|
15
17
|
|
|
16
|
-
attr_reader :app_path, :socket_path, :workers, :threads, :ship
|
|
18
|
+
attr_reader :app_path, :socket_path, :workers, :threads, :ship, :forks
|
|
17
19
|
|
|
18
20
|
def self.from_env(argv, env = ENV)
|
|
19
21
|
socket_path = env.fetch('MS_SOCKET_PATH') do
|
|
@@ -24,34 +26,48 @@ module Shuttlebay
|
|
|
24
26
|
socket_path: socket_path,
|
|
25
27
|
workers: Integer(env.fetch('MS_BAY_WORKERS', '0'), 10),
|
|
26
28
|
threads: Integer(env.fetch('MS_BAY_THREADS', '3'), 10),
|
|
27
|
-
ship: env.fetch('MS_SHIP', 'rack')
|
|
29
|
+
ship: env.fetch('MS_SHIP', 'rack'),
|
|
30
|
+
forks: parse_forks(env.fetch('MS_BAY_FORKS', '[]'))
|
|
28
31
|
)
|
|
29
32
|
end
|
|
30
33
|
|
|
31
|
-
def
|
|
34
|
+
def self.parse_forks(json)
|
|
35
|
+
JSON.parse(json).map do |fork|
|
|
36
|
+
Group.new(fork.fetch('ship'), fork.fetch('app'), fork.fetch('socket'),
|
|
37
|
+
Integer(fork.fetch('workers')).clamp(1..), Integer(fork.fetch('threads')))
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def initialize(app_path:, socket_path:, workers:, threads:, ship:, forks: [], logger: Shuttlebay.logger)
|
|
32
42
|
raise ArgumentError, "workers must be >= 0 (got #{workers})" if workers.negative?
|
|
33
43
|
raise ArgumentError, "threads must be >= 1 (got #{threads})" if threads < 1
|
|
44
|
+
forks.each { |fork| raise ArgumentError, "#{fork.ship}: threads must be >= 1" if fork.threads < 1 }
|
|
34
45
|
|
|
35
46
|
@app_path = app_path
|
|
36
47
|
@socket_path = socket_path
|
|
37
48
|
@workers = workers
|
|
38
49
|
@threads = threads
|
|
39
50
|
@ship = ship
|
|
51
|
+
@forks = forks
|
|
40
52
|
@logger = logger
|
|
41
53
|
@master_pid = Process.pid
|
|
42
54
|
end
|
|
43
55
|
|
|
44
56
|
def run
|
|
45
57
|
Shuttlebay.running = true
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
58
|
+
primary = Group.new(ship, app_path, socket_path, workers.clamp(1..), threads, load_app(app_path))
|
|
59
|
+
forked = load_forks
|
|
60
|
+
Process.warmup if Shuttlebay.warmup?
|
|
61
|
+
[primary, *forked].each do |group|
|
|
62
|
+
group.listener = bind(group.socket_path)
|
|
63
|
+
@logger.info('app loaded, socket bound', ship: group.ship, app: group.app_path, socket: group.socket_path,
|
|
64
|
+
workers: group.workers, threads: group.threads)
|
|
65
|
+
end
|
|
50
66
|
|
|
51
|
-
if workers.zero?
|
|
52
|
-
worker(
|
|
67
|
+
if workers.zero? && forked.empty?
|
|
68
|
+
worker(primary, index: nil).run
|
|
53
69
|
else
|
|
54
|
-
supervise(
|
|
70
|
+
supervise([primary, *forked])
|
|
55
71
|
end
|
|
56
72
|
ensure
|
|
57
73
|
cleanup if Process.pid == @master_pid
|
|
@@ -59,7 +75,7 @@ module Shuttlebay
|
|
|
59
75
|
|
|
60
76
|
private
|
|
61
77
|
|
|
62
|
-
def load_app
|
|
78
|
+
def load_app(app_path)
|
|
63
79
|
require 'rack'
|
|
64
80
|
require 'rack/builder'
|
|
65
81
|
path = File.expand_path(app_path)
|
|
@@ -68,26 +84,45 @@ module Shuttlebay
|
|
|
68
84
|
Rack::Builder.parse_file(path)
|
|
69
85
|
end
|
|
70
86
|
|
|
71
|
-
def
|
|
87
|
+
def load_forks
|
|
88
|
+
forks.filter_map do |group|
|
|
89
|
+
group.app = load_app(group.app_path)
|
|
90
|
+
group
|
|
91
|
+
rescue StandardError, ScriptError => e
|
|
92
|
+
@logger.error('fork app failed to load, skipped', ship: group.ship, app: group.app_path,
|
|
93
|
+
**Logger.error_fields(e))
|
|
94
|
+
nil
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
def bind(socket_path)
|
|
72
99
|
if File.exist?(socket_path)
|
|
73
100
|
raise ArgumentError, "#{socket_path} exists and is not a socket" unless File.socket?(socket_path)
|
|
74
101
|
|
|
75
102
|
File.unlink(socket_path)
|
|
76
103
|
end
|
|
77
|
-
UNIXServer.new(socket_path).tap
|
|
104
|
+
UNIXServer.new(socket_path).tap do |server|
|
|
105
|
+
server.listen(1024)
|
|
106
|
+
bound << [server, socket_path]
|
|
107
|
+
end
|
|
78
108
|
end
|
|
79
109
|
|
|
80
|
-
def
|
|
81
|
-
|
|
82
|
-
index: index, master_pid: index && @master_pid, logger: @logger)
|
|
110
|
+
def bound
|
|
111
|
+
@bound ||= []
|
|
83
112
|
end
|
|
84
113
|
|
|
85
|
-
def
|
|
114
|
+
def worker(group, index:)
|
|
115
|
+
Worker.new(app: group.app, listener: group.listener, threads: group.threads, workers: group.workers,
|
|
116
|
+
ship: group.ship, index: index, master_pid: index && @master_pid, logger: @logger)
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
def supervise(groups)
|
|
86
120
|
wake_r, wake_w = IO.pipe
|
|
87
121
|
previous = trap_signals(SIGNALS, wake_w)
|
|
88
|
-
@app = app
|
|
89
122
|
Shuttlebay.run_hooks(:before_fork)
|
|
90
|
-
@slots =
|
|
123
|
+
@slots = groups.flat_map do |group|
|
|
124
|
+
Array.new(group.workers) { |index| Slot.new(group, index, nil, nil, RESPAWN_MIN, monotonic) }
|
|
125
|
+
end
|
|
91
126
|
@stopping = false
|
|
92
127
|
|
|
93
128
|
until @stopping && @slots.none?(&:pid)
|
|
@@ -120,14 +155,28 @@ module Shuttlebay
|
|
|
120
155
|
@slots.each do |slot|
|
|
121
156
|
next if slot.pid || slot.respawn_at > now
|
|
122
157
|
|
|
123
|
-
slot.pid = spawn_worker(slot
|
|
158
|
+
slot.pid = spawn_worker(slot)
|
|
124
159
|
slot.started_at = now
|
|
125
160
|
end
|
|
126
161
|
end
|
|
127
162
|
|
|
128
|
-
def spawn_worker(
|
|
129
|
-
|
|
130
|
-
|
|
163
|
+
def spawn_worker(slot)
|
|
164
|
+
fork do
|
|
165
|
+
become(slot.group)
|
|
166
|
+
run_worker_process(slot.index, SIGNALS) { worker(slot.group, index: slot.index) }
|
|
167
|
+
end
|
|
168
|
+
end
|
|
169
|
+
|
|
170
|
+
# In a freshly forked worker of `group`: it serves that bay alone, so the
|
|
171
|
+
# other bays' listening sockets are closed, and a fork's worker logs (and
|
|
172
|
+
# sees in MS_SHIP) the fork's name rather than its parent's.
|
|
173
|
+
def become(group)
|
|
174
|
+
bound.each { |server, path| server.close unless path == group.socket_path }
|
|
175
|
+
return if group.ship == ship
|
|
176
|
+
|
|
177
|
+
ENV['MS_SHIP'] = group.ship
|
|
178
|
+
@logger = @logger.respond_to?(:for_ship) ? @logger.for_ship(group.ship) : @logger
|
|
179
|
+
Shuttlebay.logger = @logger
|
|
131
180
|
end
|
|
132
181
|
|
|
133
182
|
def reap
|
|
@@ -153,7 +202,7 @@ module Shuttlebay
|
|
|
153
202
|
slot.respawn_at = monotonic + delay
|
|
154
203
|
return if @stopping
|
|
155
204
|
|
|
156
|
-
@logger.warn('worker exited, respawning', index: slot.index, pid: pid,
|
|
205
|
+
@logger.warn('worker exited, respawning', ship: slot.group.ship, index: slot.index, pid: pid,
|
|
157
206
|
status: status && (status.exitstatus || status.termsig),
|
|
158
207
|
uptime: uptime.round(3), respawn_in: delay)
|
|
159
208
|
end
|
|
@@ -190,10 +239,12 @@ module Shuttlebay
|
|
|
190
239
|
end
|
|
191
240
|
|
|
192
241
|
def cleanup
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
242
|
+
bound.each do |server, path|
|
|
243
|
+
server.close
|
|
244
|
+
File.unlink(path) if File.socket?(path)
|
|
245
|
+
rescue SystemCallError => e
|
|
246
|
+
@logger.warn('socket cleanup failed', socket: path, error: e.message)
|
|
247
|
+
end
|
|
197
248
|
end
|
|
198
249
|
end
|
|
199
250
|
end
|
|
Binary file
|
data/lib/shuttlebay/version.rb
CHANGED
data/lib/shuttlebay/worker.rb
CHANGED
|
@@ -41,7 +41,7 @@ module Shuttlebay
|
|
|
41
41
|
def run(trap_signals: true)
|
|
42
42
|
@wake_r, @wake_w = IO.pipe
|
|
43
43
|
previous = trap_signals(SIGNALS, @wake_w) if trap_signals
|
|
44
|
-
Shuttlebay.run_hooks(:worker_boot, @index) if forked?
|
|
44
|
+
Shuttlebay.run_hooks(:worker_boot, @index, @ship) if forked?
|
|
45
45
|
pool = spawn_pool
|
|
46
46
|
@logger.info('worker serving', index: @index, threads: @threads, reactors: @reactors)
|
|
47
47
|
|
|
@@ -158,7 +158,7 @@ module Shuttlebay
|
|
|
158
158
|
end
|
|
159
159
|
|
|
160
160
|
def run_shutdown_hooks
|
|
161
|
-
Shuttlebay.run_hooks(:worker_shutdown, @index)
|
|
161
|
+
Shuttlebay.run_hooks(:worker_shutdown, @index, @ship)
|
|
162
162
|
rescue StandardError => e
|
|
163
163
|
@logger.error('on_worker_shutdown hook failed', **Logger.error_fields(e))
|
|
164
164
|
end
|
data/lib/shuttlebay.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: shuttlebay
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1
|
|
4
|
+
version: 0.2.1
|
|
5
5
|
platform: aarch64-linux
|
|
6
6
|
authors:
|
|
7
7
|
- Abdelkader Boudih
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-11-18 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: rack
|
|
@@ -53,19 +53,33 @@ dependencies:
|
|
|
53
53
|
- !ruby/object:Gem::Version
|
|
54
54
|
version: '3.40'
|
|
55
55
|
- !ruby/object:Gem::Dependency
|
|
56
|
-
name:
|
|
56
|
+
name: guardship
|
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
|
58
|
+
requirements:
|
|
59
|
+
- - ">="
|
|
60
|
+
- !ruby/object:Gem::Version
|
|
61
|
+
version: 0.2.0
|
|
62
|
+
type: :development
|
|
63
|
+
prerelease: false
|
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
65
|
+
requirements:
|
|
66
|
+
- - ">="
|
|
67
|
+
- !ruby/object:Gem::Version
|
|
68
|
+
version: 0.2.0
|
|
69
|
+
- !ruby/object:Gem::Dependency
|
|
70
|
+
name: padded_cell
|
|
57
71
|
requirement: !ruby/object:Gem::Requirement
|
|
58
72
|
requirements:
|
|
59
73
|
- - "~>"
|
|
60
74
|
- !ruby/object:Gem::Version
|
|
61
|
-
version: '
|
|
75
|
+
version: '0.2'
|
|
62
76
|
type: :development
|
|
63
77
|
prerelease: false
|
|
64
78
|
version_requirements: !ruby/object:Gem::Requirement
|
|
65
79
|
requirements:
|
|
66
80
|
- - "~>"
|
|
67
81
|
- !ruby/object:Gem::Version
|
|
68
|
-
version: '
|
|
82
|
+
version: '0.2'
|
|
69
83
|
- !ruby/object:Gem::Dependency
|
|
70
84
|
name: railties
|
|
71
85
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -121,10 +135,12 @@ extra_rdoc_files: []
|
|
|
121
135
|
files:
|
|
122
136
|
- LICENSE.txt
|
|
123
137
|
- README.md
|
|
138
|
+
- docs/guide.md
|
|
124
139
|
- exe/shuttlebay
|
|
125
140
|
- lib/rackup/handler/shuttlebay.rb
|
|
126
141
|
- lib/shuttlebay.rb
|
|
127
142
|
- lib/shuttlebay/attached.rb
|
|
143
|
+
- lib/shuttlebay/cache.rb
|
|
128
144
|
- lib/shuttlebay/capybara.rb
|
|
129
145
|
- lib/shuttlebay/engine.rb
|
|
130
146
|
- lib/shuttlebay/handler.rb
|