ocran 1.4.4 → 1.4.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.txt +18 -0
- data/README.md +435 -16
- data/lib/ocran/cosmo_toolchain.rb +383 -0
- data/lib/ocran/direction.rb +592 -58
- data/lib/ocran/option.rb +165 -3
- data/lib/ocran/rubyopt_processor.rb +95 -0
- data/lib/ocran/runner.rb +80 -0
- data/lib/ocran/runtime_environment.rb +19 -1
- data/lib/ocran/stub_builder.rb +23 -6
- data/lib/ocran/version.rb +1 -1
- data/lib/ocran/zip_payload_builder.rb +282 -0
- data/lib/ocran/zip_writer.rb +287 -0
- data/src/Makefile +9 -0
- data/src/script_info.c +15 -2
- data/src/script_info.h +15 -1
- data/src/stub.c +51 -1
- data/src/system_utils_posix.c +40 -0
- data/src/unpack.c +4 -0
- data/src/unpack.h +10 -0
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 85cce39e5b93c48ccd16ecc3ad73f214a2349c2b203ff242fb1d478a78fbd5b9
|
|
4
|
+
data.tar.gz: f6ac990c20617164a68d70cd91e38e42afd12889c247f82204684d1843bb26ae
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 598cfd6fa803707a732ec8c10687f221b2a0618570377655a7414bcb94d9744d59364ade26ddc5b379fd72daf40adf8d001eccdac3d7af0ab20a69dfbafe1500
|
|
7
|
+
data.tar.gz: 83311e9e259a7c91f99ab6860529bd3ede2d706b3737451707f7488291679ae0ceb49bddb110128b6413f7d99b38f4b06bdd0a2362acf56a0afb9269bb41ae82
|
data/CHANGELOG.txt
CHANGED
|
@@ -1,3 +1,21 @@
|
|
|
1
|
+
=== 1.4.5
|
|
2
|
+
- Windows: detected DLLs are now also bundled into bin, next to ruby.exe, whenever they are not already resolvable from where they get packed. The Windows loader resolves a native extension's imports from the extension's own directory, ruby.exe's application directory (bin) plus its ruby_builtin_dlls SxS assembly, and the system directories - PATH is not consulted on hardened systems, and the AddDllDirectory route gems take through ruby_installer/runtime does not exist in a packed app. A DLL loaded from a gem's own tree (e.g. FreeTDS, which tiny_tds ships under ports/), from a devkit's msys64 tree inside the Ruby prefix, or from outside the prefix entirely was packed only at that original location, which the loader never searches, so the packaged application died at require time with a misleading LoadError on machines where a rich PATH did not mask the gap - the out-of-prefix case was skipped entirely by a guard that made its copy_to_bin branch unreachable. DLLs from the Windows directory keep coming from the target system and are never bundled. Companion DLLs found next to native extensions (e.g. libssl-3-x64.dll beside openssl.so in archdir) go into bin as well: a copy in archdir only helps extensions in archdir itself, while the same extension packed at a gem path (openssl and psych are gems since Ruby 3.x) resolves its imports from bin.
|
|
3
|
+
- `--cosmo-ruby`: a native gem now also counts as provided by the payload when the payload can resolve the gem's primary feature, not only when the payload has a gemspec of that name. A gemspec is not what makes a library requirable - an extension linked into the APE, or a library in the interpreter's embedded stdlib, answers `require` with nothing under /zip/lib/ruby/gems/*/specifications - so gems such as cgi and pathname (both compiled into CosmoRuby, and both ordinary native gems on Ruby 3.4+) were reported incompatible and refused builds that work. The probe runs the payload once using $LOAD_PATH.resolve_feature_path, which searches exactly as require does, built-in extensions included, but executes none of the code it finds.
|
|
4
|
+
- A Rails application with SQLite now packages into a single Actually Portable Executable with `--cosmo-ruby` alone (no compiler at all), given an interpreter with sqlite3, nokogiri, puma, nio4r, bigdecimal and racc linked in: 39.0 MB, serving its first request 1.8s after launch, against 50.4 MB and 1.5s for the same application as a native OCRAN executable, and with nothing unpacked at run time. test/test_rails.rb drives the same HTTP assertions - scaffold CRUD through SQLite with CSRF token and session cookie, dynamically added controllers, persistence across a restart - through both builds. The one remaining limitation is cryptographic and belongs to the interpreter: its openssl is an MbedTLS shim with no cipher, HMAC or PBKDF2 surface, and Rails names those at load time, so the application must fill the gap itself, set SECRET_KEY_BASE, ship no config/credentials.yml.enc, and keep the session out of the (encrypted) cookie. See the Rails section of README.md.
|
|
5
|
+
- New `--chdir-exe-dir` option (issue #32): the packaged executable starts the script with its working directory set to the directory containing the executable itself, so relative file access (e.g. reading data files placed next to the .exe) works regardless of how the executable is invoked. Implemented as a new CHDIR_TO_EXE_DIR (0x40) stub header flag; the stub passes `-C <exe dir>` to the packed Ruby interpreter. Mutually exclusive with `--chdir-first`.
|
|
6
|
+
- README: document that `__FILE__`/`__dir__`/`$0` point into the temporary extraction directory at runtime (the cause of issue #32) and show the supported `ENV["OCRAN_EXECUTABLE"]` pattern for locating files next to the executable.
|
|
7
|
+
- `--gemfile <path>` now governs the dependency run too, not just the Gemfile scan that follows it. The script is loaded inside the OCRAN process, so whatever BUNDLE_GEMFILE the environment carried used to win: running OCRAN from inside another project's `bundle exec` made the script's own `require "bundler/setup"` activate that project's bundle, and gems only the application's Gemfile provides went missing. Local development gems declared with `path:` or `gemspec` always did, since they exist nowhere but that Gemfile, so packaging them (issue #34) failed with a LoadError before anything could be packed. OCRAN now points BUNDLE_GEMFILE at the given Gemfile before loading the script, and when a different bundle has already been set up through RUBYOPT it drops Bundler's memoized state so the application's Gemfile is the one that takes effect.
|
|
8
|
+
- Test suite: build and run steps assert through a new assert_system helper that captures the child's stdout and stderr and puts them in the failure message. A failing build used to report nothing but "Expected false to be truthy" while its diagnostics scrolled past in the runner's output.
|
|
9
|
+
- `--cosmo-ruby <ruby.com>` now packages without any compiler when the given cosmopolitan Ruby runs an embedded /zip/main.rb (issue #26). Instead of compiling an APE launcher stub, OCRAN copies the interpreter and injects the application into the interpreter's own ZIP store, so the executable IS the interpreter: no cosmocc, no make, and nothing is unpacked at run time. Startup drops from 0.79s to 0.23s on Linux and from 1.52s to 0.49s on Windows for a 4-gem CLI, no temporary directory is created at all (so none can be leaked when the process is killed), and the executable is about twice as large because it cannot be LZMA-compressed. The mode is selected automatically - `--cosmo-ruby <ruby.com>` alone remains the whole command line - by detecting the capability in the binary; interpreters without it still get the launcher stub, and an explicit `--cosmo <toolchain>` (or --output-dir/--output-zip/--innosetup/--macosx-bundle) forces it. No new command line option.
|
|
10
|
+
- Behavior differences of the ZIP packaging mode, documented in README: the application runs from inside the executable, so $0, __FILE__ and __dir__ point into /zip/ocran/src instead of a temporary directory (files shipping next to the executable are still found via OCRAN_EXECUTABLE, which is unchanged); --chdir-first changes into the executable's directory; of RUBYOPT only -I and -r can be replayed; --icon and --debug-extract have no effect.
|
|
11
|
+
- ZIP packaging now passes the whole command line to the application and reports exact exit codes, including on Windows, given a cosmopolitan Ruby with the 2026-08 interpreter fixes. Previously the interpreter parsed leading option-shaped arguments as its own ("app.com --verbose" failed with "invalid option --verbose", only "app.com -- --verbose" worked) and multiplied Windows exit codes by 256 ("exit 3" arrived as 768). Both were interpreter bugs that also affected plain "ruby.com script.rb"; "--" is now an ordinary argument and interpreter options are reachable through RUBYOPT.
|
|
12
|
+
- The APE launcher stub sets COSMORUBY_WAIT_STATUS_EXIT=1 for the packaged interpreter. The stub is itself a cosmopolitan program: it fork/execs the interpreter and reads the result with WEXITSTATUS(), so with the interpreter's Windows exit-code fix in place an unmarked child's "exit 3" came back as 131 (128 + signal 3). The variable asks the interpreter to keep cosmopolitan's wait-status encoding for this one caller; the stub still reports the plain status to its own native parent.
|
|
13
|
+
- New Ocran::ZipWriter: a minimal, dependency-free ZIP appender (Zlib only) used to inject the application into the APE without shelling out to the zip command, which Windows build hosts do not have. It rewrites the central directory in place, refuses ZIP64 and archives with trailing data, rejects entries that would shadow an existing member, and writes UNIX file-type bits in the external attributes - without S_IFREG/S_IFDIR the members are readable but Ruby's load refuses to open them and directory listings come back empty.
|
|
14
|
+
- New experimental `--cosmo <path>` option (alias `--cosmo-toolchain`): build the launcher stub from its C sources with a Cosmopolitan Libc toolchain (cosmocc) at packaging time and package the application with the resulting Actually Portable Executable (APE) stub instead of the pre-built stub (issue #26). `<path>` is the cosmocc executable or the toolchain directory (containing bin/cosmocc). Console applications only (`--windows` is rejected); requires make and a Linux/macOS build host. The default output name uses the `.com` extension (APE convention); an explicit `--output` is used verbatim. Compiled stubs are cached in ~/.cache/ocran keyed on the toolchain and stub sources; compile errors surface the compiler output.
|
|
15
|
+
- Ship the stub C sources (src/) in the binary platform gems so `--cosmo` can rebuild the stub from an installed gem, not only from a source checkout.
|
|
16
|
+
- `--cosmo-ruby <ruby.com>` is a complete command line on its own: the cosmocc toolchain that builds the APE launcher stub is now discovered on the build host instead of having to be named with `--cosmo` (issue #26). Search order: the COSMOCC environment variable (the cosmocc executable or its install directory; a value that does not resolve is an error, not a silent fallback), cosmocc in PATH, then the conventional install locations ~/.cosmocc/*/bin/cosmocc, ~/cosmocc/*/bin/cosmocc, /opt/cosmocc/*/bin/cosmocc, /opt/cosmo/bin/cosmocc and /usr/local/cosmocc/bin/cosmocc (newest version first). `--cosmo <path>` remains an explicit override that always wins and still works on its own. When no toolchain is found, option parsing fails - before the dependency run - with a message naming the environment variable, PATH, the conventional locations, the download URL and the `--cosmo` flag.
|
|
17
|
+
- New experimental `--cosmo-ruby <ruby.com>` option: package a cosmopolitan-built Ruby APE as the bundled interpreter instead of the host Ruby, so the produced `.com` contains both an APE stub and an APE Ruby and runs with no host-native code (issue #26). The payload's embedded stdlib (in its /zip store) is used at runtime instead of the host's, its embedded gem directory is appended to GEM_PATH, pure-Ruby gems are packed as usual, and native-extension gems are skipped when the payload provides them (json, psych, ...) or abort the build otherwise. The payload is validated by running it once on the build host; a warning is printed when host and payload Ruby versions differ. Verified end-to-end on Linux; console-only, POSIX build hosts only.
|
|
18
|
+
|
|
1
19
|
=== 1.4.4
|
|
2
20
|
- Fix crash on distro-packaged and Homebrew Ruby when a default gem (e.g. fiddle, singleton) has a gemspec but no materialized gem directory: file collection now treats the missing directory as an empty set instead of raising Errno::ENOENT from Find.find (issue #44). The gem's files, which live in the stdlib, are still packed via the load path.
|
|
3
21
|
- Pack the real Ruby interpreter when bindir/ruby is a dispatcher script (e.g. Fedora's rubypick): the running interpreter binary (/proc/self/exe) is packed under the expected name instead, so the packed executable no longer requires Ruby on the target system.
|
data/README.md
CHANGED
|
@@ -31,6 +31,20 @@ If using Windows you can use the Inno Setup
|
|
|
31
31
|
option (`--innosetup`) to produce a proper installer that extracts once to a
|
|
32
32
|
permanent directory.
|
|
33
33
|
|
|
34
|
+
If you want **one** binary that runs on all three systems and starts
|
|
35
|
+
without extracting anything, package a cosmopolitan Ruby with
|
|
36
|
+
`--cosmo-ruby <ruby.com>` (experimental, see
|
|
37
|
+
[Experimental options](#experimental-options)). Grab the interpreter from
|
|
38
|
+
[CosmoRuby releases](https://github.com/Largo/cosmoruby/releases/latest)
|
|
39
|
+
and point OCRAN at it:
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
ocran app.rb --cosmo-ruby ./ruby.com
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
It needs no compiler and no installer, and the same file runs on Linux,
|
|
46
|
+
macOS and Windows, on x86-64 and ARM.
|
|
47
|
+
|
|
34
48
|
You can easily generate binaries for the supported Operating Systems with GitHub
|
|
35
49
|
Actions — see [Building for multiple platforms with GitHub Actions](#building-for-multiple-platforms-with-github-actions).
|
|
36
50
|
|
|
@@ -166,11 +180,169 @@ Fine-tuning flags:
|
|
|
166
180
|
* `--windows`: Force a Windows GUI application (uses `rubyw.exe`). (Windows only)
|
|
167
181
|
* `--console`: Force a console application (uses `ruby.exe`). (Windows only)
|
|
168
182
|
* `--chdir-first`: Change working directory to the app's extraction directory before the script starts.
|
|
183
|
+
* `--chdir-exe-dir`: Change working directory to the directory containing the executable before the script starts. Use this when your app reads or writes files that live next to the `.exe` using relative paths. Cannot be combined with `--chdir-first`.
|
|
169
184
|
* `--icon <ico>`: Replace the default icon with a custom `.ico` file.
|
|
170
185
|
* `--rubyopt <str>`: Set `RUBYOPT` when the executable runs.
|
|
171
186
|
* `--debug`: Enable verbose output when the generated executable runs.
|
|
172
187
|
* `--debug-extract`: Unpack to a local directory and do not delete after execution (useful for troubleshooting).
|
|
173
188
|
|
|
189
|
+
#### Experimental options:
|
|
190
|
+
|
|
191
|
+
* `--cosmo-ruby <ruby.com>`: Package a cosmopolitan-built Ruby (an APE)
|
|
192
|
+
as the bundled interpreter instead of the host Ruby. The produced
|
|
193
|
+
`.com` contains no host-native code at all and runs on Linux, Windows
|
|
194
|
+
and macOS, on x86-64 and ARM:
|
|
195
|
+
|
|
196
|
+
```
|
|
197
|
+
ocran app.rb --cosmo-ruby /path/to/ruby.com
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
**Where to get `ruby.com`.** Download a release from
|
|
201
|
+
[CosmoRuby](https://github.com/Largo/cosmoruby/releases/latest) — one
|
|
202
|
+
file, no installation:
|
|
203
|
+
|
|
204
|
+
```
|
|
205
|
+
curl -L -o ruby.com https://github.com/Largo/cosmoruby/releases/latest/download/ruby.com
|
|
206
|
+
chmod +x ruby.com
|
|
207
|
+
ocran app.rb --cosmo-ruby ./ruby.com
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
Any self-contained cosmopolitan Ruby works, but CosmoRuby is what this
|
|
211
|
+
option is developed and tested against. Releases from `v4.0.6-cosmo3`
|
|
212
|
+
onwards also support the ZIP packaging mode described below, which
|
|
213
|
+
needs no C compiler at all; earlier releases fall back to building a
|
|
214
|
+
launcher stub, which needs `cosmocc`. Building your own is documented
|
|
215
|
+
in [CosmoRuby's BUILDING.md](https://github.com/Largo/cosmoruby/blob/main/BUILDING.md).
|
|
216
|
+
|
|
217
|
+
This one option is the whole command line. The payload APE must be
|
|
218
|
+
fully self-contained, i.e. carry its standard library in its embedded
|
|
219
|
+
ZIP store (`/zip/lib/ruby/...`) — upstream cosmopolitan Ruby builds
|
|
220
|
+
do.
|
|
221
|
+
|
|
222
|
+
**Two ways of producing the executable.** OCRAN picks the better one
|
|
223
|
+
automatically, from the interpreter you give it:
|
|
224
|
+
|
|
225
|
+
| | ZIP packaging (default when supported) | APE launcher stub (fallback) |
|
|
226
|
+
|---|---|---|
|
|
227
|
+
| How | the executable **is** the interpreter, with your application injected into its ZIP store | an APE stub carries the interpreter and your application as a payload |
|
|
228
|
+
| Compiler | none | `cosmocc` (located automatically) |
|
|
229
|
+
| At every start | nothing is unpacked | ~21 MB unpacked into a temp directory |
|
|
230
|
+
| Startup (see below) | 0.23 s Linux / 0.49 s Windows | 0.79 s Linux / 1.5 s Windows |
|
|
231
|
+
| Temp directory | never created | created per run, leaked if the process is killed |
|
|
232
|
+
| Size | interpreter + application | LZMA-compressed, roughly half |
|
|
233
|
+
| `__dir__` of your script | `/zip/ocran/src` (inside the executable) | the temp directory |
|
|
234
|
+
|
|
235
|
+
ZIP packaging is used when the given interpreter runs an embedded
|
|
236
|
+
`/zip/main.rb` on startup (CosmoRuby builds do; OCRAN detects it by
|
|
237
|
+
inspecting the binary). Otherwise OCRAN falls back to the launcher
|
|
238
|
+
stub and locates a `cosmocc` toolchain for it (see `--cosmo` below).
|
|
239
|
+
Passing `--cosmo <toolchain>` explicitly always selects the launcher
|
|
240
|
+
stub, as do `--output-dir`, `--output-zip`, `--innosetup` and
|
|
241
|
+
`--macosx-bundle`, which are not single binaries.
|
|
242
|
+
|
|
243
|
+
**What changes for your application under ZIP packaging** (nothing
|
|
244
|
+
below applies to the launcher stub):
|
|
245
|
+
* Your files are not on disk at run time. `$0`, `__FILE__` and
|
|
246
|
+
`__dir__` point into the executable's archive
|
|
247
|
+
(`/zip/ocran/src/...`), on every platform and with `/` separators.
|
|
248
|
+
Reading, `require`, `require_relative` and `Dir.glob` all work
|
|
249
|
+
there; **writing does not**, and the archive cannot be a working
|
|
250
|
+
directory (`Dir.chdir("/zip")` fails).
|
|
251
|
+
* Files that ship **next to the executable** are unaffected: resolve
|
|
252
|
+
them through `ENV["OCRAN_EXECUTABLE"]`, which is the full path of
|
|
253
|
+
the running `.com` exactly as before (issue #32). Do not use
|
|
254
|
+
`__dir__` for that — it never meant "next to the exe" in either
|
|
255
|
+
mode, but here it is obviously wrong instead of subtly wrong.
|
|
256
|
+
* `--chdir-first` changes into the directory containing the
|
|
257
|
+
executable, since the application directory does not exist on disk.
|
|
258
|
+
* The whole command line reaches `ARGV`, unchanged. The interpreter
|
|
259
|
+
claims none of it, so `app.com --version`, `app.com -v` and
|
|
260
|
+
`app.com -- x` behave exactly as they would for a natively compiled
|
|
261
|
+
program (`--` is an ordinary argument, not a separator). This needs
|
|
262
|
+
a cosmopolitan Ruby with the fix from 2026-08; older builds parsed
|
|
263
|
+
leading option-shaped arguments themselves and failed with
|
|
264
|
+
`invalid option --verbose`.
|
|
265
|
+
* `RUBYOPT` is applied as far as it can be from inside the process:
|
|
266
|
+
`-I` and `-r` are replayed, other flags are reported at build time
|
|
267
|
+
and dropped (the interpreter has already started).
|
|
268
|
+
* `--icon` and `--debug-extract` have no effect and are reported.
|
|
269
|
+
* The executable is not compressed (it has to stay runnable), so
|
|
270
|
+
`--no-lzma` makes no difference to its size; individual packed files
|
|
271
|
+
are deflated.
|
|
272
|
+
* Exit codes are your application's, exactly, on every platform
|
|
273
|
+
including Windows. (Cosmopolitan encodes a POSIX wait status into
|
|
274
|
+
the Windows process exit code, so `exit 3` used to arrive as 768;
|
|
275
|
+
both the interpreter and the launcher stub now report the plain
|
|
276
|
+
status.)
|
|
277
|
+
|
|
278
|
+
Notes that apply to both:
|
|
279
|
+
* The payload is run once on the build host (via `/bin/sh`, using the
|
|
280
|
+
APE's shell-script self-bootstrap) to validate it and to query its
|
|
281
|
+
version and embedded gem directory.
|
|
282
|
+
* Dependency detection still runs under the **host** Ruby. Standard
|
|
283
|
+
library requires resolve at runtime from the payload's embedded
|
|
284
|
+
stdlib (the host stdlib is not packed); a warning is printed when
|
|
285
|
+
host and payload Ruby versions differ.
|
|
286
|
+
* Pure-Ruby application files and pure-Ruby gems are packed as usual
|
|
287
|
+
and activated via `GEM_PATH`. Native gems cannot work (the payload
|
|
288
|
+
is a statically linked `x86_64-cosmo` binary that cannot `dlopen`):
|
|
289
|
+
gems that the payload provides itself (e.g. `json`, `psych`,
|
|
290
|
+
`sqlite3`) are simply not packed, so the payload's own copy serves;
|
|
291
|
+
any other native gem aborts the build. This covers gems built from
|
|
292
|
+
source (`spec.extensions`) as well as **precompiled platform gems**
|
|
293
|
+
(e.g. `sqlite3-2.9.5-x86_64-linux-gnu`), which declare no
|
|
294
|
+
extensions but ship a prebuilt `.so` in their `lib` directory.
|
|
295
|
+
* `--add-all-core` and encoding-support packing are no-ops (the
|
|
296
|
+
payload embeds its complete stdlib and encodings).
|
|
297
|
+
* Linux/macOS build hosts only, console applications only
|
|
298
|
+
(`--windows` is rejected — cosmocc has no GUI `stubw` equivalent).
|
|
299
|
+
The launcher-stub fallback additionally needs `make`. The default
|
|
300
|
+
output name uses the `.com` extension (APE convention), e.g.
|
|
301
|
+
`app.rb` → `app.com`; an explicit `--output` is used verbatim.
|
|
302
|
+
|
|
303
|
+
**Measured** with a 8-file CLI using four pure-Ruby gems (thor,
|
|
304
|
+
terminal-table, rainbow, unicode-display_width) and cosmopolitan Ruby
|
|
305
|
+
4.0.6, average of 10 runs:
|
|
306
|
+
|
|
307
|
+
| Build | Size | Startup (Linux) | Startup (Windows 11) |
|
|
308
|
+
|---|---|---|---|
|
|
309
|
+
| ZIP packaging | 21.5 MB | 0.23 s | 0.49 s |
|
|
310
|
+
| Launcher stub, `--no-lzma` | 22.8 MB | 0.24 s | 1.09 s |
|
|
311
|
+
| Launcher stub, LZMA (default) | 11.3 MB | 0.79 s | 1.52 s |
|
|
312
|
+
|
|
313
|
+
The stub's cost is unpacking ~21 MB per run, which LZMA turns into
|
|
314
|
+
decompression time: on Linux with a warm page cache an uncompressed
|
|
315
|
+
stub build is nearly as fast, on Windows it is not. ZIP packaging
|
|
316
|
+
trades the smaller, compressed artifact for constant startup and no
|
|
317
|
+
disk writes at all.
|
|
318
|
+
|
|
319
|
+
* `--cosmo <path>` (alias `--cosmo-toolchain`): Name the
|
|
320
|
+
[Cosmopolitan Libc](https://github.com/jart/cosmopolitan) `cosmocc`
|
|
321
|
+
toolchain that builds the APE launcher stub from its C sources at
|
|
322
|
+
packaging time, overriding the automatic lookup. `<path>` is either
|
|
323
|
+
the `cosmocc` executable itself or the toolchain directory (one
|
|
324
|
+
containing `bin/cosmocc`, e.g. an unpacked
|
|
325
|
+
[cosmocc.zip](https://cosmo.zip/pub/cosmocc/cosmocc.zip)).
|
|
326
|
+
Notes:
|
|
327
|
+
* With `--cosmo-ruby`, naming a toolchain explicitly is also how you
|
|
328
|
+
ask for the launcher stub instead of ZIP packaging (for the
|
|
329
|
+
extraction semantics: a real on-disk application directory, and
|
|
330
|
+
`ARGV` untouched by the interpreter's option parser).
|
|
331
|
+
* When a toolchain is needed but not named, `--cosmo-ruby` looks for
|
|
332
|
+
one in this order: the `COSMOCC` environment variable (the `cosmocc` executable
|
|
333
|
+
or its install directory), `cosmocc` in `PATH`, and finally the
|
|
334
|
+
conventional install locations `~/.cosmocc/*/bin/cosmocc`,
|
|
335
|
+
`~/cosmocc/*/bin/cosmocc`, `/opt/cosmocc/*/bin/cosmocc`,
|
|
336
|
+
`/opt/cosmo/bin/cosmocc` and `/usr/local/cosmocc/bin/cosmocc`
|
|
337
|
+
(newest version first). If none is found the build stops with a
|
|
338
|
+
message naming all three mechanisms.
|
|
339
|
+
* Compiled stubs are cached in `~/.cache/ocran` (keyed on the
|
|
340
|
+
toolchain and stub sources), so only the first build compiles.
|
|
341
|
+
* Given **without** `--cosmo-ruby`, the packaged Ruby runtime is still
|
|
342
|
+
the host platform's Ruby — the APE property then applies to the
|
|
343
|
+
launcher stub, not to the bundled application. See
|
|
344
|
+
`docs/cosmocc-port-plan.md` for status.
|
|
345
|
+
|
|
174
346
|
### Compilation:
|
|
175
347
|
|
|
176
348
|
* OCRAN runs your script (using `Kernel#load`) and builds the output when it exits.
|
|
@@ -180,8 +352,9 @@ Fine-tuning flags:
|
|
|
180
352
|
|
|
181
353
|
### Running your application:
|
|
182
354
|
|
|
183
|
-
* The working directory is not changed by OCRAN unless you use `--chdir-first
|
|
184
|
-
directory
|
|
355
|
+
* The working directory is not changed by OCRAN unless you use `--chdir-first`
|
|
356
|
+
(extraction directory) or `--chdir-exe-dir` (directory containing the
|
|
357
|
+
executable). See "Working directory" below.
|
|
185
358
|
* When a `.exe` is running, `OCRAN_EXECUTABLE` points to the `.exe` with its full path.
|
|
186
359
|
* The temporary location of the script is available via `$0`.
|
|
187
360
|
* OCRAN does not set up the include path. Add `$:.unshift File.dirname($0)` at the start of your script if you need to `require` additional files from the same directory as your main script.
|
|
@@ -443,6 +616,55 @@ Four modes:
|
|
|
443
616
|
If files are missing from the output, try `--gem-all=gemname` first, then
|
|
444
617
|
`--gem-full=gemname`. Use `--gem-full` to include everything for all gems.
|
|
445
618
|
|
|
619
|
+
### Using OCRAN under `bundle exec`
|
|
620
|
+
|
|
621
|
+
OCRAN loads your script inside its own process to find out what it depends
|
|
622
|
+
on, so the bundle OCRAN itself runs under is the bundle that dependency run
|
|
623
|
+
sees. That matters as soon as the two are not the same.
|
|
624
|
+
|
|
625
|
+
**Running `bundle exec ocran app.rb` requires `ocran` in the application's
|
|
626
|
+
Gemfile.** This is Bundler's rule, not OCRAN's: `bundle exec` refuses to run
|
|
627
|
+
a command the bundle does not contain ("ocran is not currently included in
|
|
628
|
+
the bundle"). Add `gem "ocran"` to the Gemfile, or install OCRAN with `gem
|
|
629
|
+
install ocran` and call `ocran` without `bundle exec`. Be aware that every
|
|
630
|
+
gem the Gemfile lists is packed into the application when you build with
|
|
631
|
+
`--gemfile`, OCRAN included.
|
|
632
|
+
|
|
633
|
+
**From the application's own directory it just works.** `bundle exec ocran
|
|
634
|
+
app.rb`, with or without `--gemfile Gemfile`, packages the application
|
|
635
|
+
against its own bundle - including gems declared with `path:` or `gemspec`,
|
|
636
|
+
which exist nowhere but that Gemfile.
|
|
637
|
+
|
|
638
|
+
**Packaging one project from inside another project's bundle needs
|
|
639
|
+
`--gemfile`.** If the bundle you are in is not the application's, pass
|
|
640
|
+
`--gemfile path/to/app/Gemfile`. It governs the dependency run as well as
|
|
641
|
+
the gem list, so a `require "bundler/setup"` in your script sets up the
|
|
642
|
+
application's bundle and not the one you happen to be standing in. Without
|
|
643
|
+
it the script is loaded under the wrong bundle and usually dies with a
|
|
644
|
+
`LoadError` for one of its own gems; OCRAN warns when it can see that this
|
|
645
|
+
is what you are doing.
|
|
646
|
+
|
|
647
|
+
Gems that the surrounding bundle merely activated are not packed. The
|
|
648
|
+
contents are decided by the application's own Gemfile and by what the
|
|
649
|
+
dependency run actually loads, so building a small application from inside a
|
|
650
|
+
large development bundle does not drag that bundle along.
|
|
651
|
+
|
|
652
|
+
**The packaged executable is independent of Bundler.** It carries its own
|
|
653
|
+
gems and its own Gemfile, and it clears the variables through which Bundler
|
|
654
|
+
hands a bundle to a child process (`BUNDLER_SETUP`, `BUNDLE_GEMFILE`,
|
|
655
|
+
`BUNDLE_LOCKFILE`, `RUBYOPT`), so it runs correctly even when started from a
|
|
656
|
+
`bundle exec` shell, from a Rakefile, or from another Ruby program.
|
|
657
|
+
|
|
658
|
+
**Exception: executables built with `--cosmo-ruby` cannot do that.** There
|
|
659
|
+
the cosmopolitan Ruby *is* the executable, so its RubyGems has already acted
|
|
660
|
+
on `BUNDLER_SETUP` before any packed code can run, and the application
|
|
661
|
+
aborts with `Bundler::GemNotFound` over gems of the bundle it was launched
|
|
662
|
+
from. Start such an executable outside a bundle, or clear the variables
|
|
663
|
+
first - `Bundler.with_original_env { system("./app.com") }` from Ruby, or
|
|
664
|
+
`env -u BUNDLER_SETUP -u BUNDLE_GEMFILE ./app.com` from a shell. Building
|
|
665
|
+
with `--cosmo-ruby` under `bundle exec` is fine; only launching the result
|
|
666
|
+
from inside a bundle is not.
|
|
667
|
+
|
|
446
668
|
### Code-signing a macOS app bundle
|
|
447
669
|
|
|
448
670
|
After building with `--macosx-bundle`, sign the bundle with your Developer ID:
|
|
@@ -516,26 +738,51 @@ output, `OCRAN_EXECUTABLE` is set to the full path of the running executable:
|
|
|
516
738
|
|
|
517
739
|
### Working directory
|
|
518
740
|
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
741
|
+
By default the OCRAN executable does not change the working directory when it
|
|
742
|
+
starts. Two opt-in build options change this:
|
|
743
|
+
|
|
744
|
+
* `--chdir-first`: the working directory is always the common parent
|
|
745
|
+
directory of your source files inside the extraction directory. Do not use
|
|
746
|
+
this if your application takes filenames as command-line arguments.
|
|
747
|
+
* `--chdir-exe-dir`: the working directory is the directory that contains the
|
|
748
|
+
executable itself. Use this when your application reads or writes files that
|
|
749
|
+
are placed next to the `.exe` (config files, spreadsheets, output folders,
|
|
750
|
+
...) using relative paths. Note that relative filenames passed as
|
|
751
|
+
command-line arguments will then also resolve against the executable's
|
|
752
|
+
directory instead of the invoker's current directory.
|
|
753
|
+
|
|
754
|
+
Without these options, you should not assume anything about the current
|
|
755
|
+
working directory when your executable is invoked. It can be the directory
|
|
756
|
+
where the executable is placed (when invoked through the Windows Explorer),
|
|
757
|
+
the users' current working directory (when invoking from the Command Prompt),
|
|
758
|
+
or even `C:\\WINDOWS\\SYSTEM32` when the executable is invoked through a file
|
|
759
|
+
association.
|
|
533
760
|
|
|
534
761
|
To `require` additional files from the source directory while keeping the
|
|
535
762
|
user's working directory:
|
|
536
763
|
|
|
537
764
|
$LOAD_PATH.unshift File.dirname($0)
|
|
538
765
|
|
|
766
|
+
### Finding files next to the executable
|
|
767
|
+
|
|
768
|
+
Be aware that `__FILE__`, `__dir__` and `$0` inside a packaged application
|
|
769
|
+
point to the extracted copy of your script in the temporary extraction
|
|
770
|
+
directory — not to the directory containing the `.exe`. Anchoring paths on
|
|
771
|
+
`__dir__` (e.g. `APP_ROOT = File.expand_path(__dir__)`) therefore makes the
|
|
772
|
+
application look for its data files inside the temporary directory, which is
|
|
773
|
+
usually not what you want.
|
|
774
|
+
|
|
775
|
+
To locate files relative to the executable, either build with
|
|
776
|
+
`--chdir-exe-dir` and use plain relative paths, or anchor your paths on the
|
|
777
|
+
`OCRAN_EXECUTABLE` environment variable, which is always set to the full path
|
|
778
|
+
of the running executable:
|
|
779
|
+
|
|
780
|
+
APP_ROOT = if ENV["OCRAN_EXECUTABLE"]
|
|
781
|
+
File.dirname(ENV["OCRAN_EXECUTABLE"])
|
|
782
|
+
else
|
|
783
|
+
__dir__ # plain `ruby myapp.rb` during development
|
|
784
|
+
end
|
|
785
|
+
|
|
539
786
|
### Detecting OCRAN at build time
|
|
540
787
|
|
|
541
788
|
Check for the `Ocran` constant to detect whether OCRAN is currently building
|
|
@@ -614,6 +861,178 @@ file:
|
|
|
614
861
|
end
|
|
615
862
|
end
|
|
616
863
|
|
|
864
|
+
### Packaging a Rails application
|
|
865
|
+
|
|
866
|
+
A Rails application with a SQLite database packages into a single
|
|
867
|
+
executable that serves HTTP. `test/test_rails.rb` builds one from scratch
|
|
868
|
+
on every run (`rails new`, `bin/rails generate scaffold`, plus controllers
|
|
869
|
+
and views written on the fly), packages it and drives a full CRUD round
|
|
870
|
+
trip through the resulting binary; read it as the worked example. It is
|
|
871
|
+
opt-in — run it with `OCRAN_RAILS_TEST=1 ruby -Itest test/test_rails.rb`.
|
|
872
|
+
|
|
873
|
+
Write a small entry script next to `config/` (Rails' own `config.ru` is
|
|
874
|
+
for a Rack server, not for OCRAN) and point OCRAN at it together with the
|
|
875
|
+
directories Rails reads at run time:
|
|
876
|
+
|
|
877
|
+
ocran server.rb app config db public --no-autoload --gem-all
|
|
878
|
+
|
|
879
|
+
Four things are worth knowing before you try it.
|
|
880
|
+
|
|
881
|
+
**`--no-autoload` is required.** OCRAN's autoload walker sweeps every
|
|
882
|
+
`Module` in `ObjectSpace` and tries to `const_get` each autoloaded
|
|
883
|
+
constant. Under Rails that means dragging in `I18n::Tests`,
|
|
884
|
+
`Prism::Translation` and everything else the framework declares but never
|
|
885
|
+
loads — minutes of output ending in an error. In production Rails eager
|
|
886
|
+
loads the application itself, so the walker has nothing to contribute.
|
|
887
|
+
|
|
888
|
+
**`--gem-all` is required.** Dependency detection is `$LOADED_FEATURES`
|
|
889
|
+
based, and Rails reads a great deal it never `require`s. The first thing
|
|
890
|
+
to break without it is `activesupport/lib/active_support/locale/en.rb`,
|
|
891
|
+
which I18n reads as a data file during boot. For the same reason the
|
|
892
|
+
application's own `app`, `config`, `db` and `public` directories have to
|
|
893
|
+
be named on the command line: ERB templates, `database.yml`, locale files
|
|
894
|
+
and migrations are not `require`d and so are invisible to the dependency
|
|
895
|
+
run.
|
|
896
|
+
|
|
897
|
+
**Gems Rails declares but never loads still have to be inside.** OCRAN
|
|
898
|
+
packs the gemspec of every gem it detects, and RubyGems activates the
|
|
899
|
+
complete declared dependency graph of a gemspec, not just the code you
|
|
900
|
+
actually load. `activesupport` declares `minitest` and `drb`, `railties`
|
|
901
|
+
declares `rake`; a running server loads none of them, so nothing puts them
|
|
902
|
+
in the package, and the executable then dies at startup with
|
|
903
|
+
`Gem::MissingSpecError`. Require them from the entry script:
|
|
904
|
+
|
|
905
|
+
begin
|
|
906
|
+
gem "minitest", "~> 5.0" # 5.x has no dependencies of its own
|
|
907
|
+
rescue Gem::LoadError
|
|
908
|
+
end
|
|
909
|
+
require "minitest" # activesupport
|
|
910
|
+
require "drb" # activesupport
|
|
911
|
+
require "rake" # railties
|
|
912
|
+
|
|
913
|
+
To find the equivalent list for your own application, boot it and diff the
|
|
914
|
+
declared dependencies against what is loaded:
|
|
915
|
+
|
|
916
|
+
Gem.loaded_specs.each_value do |spec|
|
|
917
|
+
spec.runtime_dependencies.each do |dep|
|
|
918
|
+
puts "#{dep.name} <- #{spec.name}" unless Gem.loaded_specs.key?(dep.name)
|
|
919
|
+
end
|
|
920
|
+
end
|
|
921
|
+
|
|
922
|
+
**Bundler.** The dependency run happens inside the OCRAN process, so a
|
|
923
|
+
`bundler/setup` in `config/boot.rb` confines it to your bundle; use
|
|
924
|
+
`--gemfile` to point OCRAN at the same `Gemfile`. The test takes the other
|
|
925
|
+
route and removes Bundler from the generated application entirely, letting
|
|
926
|
+
RubyGems activate the gems, which is why it needs neither `bundle install`
|
|
927
|
+
nor the network.
|
|
928
|
+
|
|
929
|
+
#### Where the database goes
|
|
930
|
+
|
|
931
|
+
**Not inside the package.** With the native stub the application directory
|
|
932
|
+
is a temporary directory that is created at every start and deleted when
|
|
933
|
+
the process exits: a database written there is thrown away. With
|
|
934
|
+
`--cosmo-ruby` it is a read-only ZIP store inside the executable and the
|
|
935
|
+
write simply fails.
|
|
936
|
+
|
|
937
|
+
Put mutable state next to the executable instead, using
|
|
938
|
+
`ENV["OCRAN_EXECUTABLE"]` (see [Environment variables](#environment-variables)),
|
|
939
|
+
and tell Rails about it before `config/environment` is loaded:
|
|
940
|
+
|
|
941
|
+
base = ENV["OCRAN_EXECUTABLE"] ? File.dirname(File.expand_path(ENV["OCRAN_EXECUTABLE"])) : __dir__
|
|
942
|
+
data_dir = File.join(base, "myapp-data")
|
|
943
|
+
FileUtils.mkdir_p(data_dir)
|
|
944
|
+
|
|
945
|
+
ENV["DATABASE_URL"] ||= "sqlite3:#{File.join(data_dir, "app.sqlite3")}"
|
|
946
|
+
|
|
947
|
+
`DATABASE_URL` is the cleanest lever, because `config/database.yml`
|
|
948
|
+
resolves relative paths against `Rails.root`, which is inside the package.
|
|
949
|
+
The same applies to everything else Rails writes:
|
|
950
|
+
|
|
951
|
+
* **tmp** (cache, pids, sockets) — set `config.paths["tmp"]`, or avoid it
|
|
952
|
+
altogether with `config.cache_store = :memory_store`.
|
|
953
|
+
* **log** — the generated production environment already logs to stdout;
|
|
954
|
+
if yours does not, set `config.logger` explicitly.
|
|
955
|
+
* **Active Storage** — point `config.active_storage.service` at a root
|
|
956
|
+
under your data directory.
|
|
957
|
+
|
|
958
|
+
Ship the schema in the package and apply it on first run, rather than
|
|
959
|
+
shipping a prebuilt database file:
|
|
960
|
+
|
|
961
|
+
ActiveRecord::MigrationContext.new(Rails.root.join("db/migrate").to_s).migrate
|
|
962
|
+
|
|
963
|
+
Calling this during the dependency run as well is what gets
|
|
964
|
+
`db/migrate/*.rb` into `$LOADED_FEATURES`, and therefore into the package.
|
|
965
|
+
|
|
966
|
+
#### Rails as a single portable executable (`--cosmo-ruby`)
|
|
967
|
+
|
|
968
|
+
The same application packages into one Actually Portable Executable that
|
|
969
|
+
runs on Linux, macOS, Windows and the BSDs without an installed Ruby:
|
|
970
|
+
|
|
971
|
+
ocran server.rb app config db public --no-autoload --gem-all \
|
|
972
|
+
--cosmo-ruby /path/to/ruby.com
|
|
973
|
+
|
|
974
|
+
No compiler is involved: `--cosmo-ruby` on its own selects the ZIP mode,
|
|
975
|
+
in which the application is injected into the interpreter's own ZIP store.
|
|
976
|
+
Nothing is unpacked at run time — the application is read straight out of
|
|
977
|
+
the executable. A Rails 8.1 application built this way measures **39.0 MB
|
|
978
|
+
and answers its first request 1.8 s after launch**, against 50.4 MB and
|
|
979
|
+
1.5 s for the same application in a native OCRAN executable.
|
|
980
|
+
|
|
981
|
+
An APE cannot `dlopen`, so **every native extension has to be compiled
|
|
982
|
+
into the interpreter you point at**. For Rails that is `sqlite3`,
|
|
983
|
+
`nokogiri` (and with it libxml2 and libxslt), `puma`, `nio4r`,
|
|
984
|
+
`bigdecimal` and `racc`. CosmoRuby has had all of them since 4.0.6;
|
|
985
|
+
against an interpreter that has not, the build stops before producing
|
|
986
|
+
anything, naming the gem:
|
|
987
|
+
|
|
988
|
+
ERROR: Gem nokogiri-1.19.4-x86_64-linux-gnu is native (ships prebuilt
|
|
989
|
+
binaries (nokogiri.so)) and cannot run under the packed cosmopolitan
|
|
990
|
+
Ruby (x86_64-cosmo, static)
|
|
991
|
+
|
|
992
|
+
OCRAN drops the host copy of each gem the payload provides and lets the
|
|
993
|
+
payload's own serve. It recognises those either by a gemspec in the
|
|
994
|
+
payload or, failing that, by asking the payload whether it can resolve the
|
|
995
|
+
gem's primary feature at all: an extension linked into the binary, or a
|
|
996
|
+
library in its embedded stdlib (`cgi` and `pathname` are both), answers
|
|
997
|
+
`require` with no gemspec anywhere.
|
|
998
|
+
|
|
999
|
+
**One real limitation remains, and it is cryptographic.** CosmoRuby's
|
|
1000
|
+
`openssl` is a shim over MbedTLS: no `OpenSSL::Cipher`, no
|
|
1001
|
+
`OpenSSL::HMAC`, no PBKDF2, no `OpenSSL::Digest` class hierarchy. Rails
|
|
1002
|
+
does not merely use those, it *names* them at load time — `require "rails"`
|
|
1003
|
+
by itself dies on
|
|
1004
|
+
|
|
1005
|
+
active_support/message_encryptor.rb:116:in '<class:MessageEncryptor>':
|
|
1006
|
+
uninitialized constant OpenSSL::Cipher (NameError)
|
|
1007
|
+
|
|
1008
|
+
so an application packaged for such an interpreter has to do three things.
|
|
1009
|
+
`test/rails_app_generator.rb` does all three, and the file it writes,
|
|
1010
|
+
`openssl_gap.rb`, is a no-op on a Ruby with a complete `openssl`:
|
|
1011
|
+
|
|
1012
|
+
* **Fill in the missing pieces before Rails loads.** HMAC, PBKDF2,
|
|
1013
|
+
`OpenSSL.fixed_length_secure_compare` and an `OpenSSL::Digest` class
|
|
1014
|
+
hierarchy are a few lines each on top of Ruby's own `Digest`, and they
|
|
1015
|
+
are the real algorithms. `OpenSSL::Cipher` is not: it exists so that
|
|
1016
|
+
Rails can name the constant, and raises if anything tries to encrypt
|
|
1017
|
+
with it. A fake cipher would be worse than no cipher.
|
|
1018
|
+
* **Set `SECRET_KEY_BASE`, and ship no `config/credentials.yml.enc`.**
|
|
1019
|
+
Reading credentials decrypts them with AES-256-GCM and fails with
|
|
1020
|
+
`OpenSSL::Cipher::CipherError`. Dropping the file is the right move in
|
|
1021
|
+
any case: packaging it means packaging `config/master.key` beside it,
|
|
1022
|
+
i.e. handing the key to everyone who gets a copy of the executable.
|
|
1023
|
+
* **Keep the session off the cookie.** Rails' default `CookieStore`
|
|
1024
|
+
encrypts the session cookie, so it cannot work either;
|
|
1025
|
+
`config.session_store :cache_store` keeps the session server-side and
|
|
1026
|
+
puts only its id in the cookie. Signed cookies, the CSRF token and
|
|
1027
|
+
everything else that needs no more than an HMAC keep working, and a full
|
|
1028
|
+
scaffold CRUD round trip with CSRF token and session cookie is what the
|
|
1029
|
+
test drives through the packaged `.com`.
|
|
1030
|
+
|
|
1031
|
+
Active Record encryption, encrypted cookies and anything else that
|
|
1032
|
+
actually encrypts stay out of reach until CosmoRuby's `openssl` grows a
|
|
1033
|
+
cipher surface — mbedtls has AES, GCM and PKCS5, so it is a bounded job;
|
|
1034
|
+
it is tracked in `PORTING-NOTES.md` in the CosmoRuby repository.
|
|
1035
|
+
|
|
617
1036
|
## See elsewhere
|
|
618
1037
|
|
|
619
1038
|
- [State of Ruby Packagers](https://gist.github.com/YOU54F/3775e66e6090e0371c11601e6b75c305)
|