ocran 1.5.0 → 1.5.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d437febbbe8332ec9c9361a7ab7cd7084d323d1afc92e81105ff73e45217d194
-  data.tar.gz: e4cc31077e27c6019037efaae65d552fcb72e3621a510773533629192f512dcb
+  metadata.gz: da09ddcc836fd862e4b8832daedf24ea3dc8c122779d19528e0d5611ef4b47ff
+  data.tar.gz: c231264021b94d991f197f9520ec5403445756707c0aca60c1841a72332754df
 SHA512:
-  metadata.gz: a67462ca9e1187c8d83afa9c6665b166b0811680baf7a3c798f99cdd9fb7bcb2c3f8555f0acf88725e0fb803bc22c5fde24c40c14cf8142a62f1ffdba25ede1f
-  data.tar.gz: 35fcb3c1de9f04c1b21a7de1c9d68c395d9fb6ea8563f3a891c5008f499138f03b533bf3d2546e19a57c3ae22243a705c5d9d4990d2e4dcdd4663d1b9b90074f
+  metadata.gz: bdc488641cfdca101ebd4e2b01168838fd15d3eb418606ebaf2b2af431a08fa9e92da3ecf959ee0a485f669ea79af0707e842396011916d37365f8e553eeadcf
+  data.tar.gz: ae524b14a5b1198a8f9f761b6e38ee0af06cb6843827b10dade4d9fb8d37585066cee7866ca3fa554f03825f7293a92f32bed0e28cea0380cc3ee5d3f7062154

data/CHANGELOG.txt

@@ -1,3 +1,17 @@
+=== 1.5.1
+- Releases carry the three `--static-runtime` runtimes (Windows zip, Linux and macOS tarballs) next to the gems: the release workflow calls the runtime workflows, which build, test and attest them, and attaches what they built (a published release is immutable, so they cannot be added afterwards). A manual run of the release workflow is a dry run that builds and tests everything a release would contain and publishes nothing. The runtime workflows run on branch pushes only; for a tag the release workflow runs them.
+- The macOS runtime and macOS executables are reproducible: the runtime build sets ZERO_AR_DATE=1 (Apple's ar, libtool and ranlib wrote each member's modification time into the static libraries, and ld the objects' times into the executable's debug map), and OCRAN links macOS executables with -Wl,-S, without a debug map naming objects compiled in a temporary directory. The runtime workflows package the smoke application twice and require the same bytes.
+- A bare bin/ruby(.exe) of a packaged application gets the launch script's environment: OCRAN packs a RubyGems startup hook (rubygems/defaults/operating_system.rb in the packed site_ruby) that sets the variables the launch script exports (GEM_HOME, GEM_PATH, RUBYLIB, SSL_CERT_FILE/SSL_CERT_DIR when a CA bundle is packed, ...) when they are not set, with paths relative to where the package is, and then loads the Ruby's own hook (RubyInstaller's). A Windows service or scheduled task registered as `bin\ruby.exe src\service.rb` - the way a service has to run, since the service control manager only accepts the process it started itself - could not load the packaged gems (LoadError) and used the account's own gem directory. New README section "Running as a Windows service".
+- The standard library's unicode_normalize files are always packed. String#unicode_normalize is core but loads its implementation and tables from the standard library on first use, so an application that did not call it during the dependency run failed once packed with "cannot load such file -- unicode_normalize/normalize.rb".
+- A build that needs a launcher stub which was never compiled (OCRAN used from a git checkout, e.g. a Bundler git source) fails at once with a message saying how to build it (`make -C src install`; `ridk exec make -C src install` with RubyInstaller) or, for directory and zip output, to pass --no-wrapper-exe; it used to fail with a bare ENOENT for share/ocran/stub.exe after the whole dependency run.
+- `rake build` and `rake build_stub` work without the development bundle installed: the Rakefile sets up Bundler only for tasks other than building the stubs.
+- --verbose reports "Failed to load gemspec" only for files in gem directories; it listed every standard library file the application loaded (hundreds of lines on Ruby 3.4), which buried real errors.
+- The release workflow pushes every gem to rubygems.org with a Sigstore attestation (signed with sigstore-cli, as rubygems/release-gem does), so rubygems.org shows the gems as built and signed on GitHub Actions; the GitHub attestation alone does not reach rubygems.org.
+- A library that exits while OCRAN loads its autoload constants no longer ends the build: it is reported as a warning, like an autoload that raises. rdoc/servlet.rb aborts when webrick is missing (Ruby 3.4 no longer ships it), so packaging an application that loaded rdoc failed with exit 1, and the abort message went to stderr ahead of the buffered progress output, at the start of the log rather than the end.
+- Packaging no longer aborts with "No such file or directory" over an entry of $LOADED_FEATURES that has no file behind it. A racc parser generated with its runtime embedded marks <dir>/racc/parser.rb and racc/info.rb as loaded so that racc is not loaded twice; rdoc's RD parsers in Ruby 3.4 do, so an application that loaded them (directly or through a gem) failed with ".../lib/ruby/3.4.0/rdoc/rd/racc/parser.rb (Errno::ENOENT)" (reported on a Windows runner with Ruby 3.4.11 and ocran 1.4.6). Such entries are skipped; --verbose names them.
+- `--add-all-core` no longer aborts over an entry of the Ruby library directories that is not a readable file, e.g. Debian's packaged Ruby, whose rdoc fonts are links into fonts-lato: it packs the rest and warns which entries it left out. A configured library directory that does not exist (Debian's sitelibdir, /usr/local/lib/site_ruby/<version>) is skipped; it failed with "Unexpected library path format".
+- README: a one-line definition and quick start at the top, an FAQ, a comparison with other Ruby packagers and a section on security, dependencies and reproducibility; llms.txt summarises OCRAN for LLM-based search. The gemspec links the issue tracker and the README (bug_tracker_uri, documentation_uri) for rubygems.org.
+
 === 1.5.0
 - CI builds all three `--static-runtime` runtimes and tests them before publishing each as a workflow artifact: new static-runtime-windows.yml cross-builds the Windows runtime on Linux (Debian 13, in the official ruby image pinned by digest) and tests it on Windows (DLL imports, executables packaged on Linux run with a minimal PATH, packaging on Windows with nokogiri built by MSYS2's gcc). Each runtime is published as one reproducible archive (runtime/pack-runtime.rb; a zip for Windows, a .tar.gz for Linux and macOS), and a runtime that passed its tests gets a signed build provenance attestation for the archive and its runtime executable: `gh attestation verify <file> -R Largo/ocran`. runtime/smoke/run.rb runs executables with a minimal PATH and without Ruby's environment variables, and can run executables packaged elsewhere (OCRAN_SMOKE_PREBUILT=1).
 - The release workflow attests every gem it publishes (signed build provenance): `gh attestation verify ocran-<version>-<platform>.gem -R Largo/ocran` checks a gem from the GitHub release or from rubygems.org (`gem fetch ocran`).

data/README.md

@@ -1,12 +1,29 @@
-# OCRAN
+# OCRAN — turn Ruby scripts into standalone executables (.exe) for Windows, Linux and macOS
 
 ![OCRAN: ship your Ruby app as one file, no Ruby needed to run it](https://raw.githubusercontent.com/Largo/ocran/master/docs/social/twitter-card.png)
 
-[OCRAN (One-Click Ruby Application Next)](https://github.com/largo/ocran) packages Ruby applications for
+[![Gem Version](https://badge.fury.io/rb/ocran.svg)](https://rubygems.org/gems/ocran)
+
+**OCRAN (One-Click Ruby Application Next) is a Ruby-to-executable packager:
+`ocran app.rb` turns a Ruby script into a single `.exe` (Windows) or native
+binary (Linux, macOS) that runs on machines without Ruby installed.** It is
+the maintained successor of [OCRA](https://github.com/larsch/ocra) for
+Ruby 3.3 and newer.
+
+[OCRAN](https://github.com/largo/ocran) packages Ruby applications for
 distribution. It bundles your script, the Ruby interpreter, gems, and native
 libraries into a single self-contained artifact that runs without requiring
 Ruby to be installed on the target machine.
 
+Quick start:
+
+```
+gem install ocran
+ocran app.rb          # → app.exe on Windows, ./app on Linux/macOS
+```
+
+See the [FAQ](#faq) and the [comparison with other Ruby packagers](#comparison-with-other-ruby-packagers).
+
 OCRAN supports four output formats, all cross-platform:
 
 * **Self-extracting executable** — bundles everything into a single binary that unpacks and runs transparently, with no Ruby installation required. Produces a `.exe` on Windows, and a native executable on Linux and macOS.
@@ -256,7 +273,8 @@ goes through these steps in order of impact.
   built. CI builds and tests the three runtimes and publishes each as a
   reproducible archive (a zip for Windows, a tarball otherwise) with a
   signed build provenance attestation
-  (`gh attestation verify <archive> -R Largo/ocran`).
+  (`gh attestation verify <archive> -R Largo/ocran`); releases from 1.5.1
+  on have them attached.
 
   Dependency detection runs as usual, under the build host's Ruby, which
   must have the runtime's API version (e.g. 4.0.0). The standard library
@@ -643,6 +661,41 @@ On Windows, run the batch file:
 
     myapp\script.bat
 
+### Running as a Windows service
+
+The self-extracting `.exe`, the wrapper `.exe` and the launch script all
+start the packed `ruby.exe` as a child process. The Windows service control
+manager accepts the service dispatcher only from the process it started
+itself, so a service registered through any of them fails with error 1063
+("The service process could not connect to the service controller").
+
+Build with `--output-dir` (or `--innosetup`) instead and register the packed
+interpreter directly, e.g. with `sc create` or the
+[win32-service](https://github.com/chef/win32-service) gem's
+`Win32::Service.create`:
+
+    ocran service.rb --output-dir C:\MyService
+    sc create MyService binPath= "C:\MyService\bin\ruby.exe C:\MyService\src\service.rb"
+
+That bare `ruby.exe` still gets the package's environment: OCRAN packs a
+RubyGems startup hook (`lib/ruby/site_ruby/<version>/rubygems/defaults/operating_system.rb`)
+that sets `GEM_HOME`, `GEM_PATH`, `RUBYLIB`, `SSL_CERT_FILE` and the other
+variables of the launch script to the directory's actual location, unless
+they are already set, clears those the launch script clears
+(`BUNDLE_GEMFILE`), and then loads the Ruby installation's own one
+(RubyInstaller's). OCRAN leaves the hook out, and warns, when the building
+Ruby has such a file in its `site_ruby` already or keeps `site_ruby` outside
+its installation directory; then set those variables in the service's
+environment yourself. The hook does not change the working directory (a service
+starts in `C:\Windows\System32`), so find your files from `__dir__`. If the
+script the launch script runs is a generated `ocran-rubyopt-launcher.rb`
+(`-I`/`-r` in `RUBYOPT`), register that one.
+
+OCRAN runs the script at build time, where the service's `mainloop` cannot
+work: skip it with `defined?(Ocran)` (see "Detecting OCRAN at build time")
+and `require` there what the service only loads later, such as `net/http`,
+so that it gets packed.
+
 ### Pitfalls:
 
 * Avoid modifying load paths at runtime. Use `-I` or `RUBYLIB` if needed, but don't expect OCRAN to preserve them for runtime. OCRAN may pack sources into other directories than you
@@ -1320,10 +1373,176 @@ actually encrypts stay out of reach until CosmoRuby's `openssl` grows a
 cipher surface — mbedtls has AES, GCM and PKCS5, so it is a bounded job;
 it is tracked in `PORTING-NOTES.md` in the CosmoRuby repository.
 
-## See elsewhere
+## FAQ
+
+### How do I convert a Ruby script into an .exe?
 
-- [State of Ruby Packagers](https://gist.github.com/YOU54F/3775e66e6090e0371c11601e6b75c305)
-- [Traveling Ruby](https://github.com/trubygems/traveling-ruby)
+Install OCRAN on Windows and run it on your entry script:
+
+    gem install ocran
+    ocran app.rb
+
+The result is `app.exe`, which contains your script, the Ruby interpreter,
+the gems it loaded and the DLLs they need. On Linux and macOS the same
+command produces a native executable `./app`.
+
+### Does the target machine need Ruby installed?
+
+No. The executable carries its own Ruby interpreter, standard library,
+gems and native libraries.
+
+### Can I build a Windows .exe on Linux or macOS?
+
+Not with the default mode: OCRAN bundles the Ruby of the machine it runs
+on, so build a Windows `.exe` on Windows (for example on a GitHub Actions
+Windows runner, see
+[Building for multiple platforms](#building-for-multiple-platforms-with-github-actions)),
+or run a Windows Ruby and OCRAN under Wine on Linux/macOS.
+Two experimental modes package without the target's own Ruby:
+`--static-runtime` with a Windows runtime builds a Windows `.exe` on
+Linux (and the other way round), and `--cosmo-ruby` packages a
+cosmopolitan Ruby into one file that runs on Windows, Linux and macOS.
+
+### What is the difference between OCRA and OCRAN?
+
+OCRAN is a maintained fork of OCRA. OCRA targets Windows only and is no
+longer updated for current Ruby versions; OCRAN supports Ruby 3.3+, builds
+for Windows, Linux and macOS, and adds directory, zip, macOS `.app` and
+Inno Setup output. To migrate, call `ocran` instead of `ocra` and replace
+`OCRA_EXECUTABLE` with `OCRAN_EXECUTABLE` in your code.
+
+### Which gems and native extensions are supported?
+
+OCRAN runs your script once at build time and records every file that
+`require` and `load` pull in, including compiled extensions and the DLLs
+or shared libraries they link. In the default mode, gems with native extensions
+(for example `sqlite3`, `pg`, `nokogiri`) are packaged along with their
+libraries; the experimental `--cosmo-ruby` mode only supports the native
+extensions built into its interpreter. Use
+`--gemfile` to include everything from a Bundler `Gemfile`, and the
+[gem detection modes](#gem-content-detection-modes) to include extra files.
+
+### Can I package a Rails, GUI or command-line app?
+
+Yes. Command-line tools work out of the box, GUI apps can use
+`--windows` to hide the console on Windows, and Rails apps with SQLite are
+covered in [Packaging a Rails application](#packaging-a-rails-application).
+
+### Why does my packaged app start slowly?
+
+The default self-extracting executable unpacks Ruby to a temporary
+directory on every launch. Use `--output-dir`, `--output-zip` or
+`--innosetup` to install the files once, `--static-runtime` for a single
+executable that runs without extracting, or `--cosmo-ruby` with an
+interpreter that supports ZIP packaging (CosmoRuby `v4.0.6-cosmo3` or
+newer) to run without extracting; with other interpreters `--cosmo-ruby`
+falls back to a launcher stub that extracts on every start.
+
+### My antivirus flags the generated .exe. What can I do?
+
+Self-extracting executables that unpack an interpreter are a common
+heuristic trigger. Sign the executable (OCRAN output is code-signing
+compatible, e.g. with `signtool` on Windows), ship an
+installer built with `--innosetup`, or ship a plain directory with
+`--output-dir`.
+
+## Comparison with other Ruby packagers
+
+| Tool | Output | Platforms | Single file | Maintained for current Ruby |
+|------|--------|-----------|-------------|--------------------------|
+| **OCRAN** | `.exe` / native binary, directory, zip, macOS `.app`, Inno Setup installer | Windows, Linux, macOS | Yes | Yes |
+| [OCRA](https://github.com/larsch/ocra) | `.exe` | Windows only | Yes | No |
+| [Traveling Ruby](https://github.com/trubygems/traveling-ruby) | Prebuilt Ruby runtime you ship next to your app | Windows, Linux, macOS | No | Yes |
+| [Tebako](https://github.com/tamatebako/tebako) | Single binary with an embedded filesystem | Windows, Linux, macOS | Yes | Yes |
+| [ruby-packer](https://github.com/pmq20/ruby-packer) | Single binary with Ruby compiled from source | Windows, Linux, macOS | Yes | No |
+
+OCRAN's distinguishing features are that, in its default mode, it needs no
+C compiler to package an app (it reuses your installed Ruby), detects dependencies by running
+your script, and can produce installers and macOS app bundles directly.
+
+See also: [State of Ruby Packagers](https://gist.github.com/YOU54F/3775e66e6090e0371c11601e6b75c305).
+
+## Security, dependencies and reproducibility
+
+A packager decides which interpreter, libraries and gems end up on your
+users' machines, so where those come from matters as much as the
+packaging itself. OCRAN is built so that nothing reaches the executable
+that you did not install, run and test yourself, and so that you can
+check what went in.
+
+**No hidden downloads.** In its default mode OCRAN makes no network
+requests: it does not fetch an interpreter, prebuilt runtime, patched
+Ruby source or library tarball while packaging. The executables it
+produces, in every mode, never download a runtime when they start. Some
+packaging approaches ship a small launcher that pulls the actual runtime
+from the internet on first launch; the app then depends on a server
+being reachable and trustworthy long after you released it. An OCRAN
+executable carries the interpreter, gems and libraries it runs; whether
+your app uses the network is up to your own code. The opt-in modes that
+do fetch say so and verify what they get:
+
+* `--wasm` downloads the ruby.wasm or PicoRuby package from the npm
+  registry once, checks it against the SHA-512 integrity the registry
+  publishes, and caches it, so later builds run offline.
+* `--static-runtime` fetches the source gem of a precompiled platform gem
+  from your configured gem sources when it has to build that gem's
+  extension for the runtime.
+
+**Your Ruby, from the source you already trust.** By default OCRAN
+packages the Ruby interpreter installed on the build machine —
+RubyInstaller, a distribution package, `ruby-build`, `ruby/setup-ruby` in
+CI, or your own build. It does not substitute a separately built or
+patched interpreter from a third party. Security fixes reach your app the
+same way they reach your development machine: update Ruby, rebuild.
+Packagers that embed their own copy of Ruby and its C libraries (OpenSSL,
+zlib, libyaml, libffi, ...) freeze those versions at whatever the
+packager last shipped, and fall behind on security fixes when the
+packager stops being updated. The `--static-runtime` runtimes are built
+by scripts in this repository (`runtime/`) that you can run yourself;
+OCRAN checks a runtime against the SHA-256 recorded in its
+`runtime.json`.
+
+**Exactly the gems and libraries you tested.** Dependencies are not
+resolved again at packaging time. OCRAN runs your script, records the
+files Ruby actually loaded, and packs those files from your installed
+gems — the versions pinned in your `Gemfile.lock` when you build under
+Bundler. Native libraries are taken from the same installation your
+extensions were loaded from. There is no second dependency resolution
+that could pick a different version than the one you ran.
+
+**Reproducible builds.** The bundled interpreter and dependencies are
+determined by three things you control: the Ruby installation (or static
+runtime), the installed gems (or `Gemfile.lock`) and the OCRAN version;
+the rest of the artifact follows from your application source and the
+OCRAN options you pass. Pin them — e.g. a fixed `ruby-version` in
+`ruby/setup-ruby` plus a committed `Gemfile.lock` — and every build
+packages the same interpreter, gems and libraries. OCRAN's own stubs are
+built reproducibly (no timestamps in the PE header), the Windows, Linux
+and macOS static runtimes are reproducible, and `--static-runtime`
+executables are byte-reproducible on all three systems; CI packages a
+test application twice and requires the same bytes.
+
+**Inspect what went in.** Build with `--output-dir` and look at the
+directory, or run an executable with `--debug-extract`. A
+`--static-runtime` executable is a plain ZIP archive behind the runtime:
+`unzip -l app.exe` lists exactly what will run, and
+`ocran/manifest.sha256` inside it records the SHA-256 of the runtime and
+of every file.
+
+**Verifiable OCRAN releases.** The launcher stubs (`stub.exe`,
+`stubw.exe` and the Linux/macOS `stub`) are about 4,000 lines of C in
+`src/`, compiled from this repository by GitHub Actions. Released gems
+and the CI-built static runtimes carry signed build provenance
+attestations (`gh attestation verify <file> -R Largo/ocran`); from 1.5.1
+on, each GitHub release has the three runtimes attached next to the gems,
+and gems are pushed to rubygems.org with Sigstore attestations. The only
+third-party binary is `lzma.exe`, taken unmodified from the official
+7-Zip release (see [Safety](#safety)).
+
+The trade-off: keeping the bundled Ruby patched is your job, just as it
+is for the Ruby you develop with. The experimental `--cosmo-ruby` mode
+packages a cosmopolitan interpreter file you choose, such as a CosmoRuby
+release, as given.
 
 ## Credits
 

data/lib/ocran/build_helper.rb

@@ -30,9 +30,16 @@ module Ocran
 
     def export(name, value)
       verbose "export #{name}=#{replace_placeholder(value)}"
+      exported_env[name.to_s] = value.to_s
       super
     end
 
+    # The variables exported so far (name => value with the extraction root
+    # placeholder), whatever the builder does with them.
+    def exported_env
+      @exported_env ||= {}
+    end
+
     def replace_placeholder(s)
       s.to_s.gsub(EXTRACT_ROOT.to_s, "<tempdir>")
     end

data/lib/ocran/direction.rb

@@ -6,6 +6,7 @@ require_relative "refine_pathname"
 require_relative "host_config_helper"
 require_relative "command_output"
 require_relative "build_constants"
+require_relative "env_hook"
 
 module Ocran
   class Direction
@@ -396,19 +397,30 @@ module Ocran
       # Convert all relative paths to absolute paths before building.
       # NOTE: In the future, different strategies may be needed before and after script execution.
       features.filter_map do |feature|
-        if feature.absolute?
-          feature
-        elsif (load_path = @post_env.find_load_path(feature))
-          feature.expand_path(@post_env.expand_path(load_path))
-        else
-          # This message occurs when paths for core library files (e.g., enumerator.so,
-          # rational.so, complex.so, fiber.so, thread.rb, ruby2_keywords.rb) are not
-          # found. These are integral to Ruby's standard libraries or extensions and
-          # may not be located via normal load path searches, especially in RubyInstaller
-          # environments.
-          verbose "Load path not found for #{feature}, skip this feature"
-          nil
+        path =
+          if feature.absolute?
+            feature
+          elsif (load_path = @post_env.find_load_path(feature))
+            feature.expand_path(@post_env.expand_path(load_path))
+          else
+            # This message occurs when paths for core library files (e.g., enumerator.so,
+            # rational.so, complex.so, fiber.so, thread.rb, ruby2_keywords.rb) are not
+            # found. These are integral to Ruby's standard libraries or extensions and
+            # may not be located via normal load path searches, especially in RubyInstaller
+            # environments.
+            verbose "Load path not found for #{feature}, skip this feature"
+            next
+          end
+        # An entry can be in $LOADED_FEATURES without a file behind it: a
+        # racc parser generated with its runtime embedded marks
+        # <dir>/racc/parser.rb and racc/info.rb as loaded so that racc is not
+        # loaded twice (rdoc's RD parsers in Ruby 3.4 do). Copying it failed
+        # the whole build with ENOENT; there is nothing to pack.
+        unless path.file?
+          verbose "#{path} is marked as loaded but is not a file, skip this feature"
+          next
         end
+        path
       end
     end
 
@@ -471,6 +483,19 @@ module Ocran
         end
       end
 
+      # String#unicode_normalize (and unicode_normalized?, unicode_normalize!)
+      # is core, but core requires its implementation from the standard
+      # library (unicode_normalize/normalize.rb, which loads tables.rb) on
+      # first use. An app that does not call it during the dependency run
+      # fails with a LoadError once packed, so always pack these files.
+      # Not with --cosmo-ruby: the payload has its own standard library.
+      unless @option.cosmo_ruby
+        known = features.to_set { |f| f.path_key }
+        (Pathname(RbConfig::CONFIG["rubylibdir"]) / "unicode_normalize").glob("*.rb").sort.each do |path|
+          features << path unless known.include?(path.path_key)
+        end
+      end
+
       say "Building #{@option.output_executable}"
       require_relative "build_helper"
       builder.extend(BuildHelper)
@@ -955,14 +980,23 @@ module Ocran
       elsif @option.add_all_core?
         say "Will include all ruby core libraries"
         all_core_dir.each do |path|
+          # A configured directory that was never created (Debian's
+          # sitelibdir, /usr/local/lib/site_ruby/<ver>) holds nothing.
+          next unless path.directory?
+
           # Match the load path against standard library, site_ruby, and vendor_ruby paths
           unless (subdir = path.to_posix.match(RUBY_LIBRARY_PATH_REGEX)&.[](1))
             raise "Unexpected library path format (does not match core dirs): #{path}"
           end
-          path.find.each do |src|
-            next if src.directory?
+          files, skipped = self.class.core_library_files(path)
+          files.each do |src|
             builder.copy_to_lib(src, Pathname(subdir) / src.relative_path_from(path))
           end
+          next if skipped.empty?
+
+          warning "Not packed: #{skipped.size} entr#{skipped.size == 1 ? "y" : "ies"} of #{path} that " \
+                  "#{skipped.size == 1 ? "is" : "are"} not a readable file (#{skipped.first(3).join(", ")}" \
+                  "#{skipped.size > 3 ? ", ..." : ""})"
         end
       end
 
@@ -1215,6 +1249,8 @@ module Ocran
       end
       builder.set_env_path("GEM_PATH", *gem_paths)
 
+      pack_env_hook(builder)
+
       # Add the opcode to launch the script
       installed_ruby_exe = BINDIR / ruby_executable
       target_script = builder.resolve_source_path(@option.script, inst_src_prefix)
@@ -1224,6 +1260,58 @@ module Ocran
       builder.exec(installed_ruby_exe, target_script, *@option.argv)
     end
 
+    # Packs the RubyGems startup hook that sets the exported variables for a
+    # bare bin/ruby of the package (see EnvHook), e.g. a Windows service
+    # registered as `bin\ruby.exe src\service.rb`. It goes into the packed
+    # sitelibdir, the first default load path entry, in every output that
+    # carries the host Ruby: the stub and the launch scripts set the
+    # variables themselves, so there it only loads the original file.
+    # Not with --cosmo-ruby and --static-runtime, which bring their own
+    # Ruby and set up their environment themselves.
+    def pack_env_hook(builder)
+      return if @option.cosmo_ruby || @option.static_runtime?
+
+      site = Pathname(sitelibdir)
+      unless site.subpath?(exec_prefix)
+        verbose "Not packing #{EnvHook::FEATURE}: sitelibdir #{site} is outside #{exec_prefix}"
+        return
+      end
+      if (site / EnvHook::FEATURE).exist?
+        warning "Not packing ocran's #{EnvHook::FEATURE}: #{site} has one already, so a bare " \
+                "#{BINDIR / ruby_executable} of the package will not get the launch script's environment"
+        return
+      end
+
+      target = site.relative_path_from(exec_prefix) / EnvHook::FEATURE
+      say "Adding #{target} (sets the launch environment for a bare #{BINDIR / ruby_executable})"
+      require "tempfile"
+      hook_file = Tempfile.new(["ocran-operating_system", ".rb"])
+      hook_file.write(EnvHook.source(builder.exported_env, target.dirname.each_filename.count))
+      hook_file.close
+      verbose File.read(hook_file.path)
+      # Kept until the build finishes, like the RUBYOPT launcher.
+      @env_hook_file = hook_file
+      builder.cp(hook_file.path, target)
+    end
+    private :pack_env_hook
+
+    # The files below +dir+ that --add-all-core packs, and the entries it
+    # cannot: a Ruby installation can list what is not there - Debian's
+    # rdoc links its fonts into a fonts-lato package that may not be
+    # installed, and a stdlib entry that vanished or cannot be read used to
+    # abort the whole build (ENOENT) over a file no application needs.
+    # Returns [files, skipped], both lists of Pathname.
+    def self.core_library_files(dir)
+      files = []
+      skipped = []
+      dir.find do |src|
+        next if src.directory?
+
+        (src.file? && src.readable? ? files : skipped) << src
+      end
+      [files, skipped]
+    end
+
     # Maps an absolute build-machine path to its location (relative to the
     # extraction root) inside the packed application, mirroring where
     # #construct places files. Returns nil when the path is not packed.

data/lib/ocran/env_hook.rb

@@ -0,0 +1,98 @@
+# frozen_string_literal: true
+require_relative "build_constants"
+
+module Ocran
+  # The RubyGems startup hook that gives a bare interpreter of a packed
+  # application the environment of its launch script.
+  #
+  # The launch script (app.bat / app.sh), the wrapper executable and the
+  # self-extracting stub all set the variables Direction#construct exports
+  # (GEM_HOME, GEM_PATH, RUBYLIB, SSL_CERT_FILE, ...) before they start
+  # bin/ruby. Some hosts can only start the interpreter itself: the Windows
+  # service control manager accepts the service dispatcher only from the
+  # process it started (error 1063), so a service registered as
+  # `bin\ruby.exe src\service.rb` runs without any of them - gems packed
+  # outside the Ruby installation are not found, and the CA bundle ocran
+  # packed is only used where OpenSSL happens to look on its own.
+  #
+  # RubyGems requires "rubygems/defaults/operating_system" while it loads,
+  # before it computes its paths and before anything of the application
+  # runs; distributions use that file to customise RubyGems (RubyInstaller
+  # sets up its DLL search paths in it). ocran packs its own file of that
+  # name in the packed sitelibdir, which comes first on the default load
+  # path. It sets each variable the launch script would set and that is
+  # not set yet, resolved against the directory the package is in at run
+  # time, and then requires the next operating_system.rb on the load path
+  # - the distribution's own - so that keeps working as before.
+  #
+  # Under the launch script or the stub every variable is already set, so
+  # the hook only passes on to the original file there.
+  module EnvHook
+    include BuildConstants
+
+    # The feature RubyGems requires at startup.
+    FEATURE = "rubygems/defaults/operating_system.rb"
+
+    # Builds the hook's source for the variables in +env+ (name => value as
+    # exported at build time, with the extraction root placeholder) for a
+    # hook packed +depth+ directories below the package root (the depth of
+    # the packed sitelibdir plus the two of FEATURE's own directories).
+    def self.source(env, depth)
+      root_rel = ([".."] * depth).join("/")
+      entries = env.map { |name, value|
+        "  #{name.to_s.dump} => #{value.to_s.dump},"
+      }
+
+      <<~RUBY
+        # frozen_string_literal: true
+        # Generated by OCRAN. Gives a bare bin/ruby of this package the
+        # environment of the launch script: each variable below that is not
+        # set yet is set, with "#{EXTRACT_ROOT}/" standing for the package root,
+        # and each one the launch script empties is removed, as it does,
+        # and then the next #{FEATURE} on the load path (the one the Ruby
+        # installation ships, if any) is loaded, as RubyGems would have done.
+        # RUBYOPT, LD_LIBRARY_PATH and DYLD_LIBRARY_PATH take effect for child
+        # processes only: they are read before this file runs.
+        ocran_env = {
+        #{entries.join("\n")}
+        }
+        ocran_root = File.expand_path(#{root_rel.dump}, __dir__)
+        ocran_windows = File::ALT_SEPARATOR == "\\\\"
+        ocran_same = lambda do |entry, dir|
+          entry = File.expand_path(entry)
+          ocran_windows ? entry.casecmp?(dir) : entry == dir
+        end
+        ocran_feature = lambda { |dir| File.join(File.expand_path(dir), #{FEATURE.dump}) }
+        ocran_this = lambda { |file| File.file?(file) && File.identical?(file, __FILE__) }
+        # RUBYLIB goes where Ruby would have put it: just before the
+        # directory this file was found in, the first default entry.
+        ocran_index = $LOAD_PATH.index { |dir| ocran_this.call(ocran_feature.call(dir)) } || 0
+        ocran_gem_paths = false
+        ocran_env.each do |name, value|
+          # Removed by the launch script, whatever the parent set: an
+          # inherited BUNDLE_GEMFILE must not replace the packaged Gemfile.
+          if value.empty?
+            ENV.delete(name)
+            next
+          end
+          next if ENV.key?(name)
+
+          value = value.gsub(%r{#{Regexp.escape(EXTRACT_ROOT.to_s)}[/\\\\]}) { "\#{ocran_root}/" }
+          value = value.tr("/", "\\\\") if ocran_windows
+          ENV[name] = value
+          case name
+          when "GEM_HOME", "GEM_PATH"
+            ocran_gem_paths = true
+          when "RUBYLIB"
+            dirs = value.split(File::PATH_SEPARATOR).reject(&:empty?).map { |dir| File.expand_path(dir) }
+            dirs.reject! { |dir| $LOAD_PATH.any? { |entry| ocran_same.call(entry, dir) } }
+            $LOAD_PATH.insert(ocran_index, *dirs.uniq)
+          end
+        end
+        Gem.clear_paths if ocran_gem_paths && defined?(Gem.clear_paths)
+        ocran_original = $LOAD_PATH.map(&ocran_feature).find { |file| File.file?(file) && !ocran_this.call(file) }
+        require ocran_original if ocran_original
+      RUBY
+    end
+  end
+end

data/lib/ocran/gem_spec_queryable.rb

@@ -90,8 +90,12 @@ module Ocran
             # Skip if found in known Gem dir
           elsif (spec = GemSpecQueryable.find_spec(feature))
             gems << spec
-          else
-            puts "Failed to load gemspec for #{feature}" if verbose
+          elsif verbose && Gem.path.any? { |gem_path| feature.subpath?(File.join(gem_path, "gems")) }
+            # Only a feature in a gem directory is expected to have a
+            # gemspec. Standard library and application files are not gems,
+            # and reporting each of them buried real errors under hundreds
+            # of lines.
+            puts "Failed to load gemspec for #{feature}"
           end
 
           gems

data/lib/ocran/option.rb

@@ -886,5 +886,45 @@ EOF
       return false unless Gem.win_platform?
       @options[:windowed?]
     end
+
+    # Where the compiled launcher stubs live (StubBuilder::STUB_PATH and
+    # STUBW_PATH). A released gem ships them; a git checkout, such as
+    # Bundler's `gem "ocran", git: ...`, has only their sources in src/.
+    STUB_DIR = File.expand_path("../../share/ocran", __dir__)
+
+    # The prebuilt stub the build will copy, or nil when it copies none:
+    # native compilation, --wasm, --static-runtime and --cosmo-ruby's ZIP
+    # mode use no stub, --cosmo compiles its own, and installer, directory
+    # and zip output only need one for the wrapper executable.
+    def prebuilt_stub
+      return nil if spinel? || roundhouse? || wasm_runtime || static_runtime? || cosmo_zip? || cosmo_cc
+      return nil if (inno_setup_script || output_dir || output_zip) && !wrapper_exe?
+
+      name = if !Gem.win_platform? then "stub"
+             elsif windowed? && !macosx_bundle then "stubw.exe"
+             else "stub.exe"
+             end
+      File.join(STUB_DIR, name)
+    end
+
+    # Fails when the build needs a stub that was never compiled. Checked
+    # before the dependency run, which can take minutes, since the build
+    # would otherwise end in a bare ENOENT for the stub after it.
+    def check_prebuilt_stub(path = prebuilt_stub)
+      return if path.nil? || File.exist?(path)
+
+      root = File.dirname(STUB_DIR, 2)
+      build = if RUBY_PLATFORM.include?("mswin")
+                "`nmake -f Makefile.mswin install` in #{File.join(root, "src")} from a Visual Studio developer prompt"
+              elsif Gem.win_platform?
+                "`ridk exec make -C src install` in #{root}"
+              else
+                "`make -C src install` in #{root}"
+              end
+      message = "The launcher stub #{path} has not been built (OCRAN from a git checkout " \
+                "ships only its sources): run #{build}"
+      message += ", or pass --no-wrapper-exe to skip the wrapper executable" if inno_setup_script || output_dir || output_zip
+      raise message
+    end
   end
 end

data/lib/ocran/runner.rb

@@ -29,6 +29,7 @@ module Ocran
       load File.expand_path("option.rb", __dir__)
       @option = Option.new.tap do |opt|
         opt.parse(ARGV)
+        opt.check_prebuilt_stub
       rescue *USER_ERRORS => e
         # Invalid option arguments, usually; report them as such.
         user_error e
@@ -207,6 +208,14 @@ module Ocran
             # environment, so it is preferable to ignore exceptions other than
             # critical errors.
             warning "#{mod}::#{const} loading failed: #{e.message}"
+          rescue SystemExit => e
+            # Some libraries exit when an optional dependency is missing -
+            # rdoc/servlet.rb aborts without webrick, which Ruby 3.4 no
+            # longer ships. That ended the whole build (exit 1) over a
+            # constant the application never asked for; the abort message
+            # went to stderr ahead of the buffered progress output.
+            warning "#{mod}::#{const} loading exited (status #{e.status})" \
+                    "#{e.message == "exit" ? "" : ": #{e.message}"}"
           end
         end
         checked_modules[mod] = true

data/lib/ocran/runtime_sdk.rb

@@ -188,6 +188,10 @@ module Ocran
         cmd = cmd.map { |a| as_dll.fetch(a, a) }
         cmd += ["-o", output.to_s] unless args.any? { |a| a.include?("@OUTPUT@") }
         cmd += ["-Wl,-sectcreate,__OCRAN,__zip,#{archive}"] if archive
+        # macOS: no debug map (the objects' paths, here in a temporary
+        # directory, would differ every time), and nothing for strip to do,
+        # which would break the signature the linker writes.
+        cmd += ["-Wl,-S"] if @info["target_os"].to_s.start_with?("darwin")
         run(*cmd)
       end
       strip = strip_tool
@@ -242,8 +246,11 @@ module Ocran
     # Compilers, linkers and strip run with SOURCE_DATE_EPOCH set: binutils
     # then writes that, not the current time, into a PE header it rewrites
     # (strip does), so that the same application links to the same bytes.
+    # ZERO_AR_DATE does the same for Apple's ld, which otherwise writes the
+    # objects' modification times into the executable's debug map.
     def run(*cmd)
-      out, status = Open3.capture2e({ "SOURCE_DATE_EPOCH" => ENV["SOURCE_DATE_EPOCH"] || "1767225600" }, *cmd)
+      env = { "SOURCE_DATE_EPOCH" => ENV["SOURCE_DATE_EPOCH"] || "1767225600", "ZERO_AR_DATE" => "1" }
+      out, status = Open3.capture2e(env, *cmd)
       raise "#{cmd.first} failed:\n#{out.lines.last(25).join}" unless status.success?
 
       out

data/lib/ocran/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Ocran
-  VERSION = "1.5.0"
+  VERSION = "1.5.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ocran
 version: !ruby/object:Gem::Version
-  version: 1.5.0
+  version: 1.5.1
 platform: ruby
 authors:
 - Andi Idogawa
@@ -59,6 +59,7 @@ files:
 - lib/ocran/ed_resource.rb
 - lib/ocran/ed_resource_builder.rb
 - lib/ocran/empty_source
+- lib/ocran/env_hook.rb
 - lib/ocran/file_path_set.rb
 - lib/ocran/gem_spec_queryable.rb
 - lib/ocran/host_config_helper.rb
@@ -120,6 +121,8 @@ metadata:
   homepage_uri: https://github.com/largo/ocran
   source_code_uri: https://github.com/largo/ocran
   changelog_uri: https://github.com/largo/ocran/blob/master/CHANGELOG.txt
+  bug_tracker_uri: https://github.com/largo/ocran/issues
+  documentation_uri: https://github.com/largo/ocran#readme
 rdoc_options: []
 require_paths:
 - lib