rake-compiler-dock 0.7.2 → 1.2.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +4 -0
  3. data/.github/workflows/ci.yml +168 -0
  4. data/.gitignore +14 -0
  5. data/Dockerfile.jruby +12 -7
  6. data/Dockerfile.mri.erb +271 -0
  7. data/History.md +63 -0
  8. data/README.md +138 -27
  9. data/Rakefile +71 -4
  10. data/{lib/rake_compiler_dock → build}/gem_helper.rb +0 -15
  11. data/build/parallel_docker_build.rb +106 -0
  12. data/build/patches/{ruby-2.5.3 → ruby-2.5.9}/no_sendfile.patch +1 -1
  13. data/build/patches/ruby-3.1.0/no_sendfile.patch +24 -0
  14. data/build/patches2/rake-compiler-1.1.6/0004-Enable-build-of-static-libruby.patch +28 -0
  15. data/build/rcd-env.sh +6 -0
  16. data/build/runas +1 -6
  17. data/build/sudoers +1 -1
  18. data/lib/rake_compiler_dock/docker_check.rb +12 -6
  19. data/lib/rake_compiler_dock/starter.rb +69 -53
  20. data/lib/rake_compiler_dock/version.rb +2 -2
  21. data/lib/rake_compiler_dock.rb +10 -0
  22. data/mingw64-ucrt/Dockerfile +62 -0
  23. data/mingw64-ucrt/binutils-mingw-w64-ignore-check-errors.patch +13 -0
  24. data/mingw64-ucrt/gcc-mingw-w64-only-c-c++.patch +13 -0
  25. data/mingw64-ucrt/mingw-w64-enable-ucrt.patch +22 -0
  26. data/rake-compiler-dock.gemspec +2 -2
  27. data/test/env/Dockerfile.alpine +17 -0
  28. data/test/env/Dockerfile.centos +17 -0
  29. data/test/env/Dockerfile.debian +24 -0
  30. data/test/rcd_test/Gemfile +11 -0
  31. data/test/rcd_test/Rakefile +74 -0
  32. data/test/rcd_test/ext/java/RcdTestExtService.java +19 -0
  33. data/test/rcd_test/ext/java/RubyRcdTest.java +16 -0
  34. data/test/rcd_test/ext/mri/extconf.rb +20 -0
  35. data/test/rcd_test/ext/mri/rcd_test_ext.c +35 -0
  36. data/test/rcd_test/ext/mri/rcd_test_ext.h +6 -0
  37. data/test/rcd_test/lib/rcd_test.rb +6 -0
  38. data/test/rcd_test/rcd_test.gemspec +27 -0
  39. data/test/rcd_test/test/test_basic.rb +22 -0
  40. data/test/test_environment_variables.rb +56 -31
  41. data/test/test_parallel_docker_build.rb +70 -0
  42. data.tar.gz.sig +1 -0
  43. metadata +83 -14
  44. metadata.gz.sig +0 -0
  45. data/Dockerfile.mri +0 -122
  46. data/build/patches/rake-compiler-1.0.7/enable-static.diff +0 -13
data/README.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # rake-compiler-dock
2
2
 
3
- **Easy to use and reliable cross compiler environment for building Windows, Linux and JRuby binary gems.**
3
+ **Easy to use and reliable cross compiler environment for building Windows, Linux, Mac and JRuby binary gems.**
4
4
 
5
- It provides cross compilers and Ruby environments for 2.2 and newer versions of the [RubyInstaller](http://rubyinstaller.org/) and Linux runtime environments.
5
+ It provides cross compilers and Ruby environments for 2.4 and newer versions of the [RubyInstaller](http://rubyinstaller.org/) and Linux runtime environments.
6
6
  They are prepared for use with [rake-compiler](https://github.com/rake-compiler/rake-compiler).
7
7
  It is used by [many gems with C or JRuby extentions](https://github.com/rake-compiler/rake-compiler-dock/wiki/Projects-using-rake-compiler-dock).
8
8
 
@@ -12,11 +12,7 @@ It is also more reliable, since the underlying docker images are versioned and i
12
12
 
13
13
  ## Installation
14
14
 
15
- Install docker natively on Linux:
16
-
17
- $ sudo apt-get install docker.io
18
-
19
- ... or install [docker-toolbox for Windows and OSX](https://www.docker.com/docker-toolbox) or boot2docker on [Windows](https://github.com/boot2docker/windows-installer/releases) or [OS X](https://github.com/boot2docker/osx-installer/releases) .
15
+ Install docker [following the instructions on the docker website](https://docs.docker.com/engine/install/) ... or install [docker-toolbox for Windows and OSX](https://github.com/docker/toolbox/releases) or boot2docker on [Windows](https://github.com/boot2docker/windows-installer/releases) or [OS X](https://github.com/boot2docker/osx-installer/releases) .
20
16
 
21
17
  Install rake-compiler-dock as a gem. The docker image is downloaded later on demand:
22
18
 
@@ -34,16 +30,23 @@ Rake-compiler-dock provides the necessary tools to build Ruby extensions for Win
34
30
  It is intended to be used in conjunction with [rake-compiler's](https://github.com/rake-compiler/rake-compiler) cross build capability.
35
31
  Your Rakefile should enable cross compilation like so:
36
32
 
37
- Rake::ExtensionTask.new('my_extension', my_gem_spec) do |ext|
38
- ext.cross_compile = true
39
- ext.cross_platform = %w[x86-mingw32 x64-mingw32 x86-linux x86_64-linux]
40
- end
33
+ ```ruby
34
+ exttask = Rake::ExtensionTask.new('my_extension', my_gem_spec) do |ext|
35
+ ext.cross_compile = true
36
+ ext.cross_platform = %w[x86-mingw32 x64-mingw-ucrt x64-mingw32 x86-linux x86_64-linux x86_64-darwin arm64-darwin]
37
+ end
38
+ ```
39
+
40
+ See below, how to invoke cross builds in your Rakefile.
41
41
 
42
- It may also be used to build ffi based binary gems like [libusb](https://github.com/larskanis/libusb), but currently doesn't provide any additional build helpers for this use case, beyond docker invocation and cross compilers.
42
+ Additionally it may also be used to build ffi based binary gems like [libusb](https://github.com/larskanis/libusb), but currently doesn't provide any additional build helpers for this use case, beyond docker invocation and cross compilers.
43
43
 
44
44
  ### Interactive Usage
45
45
 
46
46
  Rake-compiler-dock offers the shell command `rake-compiler-dock` and a [ruby API](http://www.rubydoc.info/gems/rake-compiler-dock/RakeCompilerDock) for issuing commands within the docker image, described below.
47
+ There are dedicated images for `x86-mingw32`, `x64-mingw-ucrt`, `x64-mingw32`, `x86-linux`, `x86_64-linux`, `x86_64-darwin`, `arm64-darwin` and `jruby` targets.
48
+ The images contain all supported cross ruby versions, with the exception of `x64-mingw32`, which has versions before 3.1 only, and `x64-mingw-ucrt`, which has only ruby-3.1+.
49
+ This is to match the [changed platform of RubyInstaller-3.1](https://rubyinstaller.org/2021/12/31/rubyinstaller-3.1.0-1-released.html).
47
50
 
48
51
  `rake-compiler-dock` without arguments starts an interactive shell session.
49
52
  This is best suited to try out and debug a build.
@@ -55,16 +58,22 @@ But note, that all other changes to the file system of the container are dropped
55
58
  All commands are executed with the same user and group of the host.
56
59
  This is done by copying user account data into the container and sudo to it.
57
60
 
58
- To build x86 and x64 Windows and Linux binary gems interactively, it can be called like this:
61
+ To build x86 Windows and x86_64 Linux binary gems interactively, it can be called like this:
59
62
 
60
63
  user@host:$ cd your-gem-dir/
61
- user@host:$ rake-compiler-dock # this enters a container with an interactive shell
64
+ user@host:$ rake-compiler-dock # this enters a container with an interactive shell for x86 Windows (default)
62
65
  user@5b53794ada92:$ bundle
63
66
  user@5b53794ada92:$ rake cross native gem
64
67
  user@5b53794ada92:$ exit
65
68
  user@host:$ ls pkg/*.gem
66
- your-gem-1.0.0.gem your-gem-1.0.0-x64-mingw32.gem your-gem-1.0.0-x86-mingw32.gem
67
- your-gem-1.0.0-x86_64-linux.gem your-gem-1.0.0-x86-linux.gem
69
+ your-gem-1.0.0.gem your-gem-1.0.0-x86-mingw32.gem
70
+
71
+ user@host:$ RCD_PLATFORM=x86_64-linux rake-compiler-dock # this enters a container for amd64 Linux target
72
+ user@adc55b2b92a9:$ bundle
73
+ user@adc55b2b92a9:$ rake cross native gem
74
+ user@adc55b2b92a9:$ exit
75
+ user@host:$ ls pkg/*.gem
76
+ your-gem-1.0.0.gem your-gem-1.0.0-x86_64-linux.gem
68
77
 
69
78
  Or non-interactive:
70
79
 
@@ -72,13 +81,64 @@ Or non-interactive:
72
81
 
73
82
  The environment variable `RUBY_CC_VERSION` is predefined as described [below](#environment-variables).
74
83
 
75
- If necessary, additional software from the Ubuntu repositories can be installed, prior to the build command.
76
- This is local to the running session, only:
84
+ If necessary, additional software can be installed, prior to the build command.
85
+ This is local to the running session, only.
86
+
87
+ For Windows and Mac:
77
88
 
78
89
  sudo apt-get update && sudo apt-get install your-package
79
90
 
91
+ For Linux:
92
+
93
+ sudo yum install your-package
94
+
80
95
  You can also choose between different executable ruby versions by `rvm use <version>` .
81
- The current default is 2.5.
96
+ The current default is 3.1.
97
+
98
+
99
+ ### As a CI System Container
100
+
101
+ The OCI images provided by rake-compiler-dock can be used without the `rake-compiler-dock` gem or wrapper. This may be useful if your CI pipeline is building native gems.
102
+
103
+ For example, a Github Actions job might look like this:
104
+
105
+ ``` yaml
106
+ jobs:
107
+ native-gem:
108
+ name: "native-gem"
109
+ runs-on: ubuntu-latest
110
+ container:
111
+ image: "larskanis/rake-compiler-dock-mri-x86_64-linux:1.2.0"
112
+ steps:
113
+ - uses: actions/checkout@v2
114
+ - run: bundle install && bundle exec rake gem:x86_64-linux:rcd
115
+ - uses: actions/upload-artifact@v2
116
+ with:
117
+ name: native-gem
118
+ path: gems
119
+ retention-days: 1
120
+ ```
121
+
122
+ Where the referenced rake task might be defined by:
123
+
124
+ ``` ruby
125
+ cross_platforms = ["x64-mingw32", "x86_64-linux", "x86_64-darwin", "arm64-darwin"]
126
+
127
+ namespace "gem" do
128
+ cross_platforms.each do |platform|
129
+ namespace platform do
130
+ task "rcd" do
131
+ Rake::Task["native:#{platform}"].invoke
132
+ Rake::Task["pkg/#{rcee_precompiled_spec.full_name}-#{Gem::Platform.new(platform)}.gem"].invoke
133
+ end
134
+ end
135
+ end
136
+ end
137
+
138
+ ```
139
+
140
+ For an example of rake tasks that support this style of invocation, visit https://github.com/flavorjones/ruby-c-extensions-explained/tree/main/precompiled
141
+
82
142
 
83
143
  ### JRuby support
84
144
 
@@ -102,16 +162,61 @@ To build java binary gems interactively, it can be called like this:
102
162
  To make the build process reproducible for other parties, it is recommended to add rake-compiler-dock to your Rakefile.
103
163
  This can be done like this:
104
164
 
105
- task 'gem:native' do
165
+ ```ruby
166
+ task 'gem:native' do
167
+ require 'rake_compiler_dock'
168
+ sh "bundle package --all" # Avoid repeated downloads of gems by using gem files from the host.
169
+ %w[ x86-mingw32 x64-mingw-ucrt x64-mingw32 x86-linux x86_64-linux arm-linux aarch64-linux x86_64-darwin arm64-darwin ].each do |plat|
170
+ RakeCompilerDock.sh "bundle --local && rake native:#{plat} gem", platform: plat
171
+ end
172
+ RakeCompilerDock.sh "bundle --local && rake java gem", rubyvm: :jruby
173
+ end
174
+ ```
175
+
176
+ This runs the `bundle` and `rake` commands once for each platform.
177
+ That is once for the jruby gems and 6 times for the specified MRI platforms.
178
+
179
+ ### Run builds in parallel
180
+
181
+ rake-compiler-dock uses dedicated docker images per build target (since rake-compiler-dock-1.0).
182
+ Because each target runs in a separate docker container, it is simple to run all targets in parallel.
183
+ The following example defines `rake gem:native` as a multitask and separates the preparation which should run only once.
184
+ It also shows how gem signing can be done with parallel builds.
185
+ Please note, that parallel builds only work reliable, if the specific platform gem is requested (instead of just "rake gem").
186
+
187
+ ```ruby
188
+ namespace "gem" do
189
+ task 'prepare' do
106
190
  require 'rake_compiler_dock'
107
- sh "bundle package" # Avoid repeated downloads of gems by using gem files from the host.
108
- RakeCompilerDock.sh "bundle --local && rake cross native gem"
109
- RakeCompilerDock.sh "bundle --local && rake java gem", rubyvm: :jruby
191
+ require 'io/console'
192
+ sh "bundle package --all"
193
+ sh "cp ~/.gem/gem-*.pem build/gem/ || true"
194
+ ENV["GEM_PRIVATE_KEY_PASSPHRASE"] = STDIN.getpass("Enter passphrase of gem signature key: ")
195
+ end
196
+
197
+ exttask.cross_platform.each do |plat|
198
+ desc "Build all native binary gems in parallel"
199
+ multitask 'native' => plat
200
+
201
+ desc "Build the native gem for #{plat}"
202
+ task plat => 'prepare' do
203
+ RakeCompilerDock.sh <<-EOT, platform: plat
204
+ (cp build/gem/gem-*.pem ~/.gem/ || true) &&
205
+ bundle --local &&
206
+ rake native:#{plat} pkg/#{exttask.gem_spec.full_name}-#{plat}.gem
207
+ EOT
208
+ end
110
209
  end
210
+ end
211
+ ```
212
+
213
+ ### Add to your Gemfile
111
214
 
112
215
  Rake-compiler-dock uses [semantic versioning](http://semver.org/), so you should add it into your Gemfile, to make sure, that future changes will not break your build.
113
216
 
114
- gem 'rake-compiler-dock', '~> 0.7.0'
217
+ ```ruby
218
+ gem 'rake-compiler-dock', '~> 1.2'
219
+ ```
115
220
 
116
221
  See [the wiki](https://github.com/rake-compiler/rake-compiler-dock/wiki/Projects-using-rake-compiler-dock) for projects which make use of rake-compiler-dock.
117
222
 
@@ -122,14 +227,18 @@ Rake-compiler-dock makes use of several environment variables.
122
227
 
123
228
  The following variables are recognized by rake-compiler-dock:
124
229
 
125
- * `RCD_RUBYVM` - The flavour of docker image to be used.
230
+ * `RCD_RUBYVM` - The ruby VM and toolchain to be used.
126
231
  Must be one of `mri`, `jruby`.
232
+ * `RCD_PLATFORM` - The target rubygems platform.
233
+ Must be a space separated list out of `x86-mingw32`, `x64-mingw-ucrt`, `x64-mingw32`, `x86-linux`, `x86_64-linux`, `arm-linux`, `aarch64-linux`, `x86_64-darwin` and `arm64-darwin`.
234
+ It is ignored when `rubyvm` is set to `:jruby`.
127
235
  * `RCD_IMAGE` - The docker image that is downloaded and started.
128
236
  Defaults to "larskanis/rake-compiler-dock:IMAGE_VERSION" with an image version that is determined by the gem version.
129
237
 
130
238
  The following variables are passed through to the docker container without modification:
131
239
 
132
240
  * `http_proxy`, `https_proxy`, `ftp_proxy` - See [Frequently asked questions](https://github.com/rake-compiler/rake-compiler-dock/wiki/FAQ) for more details.
241
+ * `GEM_PRIVATE_KEY_PASSPHRASE` - To avoid interactive password prompts in the container.
133
242
 
134
243
  The following variables are provided to the running docker container:
135
244
 
@@ -137,7 +246,7 @@ The following variables are provided to the running docker container:
137
246
  * `RCD_HOST_RUBY_PLATFORM` - The `RUBY_PLATFORM` of the host ruby.
138
247
  * `RCD_HOST_RUBY_VERSION` - The `RUBY_VERSION` of the host ruby.
139
248
  * `RUBY_CC_VERSION` - The target ruby versions for rake-compiler.
140
- The default is defined in the [Dockerfile](https://github.com/rake-compiler/rake-compiler-dock/blob/master/Dockerfile), but can be changed as a parameter to rake.
249
+ The default is defined in the [Dockerfile](https://github.com/rake-compiler/rake-compiler-dock/blob/94770238d68d71df5f70abe76097451a575ce46c/Dockerfile.mri.erb#L229), but can be changed as a parameter to rake.
141
250
  * `RCD_MOUNTDIR` - The directory which is mounted into the docker container.
142
251
  Defaults to pwd.
143
252
  * `RCD_WORKDIR` - The working directory within the docker container.
@@ -145,7 +254,9 @@ The following variables are provided to the running docker container:
145
254
 
146
255
  Other environment variables can be set or passed through to the container like this:
147
256
 
148
- RakeCompilerDock.sh "rake cross native gem OPENSSL_VERSION=#{ENV['OPENSSL_VERSION']}"
257
+ ```ruby
258
+ RakeCompilerDock.sh "rake cross native gem OPENSSL_VERSION=#{ENV['OPENSSL_VERSION']}"
259
+ ```
149
260
 
150
261
 
151
262
  ## More information
data/Rakefile CHANGED
@@ -1,11 +1,65 @@
1
+ require 'erb'
2
+ require "rake/clean"
1
3
  require "rake_compiler_dock"
2
- require "rake_compiler_dock/gem_helper"
4
+ require_relative "build/gem_helper"
5
+ require_relative "build/parallel_docker_build"
3
6
 
4
7
  RakeCompilerDock::GemHelper.install_tasks
5
8
 
6
- task :build do
7
- sh "docker build -t larskanis/rake-compiler-dock-mri:#{RakeCompilerDock::IMAGE_VERSION} -f Dockerfile.mri ."
8
- sh "docker build -t larskanis/rake-compiler-dock-jruby:#{RakeCompilerDock::IMAGE_VERSION} -f Dockerfile.jruby ."
9
+ DOCKERHUB_USER = ENV['DOCKERHUB_USER'] || "larskanis"
10
+ docker_build_cmd = Shellwords.split(ENV['RCD_DOCKER_BUILD'] || "docker build")
11
+
12
+ platforms = [
13
+ ["x86-mingw32", "i686-w64-mingw32"],
14
+ ["x64-mingw32", "x86_64-w64-mingw32"],
15
+ ["x64-mingw-ucrt", "x86_64-w64-mingw32"],
16
+ ["x86-linux", "i686-redhat-linux"],
17
+ ["x86_64-linux", "x86_64-redhat-linux"],
18
+ ["x86_64-darwin", "x86_64-apple-darwin"],
19
+ ["arm64-darwin", "aarch64-apple-darwin"],
20
+ ["arm-linux", "arm-linux-gnueabihf"],
21
+ ["aarch64-linux", "aarch64-linux-gnu"],
22
+ ]
23
+
24
+ namespace :build do
25
+
26
+ platforms.each do |platform, target|
27
+ sdf = "Dockerfile.mri.#{platform}"
28
+
29
+ desc "Build image for platform #{platform}"
30
+ task platform => sdf
31
+ task sdf do
32
+ sh(*docker_build_cmd, "-t", "#{DOCKERHUB_USER}/rake-compiler-dock-mri-#{platform}:#{RakeCompilerDock::IMAGE_VERSION}", "-f", "Dockerfile.mri.#{platform}", ".")
33
+ end
34
+
35
+ df = ERB.new(File.read("Dockerfile.mri.erb"), trim_mode: ">").result(binding)
36
+ File.write(sdf, df)
37
+ CLEAN.include(sdf)
38
+ end
39
+
40
+ desc "Build image for JRuby"
41
+ task :jruby => "Dockerfile.jruby"
42
+ task "Dockerfile.jruby" do
43
+ sh(*docker_build_cmd, "-t", "#{DOCKERHUB_USER}/rake-compiler-dock-jruby:#{RakeCompilerDock::IMAGE_VERSION}", "-f", "Dockerfile.jruby", ".")
44
+ end
45
+
46
+ RakeCompilerDock::ParallelDockerBuild.new(platforms.map{|pl, _| "Dockerfile.mri.#{pl}" } + ["Dockerfile.jruby"], workdir: "tmp/docker", docker_build_cmd: docker_build_cmd)
47
+
48
+ desc "Build images for all MRI platforms in parallel"
49
+ multitask :mri => platforms.map(&:first)
50
+
51
+ desc "Build images for all platforms in parallel"
52
+ multitask :all => platforms.map(&:first) + ["jruby"]
53
+ end
54
+
55
+ task :build => "build:all"
56
+
57
+ namespace :prepare do
58
+ desc "Build cross compiler for x64-mingw-ucrt aka RubyInstaller-3.1+"
59
+ task "mingw64-ucrt" do
60
+ sh(*docker_build_cmd, "-t", "#{DOCKERHUB_USER}/mingw64-ucrt:20.04", ".",
61
+ chdir: "mingw64-ucrt")
62
+ end
9
63
  end
10
64
 
11
65
  desc "Run tests"
@@ -34,3 +88,16 @@ task :update_lists do
34
88
  EOT
35
89
  end
36
90
  end
91
+
92
+ namespace :release do
93
+ desc "push all docker images"
94
+ task :images do
95
+ jimg = "#{DOCKERHUB_USER}/rake-compiler-dock-jruby:#{RakeCompilerDock::IMAGE_VERSION}"
96
+ sh "docker", "push", jimg
97
+
98
+ platforms.each do |platform, _|
99
+ img = "#{DOCKERHUB_USER}/rake-compiler-dock-mri-#{platform}:#{RakeCompilerDock::IMAGE_VERSION}"
100
+ sh "docker", "push", img
101
+ end
102
+ end
103
+ end
@@ -1,18 +1,3 @@
1
- # This file is part of Libusb for Ruby.
2
- #
3
- # Libusb for Ruby is free software: you can redistribute it and/or modify
4
- # it under the terms of the GNU Lesser General Public License as published by
5
- # the Free Software Foundation, either version 3 of the License, or
6
- # (at your option) any later version.
7
- #
8
- # Libusb for Ruby is distributed in the hope that it will be useful,
9
- # but WITHOUT ANY WARRANTY; without even the implied warranty of
10
- # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11
- # GNU Lesser General Public License for more details.
12
- #
13
- # You should have received a copy of the GNU Lesser General Public License
14
- # along with Libusb for Ruby. If not, see <http://www.gnu.org/licenses/>.
15
-
16
1
  require "bundler/gem_helper"
17
2
 
18
3
  module RakeCompilerDock
@@ -0,0 +1,106 @@
1
+ require "fileutils"
2
+ require "rake"
3
+
4
+ module RakeCompilerDock
5
+ # Run docker builds in parallel, but ensure that common docker layers are reused
6
+ class ParallelDockerBuild
7
+ include Rake::DSL
8
+
9
+ attr_reader :docker_build_cmd
10
+
11
+ def initialize(dockerfiles, workdir: "tmp/docker", inputdir: ".", task_prefix: "common-",
12
+ docker_build_cmd: %w["docker", "build"])
13
+ @docker_build_cmd = docker_build_cmd
14
+ FileUtils.mkdir_p(workdir)
15
+
16
+ files = parse_dockerfiles(dockerfiles, inputdir)
17
+ # pp files
18
+
19
+ vcs = find_commons(files)
20
+ # pp vcs
21
+
22
+ define_common_tasks(vcs, workdir, task_prefix)
23
+ end
24
+
25
+ # Read given dockerfiles from inputdir and split into a list of commands.
26
+ #
27
+ # Returns:
28
+ # {"File0"=>[" FROM a\n", " RUN a\n", " RUN d\n"],
29
+ # "File1"=>[" FROM a\n",
30
+ # ...
31
+ def parse_dockerfiles(dockerfiles, inputdir)
32
+ dockerfiles.map do |fn|
33
+ [fn, File.read(File.join(inputdir, fn))]
34
+ end.map do |fn, f|
35
+ # Split file contant in lines unless line ends with backslash
36
+ fc = f.each_line.with_object([]) do |line, lines|
37
+ if lines.last=~/\\\n\z/
38
+ lines.last << line
39
+ else
40
+ lines << line
41
+ end
42
+ end
43
+ [fn, fc]
44
+ end.to_h
45
+ end
46
+
47
+ # Build a tree of common parts of given files.
48
+ #
49
+ # Returns:
50
+ # {["File0", "File1", "File2", "File3"]=>
51
+ # [[" FROM a\n"],
52
+ # {["File0", "File1"]=>
53
+ # [[" RUN a\n", " RUN d\n"], {["File1"]=>[[" RUN f\n"], {}]}],
54
+ # ["File2", "File3"]=>[[" RUN b\n", " RUN c\n", " RUN d\n"], {}]}]}
55
+ def find_commons(files, vmask=nil, li=0)
56
+ vmask ||= files.keys
57
+ vcs = Hash.new { [] }
58
+ files.each do |fn, lines|
59
+ next unless vmask.include?(fn)
60
+ vcs[lines[li]] += [fn]
61
+ end
62
+
63
+ vcs.map do |line, vc|
64
+ next unless line
65
+ nvcs = find_commons(files, vc, li+1)
66
+ if nvcs.first && nvcs.first[0] == vc
67
+ # Append lines that are equal between file(s)
68
+ nl = [[line] + nvcs.first[1][0], nvcs.first[1][1]]
69
+ else
70
+ nl = [[line], nvcs]
71
+ end
72
+ [vc, nl]
73
+ end.compact.to_h
74
+ end
75
+
76
+ # Write intermediate dockerfiles to workdir and define rake tasks
77
+ #
78
+ # The rake tasks are named after the dockerfiles given to #new .
79
+ # This also adds dependant intermediate tasks as prerequisites.
80
+ def define_common_tasks(vcs, workdir, task_prefix, plines=[])
81
+ vcs.map do |files, (lines, nvcs)|
82
+ fn = "#{task_prefix}#{files.join}"
83
+ File.write(File.join(workdir, fn), (plines + lines).join)
84
+ task fn do
85
+ docker_build(fn, workdir)
86
+ end
87
+
88
+ nfn = define_common_tasks(nvcs, workdir, task_prefix, plines + lines)
89
+ nfn.each do |file|
90
+ task file => fn
91
+ end
92
+ files.each do |file|
93
+ task file => fn
94
+ end
95
+ fn
96
+ end
97
+ end
98
+
99
+ # Run an intermediate dockerfile without tag
100
+ #
101
+ # The layers will be reused in subsequent builds, even if they run in parallel.
102
+ def docker_build(filename, workdir)
103
+ sh(*docker_build_cmd, "-f", File.join(workdir, filename), ".")
104
+ end
105
+ end
106
+ end
@@ -17,7 +17,7 @@ index 82c5940538..a8d3661ec8 100644
17
17
  @@ -10721,7 +10721,6 @@ nogvl_copy_stream_wait_write(struct copy_stream_struct *stp)
18
18
  }
19
19
 
20
- #if defined __linux__ && defined __NR_copy_file_range
20
+ #if defined HAVE_COPY_FILE_RANGE || (defined __linux__ && defined __NR_copy_file_range)
21
21
  -# define USE_COPY_FILE_RANGE
22
22
  #endif
23
23
 
@@ -0,0 +1,24 @@
1
+ diff --git a/configure b/configure
2
+ index ebe3d8c..a336b73 100755
3
+ --- a/configure
4
+ +++ b/configure
5
+ @@ -18943,7 +18943,6 @@ do :
6
+ ac_fn_c_check_func "$LINENO" "sendfile" "ac_cv_func_sendfile"
7
+ if test "x$ac_cv_func_sendfile" = xyes; then :
8
+ cat >>confdefs.h <<_ACEOF
9
+ -#define HAVE_SENDFILE 1
10
+ _ACEOF
11
+
12
+ fi
13
+ diff --git a/io.c b/io.c
14
+ index 48592ac51a..2635ac8ec6 100644
15
+ --- a/io.c
16
+ +++ b/io.c
17
+ @@ -11111,6 +11111,7 @@ nogvl_copy_stream_wait_write(struct copy_stream_struct *stp)
18
+ return 0;
19
+ }
20
+
21
+ +#undef USE_COPY_FILE_RANGE
22
+ #ifdef USE_COPY_FILE_RANGE
23
+
24
+ static ssize_t
@@ -0,0 +1,28 @@
1
+ diff --git a/tasks/bin/cross-ruby.rake b/tasks/bin/cross-ruby.rake
2
+ index 8317a2a..5a1b3ea 100644
3
+ --- a/tasks/bin/cross-ruby.rake
4
+ +++ b/tasks/bin/cross-ruby.rake
5
+ @@ -116,10 +116,22 @@ RUBY_CC_VERSIONS.split(":").each do |ruby_cc_version|
6
+ "--host=#{mingw_host}",
7
+ "--target=#{mingw_target}",
8
+ "--build=#{RUBY_BUILD}",
9
+ - '--enable-shared',
10
+ + '--enable-install-static-library',
11
+ + '--disable-jit-support',
12
+ '--disable-install-doc',
13
+ '--with-ext=',
14
+ ]
15
+ + if mingw_host=~/darwin/
16
+ + options += [
17
+ + '--enable-static',
18
+ + '--disable-shared',
19
+ + ]
20
+ + else
21
+ + options += [
22
+ + '--enable-static',
23
+ + '--enable-shared',
24
+ + ]
25
+ + end
26
+
27
+ # Force Winsock2 for Ruby 1.8, 1.9 defaults to it
28
+ options << "--with-winsock2" if major == "1.8"
data/build/rcd-env.sh ADDED
@@ -0,0 +1,6 @@
1
+ # set up a working RCD build environment
2
+ export RAKE_EXTENSION_TASK_NO_NATIVE=true
3
+ if ! test -e "$HOME"/.rake-compiler ; then
4
+ ln -s /usr/local/rake-compiler "$HOME"/.rake-compiler
5
+ fi
6
+ mkdir -p "$HOME"/.gem
data/build/runas CHANGED
@@ -4,9 +4,4 @@ groupadd -o -g "$GID" "$GROUP"
4
4
  mkdir -p /tmp/home
5
5
  useradd -o -g "$GID" -u "$UID" -G rvm,sudo -p "" -b /tmp/home -m "$USER"
6
6
 
7
- HOME=$(bash <<< "echo ~$USER")
8
- ln -s /usr/local/rake-compiler "$HOME"/.rake-compiler
9
-
10
- sudo -u "$USER" --set-home \
11
- BASH_ENV=/etc/rubybashrc \
12
- -- "$@"
7
+ /usr/bin/sudo -u "$USER" -H BASH_ENV=/etc/rubybashrc -- "$@"
data/build/sudoers CHANGED
@@ -1 +1 @@
1
- Defaults env_keep += "http_proxy https_proxy ftp_proxy RCD_HOST_RUBY_PLATFORM RCD_HOST_RUBY_VERSION RCD_IMAGE RUBY_CC_VERSION"
1
+ Defaults env_keep += "http_proxy https_proxy ftp_proxy GEM_PRIVATE_KEY_PASSPHRASE RCD_HOST_RUBY_PLATFORM RCD_HOST_RUBY_VERSION RCD_IMAGE RUBY_CC_VERSION LD_LIBRARY_PATH DEVTOOLSET_ROOTPATH"
@@ -8,6 +8,7 @@ module RakeCompilerDock
8
8
 
9
9
  attr_reader :io
10
10
  attr_reader :pwd
11
+ attr_reader :docker_command
11
12
  attr_accessor :machine_name
12
13
 
13
14
  def initialize(io, pwd, machine_name="rake-compiler-dock")
@@ -53,12 +54,17 @@ module RakeCompilerDock
53
54
  end
54
55
  end
55
56
 
57
+ COMMANDS = %w[docker podman]
56
58
  def docker_version
57
- @docker_version_text, @docker_version_status = run("docker version")
59
+ COMMANDS.find do |command|
60
+ @docker_version_text, @docker_version_status = run("#{command} version")
61
+ @docker_command = command
62
+ @docker_version_status == 0
63
+ end
58
64
  end
59
65
 
60
66
  def ok?
61
- @docker_version_status == 0 && @docker_version_text =~ /version/ && doma_pwd_ok?
67
+ @docker_version_status == 0 && @docker_version_text =~ /version/i && doma_pwd_ok?
62
68
  end
63
69
 
64
70
  def docker_client_avail?
@@ -216,16 +222,16 @@ module RakeCompilerDock
216
222
  help << yellow("Please download and install the docker-toolbox:")
217
223
  help << yellow(" https://www.docker.com/docker-toolbox")
218
224
  when /linux/
219
- help << red("Docker is not available.")
225
+ help << red("Neither Docker nor Podman is available.")
220
226
  help << ""
221
- help << yellow("Install on Ubuntu/Debian:")
227
+ help << yellow("Install Docker on Ubuntu/Debian:")
222
228
  help << " sudo apt-get install docker.io"
223
229
  help << ""
224
- help << yellow("Install on Fedora/Centos/RHEL")
230
+ help << yellow("Install Docker on Fedora/Centos/RHEL")
225
231
  help << " sudo yum install docker"
226
232
  help << " sudo systemctl start docker"
227
233
  help << ""
228
- help << yellow("Install on SuSE")
234
+ help << yellow("Install Docker on SuSE")
229
235
  help << " sudo zypper install docker"
230
236
  help << " sudo systemctl start docker"
231
237
  when /darwin/