cibuildgem 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.md +21 -0
- data/README.md +78 -0
- data/exe/cibuildgem +7 -0
- data/lib/cibuildgem/cli.rb +152 -0
- data/lib/cibuildgem/compilation_tasks.rb +122 -0
- data/lib/cibuildgem/create_makefile_finder.rb +17 -0
- data/lib/cibuildgem/errors.rb +5 -0
- data/lib/cibuildgem/ruby_series.rb +56 -0
- data/lib/cibuildgem/tasks/wrapper.rake +29 -0
- data/lib/cibuildgem/templates/github/workflows/cibuildgem.yaml.tt +101 -0
- data/lib/cibuildgem/version.rb +5 -0
- data/lib/cibuildgem.rb +10 -0
- metadata +104 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: d1109ef83a9f4e37eb633c5b816db075acbaad093f96d760c7f7be2218142dc4
|
|
4
|
+
data.tar.gz: 2c1ba9f62ca2278a029550ace8fbbf5cf599e3ffc0de5b9c6066fee967086843
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 6f730003c2304c4f419b6e5e21e2aa0e5fe129398d5dc80cefe459d70c050a236e14ec1866da845573c0359a9c84b2e0f8c35f8e0751f7fa23dccc1a42131c90
|
|
7
|
+
data.tar.gz: d46afc9e5cc500a1a578933e12ad381373f708736193c20e7eda19317bac3a4687cedec88831db14905e1f19018cc9d946c4b5f6b6124abcf52d3bc51a5a694e
|
data/LICENSE.md
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Shopify
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
> [!NOTE]
|
|
2
|
+
> **This tool is currently in active development.** We are very much looking for your feedback.
|
|
3
|
+
|
|
4
|
+
## ๐๏ธ Preamble
|
|
5
|
+
|
|
6
|
+
#### The problem this tool tries to solve.
|
|
7
|
+
|
|
8
|
+
A major bottleneck for every Ruby developers when running `bundle install` is the compilation of native gem extensions. To illustrate this issue, running `bundle install` on a new Rails application takes around **25 seconds** on a MacBook Pro M4, and 80% of that time is spent compiling the couple dozens of gems having native extensions. It would take only 5 seconds if it wasn't for the compilation.
|
|
9
|
+
|
|
10
|
+
#### How we can solve this problem for the Ruby community.
|
|
11
|
+
|
|
12
|
+
The Python community had exactly the same issue and came up with the amazing [cibuildwheel](https://github.com/pypa/cibuildwheel) solution to provide a CI based compilation approach to help maintainers ship their libraries will precompiled binaries for various platforms.
|
|
13
|
+
|
|
14
|
+
This tool modestly tries to follow the same approach by helping ruby maintainers ship their gems with precompiled binaries.
|
|
15
|
+
|
|
16
|
+
#### Existing solutions.
|
|
17
|
+
|
|
18
|
+
Precompilation isn't new in the Ruby ecosystem and some maintainers have been releasing their gems with precompiled binaries to speedup the installation process since a while (e.g. [nokogiri](https://rubygems.org/gems/nokogiri), [grpc](https://rubygems.org/gems/grpc), [karafka-rdkafka](https://rubygems.org/gems/karafka-rdkafka)). One of the most popular tool that enables to precompile binaries for different platform is the great [rake-compiler-dock](https://github.com/rake-compiler/rake-compiler-dock) toolchain.
|
|
19
|
+
It uses a cross compilation approach by periodically building docker images for various platforms and spinning up containers to compile the binaries.
|
|
20
|
+
|
|
21
|
+
As noted by [@flavorjones](https://github.com/flavorjones), this toolchain works great but it's complex and brittle compared to the more simple process of compiling on the target platform.
|
|
22
|
+
|
|
23
|
+
## ๐ป cibuildgem
|
|
24
|
+
|
|
25
|
+
> [!NOTE]
|
|
26
|
+
> cibuildgem is for now not able to compile projects that needs to link on external libraries. Unless the project vendors those libraries or uses [mini_portile](https://github.com/flavorjones/mini_portile).
|
|
27
|
+
|
|
28
|
+
### How to use it
|
|
29
|
+
|
|
30
|
+
While cibuildgem is generally **not** meant to be used locally, it provides a command to generate the right GitHub workflow for your project:
|
|
31
|
+
|
|
32
|
+
1. Install cibuildgem: `gem install cibuildgem`
|
|
33
|
+
2. Generate the workflow: `cd` in your gem's folder and run `cibuildgem ci_template`
|
|
34
|
+
3. Commit the `.github/workflows/cibuildgem.yaml` file.
|
|
35
|
+
|
|
36
|
+
### Triggering the workflow
|
|
37
|
+
|
|
38
|
+
Once pushed in your repository **default** branch, the workflow that we just generated is actionable manually on the GitHub action page. It will run in sequence:
|
|
39
|
+
|
|
40
|
+
1. Compile the gem on the target platform (defaults to MacOS ARM, MacOS Intel, Windows, Ubuntu 24)
|
|
41
|
+
2. Once the compilation succeeds on all platform, it proceeds to run the test suite on the target platform. This will trigger many CI steps as the testing matrix is big.
|
|
42
|
+
3. Once the test suite passes for all platforms and all Ruby versions the gem is compatible with, the action proceeds to installing the gem we just packaged. This step ensure that the gem is actually installable.
|
|
43
|
+
4. [OPTIONAL] When trigering the workflow manually, you can tick the box to automatically release the gems that were packaged. This works using the RubyGems trusted publisher feature (documentation to write later). If you do no want the tool to make the release, you can download all the GitHub artifacts that were uploaded. It will contain all the gems with precompiled binaries in the `pkg` folder. You are free to download them locally and release them yourself from your machine.
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
### Changes to make in your gem to support precompiled binaries
|
|
47
|
+
|
|
48
|
+
Due to the RubyGems specification, we can't release a gem with precompiled binaries for a specific Ruby version. Because the Ruby ABI is incompatible between minor versions, Rake Compiler (the tool underneath cibuildgem), compiles the binary for every minor Ruby versions your gem supports. All those binaries will be packaged in the gem (called a fat gem) in different folder such as `3.0/date.so`, `3.1/date.so` etc...
|
|
49
|
+
At runtime, your gem need to require the right binary based on the running ruby version.
|
|
50
|
+
|
|
51
|
+
```ruby
|
|
52
|
+
# Before
|
|
53
|
+
|
|
54
|
+
require 'date_core.so'
|
|
55
|
+
|
|
56
|
+
# After
|
|
57
|
+
|
|
58
|
+
begin
|
|
59
|
+
ruby_version = /(\d+\.\d+)/.match(::RUBY_VERSION)
|
|
60
|
+
require "#{ruby_version}/date_core"
|
|
61
|
+
rescue LoadError
|
|
62
|
+
# It's important to leave for users that can not or don't want to use the gem with precompiled binaries.
|
|
63
|
+
require "date_core"
|
|
64
|
+
end
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Supported platforms/Ruby versions
|
|
68
|
+
|
|
69
|
+
| | MacOS Intel | MacOS ARM | Windows x64 UCRT | Linux GNU x86_64|Linux AARCH64 |
|
|
70
|
+
|---------|------------- | --------- | ------------|-----------------|-----------------|
|
|
71
|
+
| Ruby 3.1| ๐ข | ๐ข | ๐ข | ๐ข | ๐ข |
|
|
72
|
+
| Ruby 3.2| ๐ข | ๐ข | ๐ข | ๐ข | ๐ข |
|
|
73
|
+
| Ruby 3.3| ๐ข | ๐ข | ๐ข | ๐ข | ๐ข |
|
|
74
|
+
| Ruby 3.4| ๐ข | ๐ข | ๐ข | ๐ข | ๐ข |
|
|
75
|
+
|
|
76
|
+
## ๐งช Development
|
|
77
|
+
|
|
78
|
+
If you'd like to run a end-to-end test, the `date` gem is vendored in this project. You can trigger a manual run to do the whole compile, test, install dance from the GitHub action menu.
|
data/exe/cibuildgem
ADDED
|
@@ -0,0 +1,152 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "thor"
|
|
4
|
+
require "rake/extensiontask"
|
|
5
|
+
|
|
6
|
+
module Cibuildgem
|
|
7
|
+
class CLI < Thor
|
|
8
|
+
include Thor::Actions
|
|
9
|
+
|
|
10
|
+
source_root(File.expand_path("templates", __dir__))
|
|
11
|
+
|
|
12
|
+
class << self
|
|
13
|
+
def exit_on_failure?
|
|
14
|
+
true
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
desc "compile", "Compile a gem's native extension."
|
|
19
|
+
long_desc <<~MSG
|
|
20
|
+
This command will read a gem's gemspec file and setup a Rake Compiler task to be executed.
|
|
21
|
+
You need to run this command at the root of the project.
|
|
22
|
+
|
|
23
|
+
It's not required for a gem to define a `Rake::ExtensionTask`. However, if such task exists,
|
|
24
|
+
it will be executed as part of the compilation process.
|
|
25
|
+
|
|
26
|
+
The Rakefile of the project will be loaded and you may enhance or add other prequisite tasks
|
|
27
|
+
to the compilation.
|
|
28
|
+
MSG
|
|
29
|
+
def compile
|
|
30
|
+
run_rake_tasks!("cibuildgem:setup", :compile)
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
desc "package", "Compile and package a 'fat gem'.", hide: true
|
|
34
|
+
long_desc <<~MSG
|
|
35
|
+
This command should normally run on CI, using the cibuildgem workflow. It will not work locally unless
|
|
36
|
+
the environment is properly setup.
|
|
37
|
+
|
|
38
|
+
Based on a gem's gemspec, create a tailored-made Rake Compiler task to create two gems:
|
|
39
|
+
- A gem with precompiled binary compatible on the platform running the command.
|
|
40
|
+
- A gem without precompiled binary (Ruby platform).
|
|
41
|
+
|
|
42
|
+
The gem with precompiled binaries will be packaged with multiple binaries compatible for different
|
|
43
|
+
Ruby ABI (depending on what Ruby version the gem supports).
|
|
44
|
+
MSG
|
|
45
|
+
method_option "gemspec", type: "string", required: false, desc: "The gemspec to use. Defaults to the gemspec from the current working directory."
|
|
46
|
+
def package
|
|
47
|
+
ENV["RUBY_CC_VERSION"] ||= compilation_task.ruby_cc_version
|
|
48
|
+
|
|
49
|
+
run_rake_tasks!("cibuildgem:setup", :cross, :native, :gem)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
desc "test", "Run the test suites of the target gem"
|
|
53
|
+
long_desc <<~EOM
|
|
54
|
+
cibuildgem will run the test suite of the gem. It either expects a `spec` or `test` task defined.
|
|
55
|
+
EOM
|
|
56
|
+
def test
|
|
57
|
+
run_rake_tasks!(:test)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
desc "copy_from_staging_to_lib", "Copy the staging binary. For internal usage.", hide: true
|
|
61
|
+
def copy_from_staging_to_lib
|
|
62
|
+
run_rake_tasks!("cibuildgem:setup", "copy:stage:lib")
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
desc "clean", "Cleanup temporary compilation artifacts."
|
|
66
|
+
long_desc <<~MSG
|
|
67
|
+
Cleanup temporary artifacts used for compiling the gem's extension.
|
|
68
|
+
|
|
69
|
+
When this command is invoked, the gem's Rakefile will be loaded and you can customize which artifacts
|
|
70
|
+
to cleanup by adding files to the vanilla CLEAN rake list.
|
|
71
|
+
MSG
|
|
72
|
+
def clean
|
|
73
|
+
run_rake_tasks!("cibuildgem:setup", :clean)
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
desc "clobber", "Remove compiled binaries."
|
|
77
|
+
long_desc <<~MSG
|
|
78
|
+
Remove compiled binaries.
|
|
79
|
+
|
|
80
|
+
When this command is invoked, the gem's Rakefile will be loaded and you can customize the list of files
|
|
81
|
+
to remove by adding files to the vanilla CLOBBER rake list.
|
|
82
|
+
MSG
|
|
83
|
+
def clobber
|
|
84
|
+
run_rake_tasks!("cibuildgem:setup", :clobber)
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
desc "ci_template", "Generate CI template files."
|
|
88
|
+
long_desc <<~MSG
|
|
89
|
+
Generate a GitHub workflow to perform all the steps needed for compiling a gem's extension and packaging its binaries.
|
|
90
|
+
|
|
91
|
+
This command needs to run at the root of your project and expects to see a `.gemspec` file. It will read the gemspec
|
|
92
|
+
and determine what Ruby versions needs to be used for precompiling a "fat gem".
|
|
93
|
+
MSG
|
|
94
|
+
method_option "working-directory", type: "string", required: false, desc: "If your gem lives outside of the repository root, specify where."
|
|
95
|
+
method_option "test-command", type: "string", required: false, desc: "The test command to run. Defaults to running `bundle exec rake test` and `bundle exec rake spec`."
|
|
96
|
+
def ci_template
|
|
97
|
+
ruby_requirements = compilation_task.gemspec.required_ruby_version
|
|
98
|
+
# os = ["macos-latest", "macos-15-intel", "ubuntu-latest", "windows-latest"]
|
|
99
|
+
@os = ["macos-latest", "ubuntu-22.04"] # Just this for now because the CI takes too long otherwise.
|
|
100
|
+
@latest_supported_ruby_version = RubySeries.latest_version_for_requirements(ruby_requirements)
|
|
101
|
+
@runtime_version_for_compilation = RubySeries.runtime_version_for_compilation(ruby_requirements)
|
|
102
|
+
@ruby_versions_for_testing = RubySeries.versions_to_test_against(ruby_requirements)
|
|
103
|
+
|
|
104
|
+
directory("github", ".github", context: instance_eval("binding"))
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
desc "release", "Release the gem with precompiled binaries. For internal usage.", hide: true
|
|
108
|
+
method_option "glob", type: :string, required: true, desc: "Release all the gems matching the glob"
|
|
109
|
+
def release
|
|
110
|
+
Dir.glob(options[:glob]).each do |file|
|
|
111
|
+
pathname = Pathname(file)
|
|
112
|
+
next if pathname.directory? || pathname.extname != ".gem"
|
|
113
|
+
|
|
114
|
+
Kernel.system("gem push #{file}", exception: true)
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
desc "print_ruby_cc_version", "Output the cross compile ruby version needed for the gem. For internal usage", hide: true
|
|
119
|
+
method_option "gemspec", type: "string", required: false, desc: "The gemspec to use. If the option is not passed, a gemspec file from the current working directory will be used."
|
|
120
|
+
def print_ruby_cc_version
|
|
121
|
+
print(compilation_task.ruby_cc_version)
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
desc "normalized_platform", "The platform name for compilation purposes", hide: true
|
|
125
|
+
def print_normalized_platform
|
|
126
|
+
print(compilation_task.normalized_platform)
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
private
|
|
130
|
+
|
|
131
|
+
def run_rake_tasks!(*tasks)
|
|
132
|
+
all_tasks = tasks.join(" ")
|
|
133
|
+
rakelibdir = [File.expand_path("tasks", __dir__), "rakelib"].join(File::PATH_SEPARATOR)
|
|
134
|
+
rake_compiler_path = Gem.loaded_specs["rake-compiler"].full_require_paths
|
|
135
|
+
rake_specs = Gem.loaded_specs["rake"]
|
|
136
|
+
rake_executable = rake_specs.bin_file("rake")
|
|
137
|
+
rake_path = rake_specs.full_require_paths
|
|
138
|
+
prism_path = Gem.loaded_specs["prism"].full_require_paths
|
|
139
|
+
load_paths = (rake_compiler_path + rake_path + prism_path).join(File::PATH_SEPARATOR)
|
|
140
|
+
|
|
141
|
+
system({ "RUBYLIB" => load_paths }, "bundle exec #{RbConfig.ruby} #{rake_executable} #{all_tasks} -R#{rakelibdir}", exception: true)
|
|
142
|
+
end
|
|
143
|
+
|
|
144
|
+
def compilation_task
|
|
145
|
+
@compilation_task ||= CompilationTasks.new(false)
|
|
146
|
+
rescue GemspecError => e
|
|
147
|
+
print(e.message)
|
|
148
|
+
|
|
149
|
+
Kernel.exit(false)
|
|
150
|
+
end
|
|
151
|
+
end
|
|
152
|
+
end
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "bundler"
|
|
4
|
+
require "rubygems/package_task"
|
|
5
|
+
require "rake/extensiontask"
|
|
6
|
+
require_relative "create_makefile_finder"
|
|
7
|
+
|
|
8
|
+
module Cibuildgem
|
|
9
|
+
class CompilationTasks
|
|
10
|
+
attr_reader :gemspec, :native, :create_packaging_task, :extension_task
|
|
11
|
+
|
|
12
|
+
def initialize(create_packaging_task = false, gemspec = nil)
|
|
13
|
+
@gemspec = Bundler.load_gemspec(gemspec || find_gemspec)
|
|
14
|
+
verify_gemspec!
|
|
15
|
+
|
|
16
|
+
@create_packaging_task = create_packaging_task
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def setup
|
|
20
|
+
gemspec.extensions.each do |path|
|
|
21
|
+
binary_name = parse_extconf(path)
|
|
22
|
+
define_task(path, binary_name)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
setup_packaging if create_packaging_task
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def ruby_cc_version
|
|
29
|
+
required_ruby_version = @gemspec.required_ruby_version
|
|
30
|
+
selected_rubies = RubySeries.versions_to_compile_against(required_ruby_version)
|
|
31
|
+
|
|
32
|
+
selected_rubies.map(&:to_s).join(":")
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def normalized_platform
|
|
36
|
+
platform = RUBY_PLATFORM
|
|
37
|
+
|
|
38
|
+
if darwin?
|
|
39
|
+
RUBY_PLATFORM.sub(/(.*-darwin)\d+/, '\1')
|
|
40
|
+
else
|
|
41
|
+
platform
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
private
|
|
46
|
+
|
|
47
|
+
def setup_packaging
|
|
48
|
+
Gem::PackageTask.new(gemspec) do |pkg|
|
|
49
|
+
pkg.need_zip = true
|
|
50
|
+
pkg.need_tar = true
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def define_task(path, binary_name)
|
|
55
|
+
@extension_task = Rake::ExtensionTask.new do |ext|
|
|
56
|
+
ext.name = File.basename(binary_name)
|
|
57
|
+
ext.config_script = File.basename(path)
|
|
58
|
+
ext.ext_dir = File.dirname(path)
|
|
59
|
+
ext.lib_dir = binary_lib_dir(binary_name) if binary_lib_dir(binary_name)
|
|
60
|
+
ext.gem_spec = gemspec
|
|
61
|
+
ext.cross_platform = normalized_platform
|
|
62
|
+
ext.cross_compile = true
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
disable_shared unless Gem.win_platform?
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
def binary_lib_dir(binary_name)
|
|
69
|
+
dir = File.dirname(binary_name)
|
|
70
|
+
return if dir == "."
|
|
71
|
+
|
|
72
|
+
gemspec.raw_require_paths.first + "/#{dir}"
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def darwin?
|
|
76
|
+
Gem::Platform.local.os == "darwin"
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def disable_shared
|
|
80
|
+
makefile_tasks = Rake::Task.tasks.select { |task| task.name =~ /Makefile/ }
|
|
81
|
+
|
|
82
|
+
makefile_tasks.each do |task|
|
|
83
|
+
task.enhance do
|
|
84
|
+
makefile_content = File.read(task.name)
|
|
85
|
+
makefile_content.match(/LIBRUBYARG_SHARED = (.*)/) do |match|
|
|
86
|
+
shared_flags = match[1].split(" ")
|
|
87
|
+
shared_flags.reject! { |flag| flag == "-l$(RUBY_SO_NAME)" }
|
|
88
|
+
makefile_content.gsub!(/(LIBRUBYARG_SHARED = ).*/, "\\1#{shared_flags.join(" ")}")
|
|
89
|
+
|
|
90
|
+
File.write(task.name, makefile_content)
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
def find_gemspec(glob = "*.gemspec")
|
|
97
|
+
gemspec = Dir.glob(glob).sort.first
|
|
98
|
+
return gemspec if gemspec
|
|
99
|
+
|
|
100
|
+
raise GemspecError, <<~EOM
|
|
101
|
+
Couldn't find a gemspec in the current directory.
|
|
102
|
+
Make sure to run any cibuildgem commands in the root of your gem folder.
|
|
103
|
+
EOM
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def verify_gemspec!
|
|
107
|
+
return if gemspec.extensions.any?
|
|
108
|
+
|
|
109
|
+
raise GemspecError, <<~EOM
|
|
110
|
+
Your gem has no native extention defined in its gemspec.
|
|
111
|
+
This tool can't be used on pure Ruby gems.
|
|
112
|
+
EOM
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
def parse_extconf(path)
|
|
116
|
+
visitor = CreateMakefileFinder.new
|
|
117
|
+
Prism.parse_file(path).value.accept(visitor)
|
|
118
|
+
|
|
119
|
+
visitor.binary_name
|
|
120
|
+
end
|
|
121
|
+
end
|
|
122
|
+
end
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "prism"
|
|
4
|
+
|
|
5
|
+
module Cibuildgem
|
|
6
|
+
class CreateMakefileFinder < Prism::Visitor
|
|
7
|
+
attr_reader :binary_name
|
|
8
|
+
|
|
9
|
+
def visit_call_node(node)
|
|
10
|
+
super
|
|
11
|
+
looking_for = [:create_makefile, :create_rust_makefile]
|
|
12
|
+
return unless looking_for.include?(node.name)
|
|
13
|
+
|
|
14
|
+
@binary_name = node.arguments.child_nodes.first.content
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
end
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Cibuildgem
|
|
4
|
+
module RubySeries
|
|
5
|
+
extend self
|
|
6
|
+
|
|
7
|
+
def latest_version_for_requirements(requirements)
|
|
8
|
+
latest_rubies.find do |ruby_version|
|
|
9
|
+
requirements.satisfied_by?(ruby_version)
|
|
10
|
+
end
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
# Get the minimum Ruby version to run the compilation. Getting the minimum Ruby
|
|
14
|
+
# version allows ruby/setup-ruby to download the right MSYS2 toolchain and get the
|
|
15
|
+
# right GCC version. GCC 15.1 is incompatible with Ruby 3.0 and 3.1.
|
|
16
|
+
def runtime_version_for_compilation(requirements)
|
|
17
|
+
latest_rubies.reverse.find do |ruby_version|
|
|
18
|
+
requirements.satisfied_by?(ruby_version)
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def versions_to_compile_against(requirements)
|
|
23
|
+
cross_rubies.select do |ruby_version|
|
|
24
|
+
requirements.satisfied_by?(ruby_version)
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def versions_to_test_against(requirements)
|
|
29
|
+
selected_rubies = latest_rubies.select do |ruby_version|
|
|
30
|
+
requirements.satisfied_by?(ruby_version)
|
|
31
|
+
end.reverse
|
|
32
|
+
|
|
33
|
+
selected_rubies.map do |version|
|
|
34
|
+
version.segments.tap(&:pop).join(".")
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def cross_rubies
|
|
39
|
+
[
|
|
40
|
+
Gem::Version.new("3.4.6"),
|
|
41
|
+
Gem::Version.new("3.3.8"),
|
|
42
|
+
Gem::Version.new("3.2.8"),
|
|
43
|
+
Gem::Version.new("3.1.6"),
|
|
44
|
+
]
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def latest_rubies
|
|
48
|
+
[
|
|
49
|
+
Gem::Version.new("3.4.7"),
|
|
50
|
+
Gem::Version.new("3.3.9"),
|
|
51
|
+
Gem::Version.new("3.2.9"),
|
|
52
|
+
Gem::Version.new("3.1.7"),
|
|
53
|
+
]
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../compilation_tasks"
|
|
4
|
+
|
|
5
|
+
task = Cibuildgem::CompilationTasks.new(!Rake::Task.task_defined?(:package))
|
|
6
|
+
|
|
7
|
+
task "cibuildgem:setup" do
|
|
8
|
+
Rake.application.instance_variable_get(:@tasks).delete_if do |name, _|
|
|
9
|
+
name == "native:#{task.gemspec.name}:#{task.normalized_platform}"
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
task.setup
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
task "copy:stage:lib" do
|
|
16
|
+
version = RUBY_VERSION.match(/(\d\.\d)/)[1]
|
|
17
|
+
dest = File.join(task.extension_task.lib_dir, version)
|
|
18
|
+
src = File.join("tmp", task.extension_task.cross_platform, "stage", dest)
|
|
19
|
+
|
|
20
|
+
cp_r(src, dest, remove_destination: true)
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
unless Rake::Task.task_defined?(:test)
|
|
24
|
+
task(:test) do
|
|
25
|
+
raise("Don't know how to build task 'test'") unless Rake::Task.task_defined?(:spec)
|
|
26
|
+
|
|
27
|
+
Rake::Task[:spec].invoke
|
|
28
|
+
end
|
|
29
|
+
end
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
name: "Package and release gems with precompiled binaries"
|
|
2
|
+
on:
|
|
3
|
+
workflow_dispatch:
|
|
4
|
+
inputs:
|
|
5
|
+
release:
|
|
6
|
+
description: "If the whole build passes on all platforms, release the gems on RubyGems.org"
|
|
7
|
+
required: false
|
|
8
|
+
type: boolean
|
|
9
|
+
default: false
|
|
10
|
+
jobs:
|
|
11
|
+
compile:
|
|
12
|
+
timeout-minutes: 20
|
|
13
|
+
name: "Cross compile the gem on different ruby versions"
|
|
14
|
+
strategy:
|
|
15
|
+
matrix:
|
|
16
|
+
os: <%= @os %>
|
|
17
|
+
runs-on: "${{ matrix.os }}"
|
|
18
|
+
steps:
|
|
19
|
+
- name: "Checkout code"
|
|
20
|
+
uses: "actions/checkout@v5"
|
|
21
|
+
- name: "Setup Ruby"
|
|
22
|
+
uses: "ruby/setup-ruby@v1"
|
|
23
|
+
with:
|
|
24
|
+
ruby-version: "<%= @runtime_version_for_compilation %>"
|
|
25
|
+
bundler-cache: true
|
|
26
|
+
<%- if options['working-directory'] -%>
|
|
27
|
+
working-directory: "<%= options['working-directory'] %>"
|
|
28
|
+
<%- end -%>
|
|
29
|
+
- name: "Run cibuildgem"
|
|
30
|
+
uses: "shopify/cibuildgem/.github/actions/cibuildgem@main"
|
|
31
|
+
with:
|
|
32
|
+
step: "compile"
|
|
33
|
+
<%- if options['working-directory'] -%>
|
|
34
|
+
working-directory: "<%= options['working-directory'] %>"
|
|
35
|
+
<%- end -%>
|
|
36
|
+
test:
|
|
37
|
+
timeout-minutes: 20
|
|
38
|
+
name: "Run the test suite"
|
|
39
|
+
needs: compile
|
|
40
|
+
strategy:
|
|
41
|
+
matrix:
|
|
42
|
+
os: <%= @os %>
|
|
43
|
+
rubies: <%= @ruby_versions_for_testing %>
|
|
44
|
+
type: ["cross", "native"]
|
|
45
|
+
runs-on: "${{ matrix.os }}"
|
|
46
|
+
steps:
|
|
47
|
+
- name: "Checkout code"
|
|
48
|
+
uses: "actions/checkout@v5"
|
|
49
|
+
- name: "Setup Ruby"
|
|
50
|
+
uses: "ruby/setup-ruby@v1"
|
|
51
|
+
with:
|
|
52
|
+
ruby-version: "${{ matrix.rubies }}"
|
|
53
|
+
bundler-cache: true
|
|
54
|
+
<%- if options['working-directory'] -%>
|
|
55
|
+
working-directory: "<%= options['working-directory'] %>"
|
|
56
|
+
<%- end -%>
|
|
57
|
+
- name: "Run cibuildgem"
|
|
58
|
+
uses: "shopify/cibuildgem/.github/actions/cibuildgem@main"
|
|
59
|
+
with:
|
|
60
|
+
step: "test_${{ matrix.type }}"
|
|
61
|
+
<%- if options['working-directory'] -%>
|
|
62
|
+
working-directory: "<%= options['working-directory'] %>"
|
|
63
|
+
<%- end -%>
|
|
64
|
+
<%- if options['test-command'] -%>
|
|
65
|
+
test-command: "<%= options['test-command'] %>"
|
|
66
|
+
<%- end -%>
|
|
67
|
+
install:
|
|
68
|
+
timeout-minutes: 5
|
|
69
|
+
name: "Verify the gem can be installed"
|
|
70
|
+
needs: test
|
|
71
|
+
strategy:
|
|
72
|
+
matrix:
|
|
73
|
+
os: <%= @os %>
|
|
74
|
+
runs-on: "${{ matrix.os }}"
|
|
75
|
+
steps:
|
|
76
|
+
- name: "Setup Ruby"
|
|
77
|
+
uses: "ruby/setup-ruby@v1"
|
|
78
|
+
with:
|
|
79
|
+
ruby-version: "<%= @latest_supported_ruby_version %>"
|
|
80
|
+
- name: "Run cibuildgem"
|
|
81
|
+
uses: "shopify/cibuildgem/.github/actions/cibuildgem@main"
|
|
82
|
+
with:
|
|
83
|
+
step: "install"
|
|
84
|
+
release:
|
|
85
|
+
permissions:
|
|
86
|
+
id-token: write
|
|
87
|
+
contents: read
|
|
88
|
+
timeout-minutes: 5
|
|
89
|
+
if: ${{ inputs.release }}
|
|
90
|
+
name: "Release all gems with RubyGems"
|
|
91
|
+
needs: install
|
|
92
|
+
runs-on: "ubuntu-latest"
|
|
93
|
+
steps:
|
|
94
|
+
- name: "Setup Ruby"
|
|
95
|
+
uses: "ruby/setup-ruby@v1"
|
|
96
|
+
with:
|
|
97
|
+
ruby-version: "3.4.7"
|
|
98
|
+
- name: "Run cibuildgem"
|
|
99
|
+
uses: "shopify/cibuildgem/.github/actions/cibuildgem@main"
|
|
100
|
+
with:
|
|
101
|
+
step: "release"
|
data/lib/cibuildgem.rb
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "cibuildgem/version"
|
|
4
|
+
require_relative "cibuildgem/errors"
|
|
5
|
+
|
|
6
|
+
module Cibuildgem
|
|
7
|
+
autoload :CLI, "cibuildgem/cli"
|
|
8
|
+
autoload :CompilationTasks, "cibuildgem/compilation_tasks"
|
|
9
|
+
autoload :RubySeries, "cibuildgem/ruby_series"
|
|
10
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: cibuildgem
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.1
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Shopify
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: prism
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '0'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - ">="
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '0'
|
|
26
|
+
- !ruby/object:Gem::Dependency
|
|
27
|
+
name: rake-compiler
|
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
|
29
|
+
requirements:
|
|
30
|
+
- - ">="
|
|
31
|
+
- !ruby/object:Gem::Version
|
|
32
|
+
version: '0'
|
|
33
|
+
type: :runtime
|
|
34
|
+
prerelease: false
|
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
36
|
+
requirements:
|
|
37
|
+
- - ">="
|
|
38
|
+
- !ruby/object:Gem::Version
|
|
39
|
+
version: '0'
|
|
40
|
+
- !ruby/object:Gem::Dependency
|
|
41
|
+
name: thor
|
|
42
|
+
requirement: !ruby/object:Gem::Requirement
|
|
43
|
+
requirements:
|
|
44
|
+
- - ">="
|
|
45
|
+
- !ruby/object:Gem::Version
|
|
46
|
+
version: '0'
|
|
47
|
+
type: :runtime
|
|
48
|
+
prerelease: false
|
|
49
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
50
|
+
requirements:
|
|
51
|
+
- - ">="
|
|
52
|
+
- !ruby/object:Gem::Version
|
|
53
|
+
version: '0'
|
|
54
|
+
description: |
|
|
55
|
+
Gems with native extensions are the main bottleneck for a user when running `bundle install`.
|
|
56
|
+
This gem aims to provide the Ruby community an easy to opt-in and quick way to distribute their
|
|
57
|
+
gems with precompiled binaries.
|
|
58
|
+
|
|
59
|
+
This toolchain works with a native CI based compilation approach using GitHub actions. It piggyback on
|
|
60
|
+
top of popular tools in the Ruby ecosystem that maintainers are used to such as Rake Compiler and ruby/setup-ruby.
|
|
61
|
+
email:
|
|
62
|
+
- rails@shopify.com
|
|
63
|
+
executables:
|
|
64
|
+
- cibuildgem
|
|
65
|
+
extensions: []
|
|
66
|
+
extra_rdoc_files: []
|
|
67
|
+
files:
|
|
68
|
+
- LICENSE.md
|
|
69
|
+
- README.md
|
|
70
|
+
- exe/cibuildgem
|
|
71
|
+
- lib/cibuildgem.rb
|
|
72
|
+
- lib/cibuildgem/cli.rb
|
|
73
|
+
- lib/cibuildgem/compilation_tasks.rb
|
|
74
|
+
- lib/cibuildgem/create_makefile_finder.rb
|
|
75
|
+
- lib/cibuildgem/errors.rb
|
|
76
|
+
- lib/cibuildgem/ruby_series.rb
|
|
77
|
+
- lib/cibuildgem/tasks/wrapper.rake
|
|
78
|
+
- lib/cibuildgem/templates/github/workflows/cibuildgem.yaml.tt
|
|
79
|
+
- lib/cibuildgem/version.rb
|
|
80
|
+
homepage: https://github.com/shopify/cibuildwheel
|
|
81
|
+
licenses:
|
|
82
|
+
- MIT
|
|
83
|
+
metadata:
|
|
84
|
+
allowed_push_host: https://rubygems.org
|
|
85
|
+
homepage_uri: https://github.com/shopify/cibuildwheel
|
|
86
|
+
source_code_uri: https://github.com/shopify/cibuildwheel
|
|
87
|
+
rdoc_options: []
|
|
88
|
+
require_paths:
|
|
89
|
+
- lib
|
|
90
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
91
|
+
requirements:
|
|
92
|
+
- - ">="
|
|
93
|
+
- !ruby/object:Gem::Version
|
|
94
|
+
version: 3.0.0
|
|
95
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
96
|
+
requirements:
|
|
97
|
+
- - ">="
|
|
98
|
+
- !ruby/object:Gem::Version
|
|
99
|
+
version: '0'
|
|
100
|
+
requirements: []
|
|
101
|
+
rubygems_version: 3.6.9
|
|
102
|
+
specification_version: 4
|
|
103
|
+
summary: Assist developers to distrute gems with precompiled binaries.
|
|
104
|
+
test_files: []
|