tebako 0.7.4 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CMakeLists.txt +23 -27
- data/README.adoc +689 -173
- data/common.env +1 -1
- data/lib/tebako/cli_rubies.rb +3 -5
- data/lib/tebako/packager/pass1.rb +25 -12
- data/lib/tebako/packager/pass2.rb +14 -16
- data/lib/tebako/packager/patch_buildsystem.rb +41 -7
- data/lib/tebako/packager/patch_helpers.rb +0 -4
- data/lib/tebako/packager/patch_libraries.rb +85 -68
- data/lib/tebako/packager/patch_literals.rb +15 -17
- data/lib/tebako/version.rb +1 -1
- data/src/tebako-main.cpp +7 -0
- data/version.txt +1 -1
- metadata +6 -6
data/README.adoc
CHANGED
@@ -1,70 +1,416 @@
|
|
1
|
-
= Tebako:
|
1
|
+
= Tebako: An advanced image packager for interpretive languages
|
2
2
|
|
3
|
+
Platform tests on GitHub:
|
3
4
|
image:https://github.com/tamatebako/tebako/actions/workflows/ubuntu.yml/badge.svg["Ubuntu amd64", link="https://github.com/tamatebako/tebako/actions/workflows/ubuntu.yml"]
|
4
5
|
image:https://github.com/tamatebako/tebako/actions/workflows/alpine.yml/badge.svg["Alpine", link="https://github.com/tamatebako/tebako/actions/workflows/alpine.yml"]
|
5
|
-
image:https://github.com/tamatebako/tebako/actions/workflows/macos.yml/badge.svg["
|
6
|
+
image:https://github.com/tamatebako/tebako/actions/workflows/macos.yml/badge.svg["macOS amd64", link="https://github.com/tamatebako/tebako/actions/workflows/macos.yml"]
|
6
7
|
image:https://github.com/tamatebako/tebako/actions/workflows/windows-msys.yml/badge.svg["Windows msys", link="https://github.com/tamatebako/tebako/actions/workflows/windows-msys.yml"]
|
7
8
|
|
9
|
+
Platform tests on Cirrus:
|
8
10
|
image:https://api.cirrus-ci.com/github/tamatebako/tebako.svg?branch=main&task=ubuntu-aarch64["Ubuntu aarch64", link="https://cirrus-ci.com/github/tamatebako/tebako"]
|
9
11
|
|
12
|
+
Quality:
|
10
13
|
image:https://github.com/tamatebako/tebako/actions/workflows/lint.yml/badge.svg["lint", link="https://github.com/tamatebako/tebako/actions/workflows/lint.yml"]
|
11
14
|
|
12
15
|
== Purpose
|
13
16
|
|
14
|
-
Tebako is an executable packager
|
15
|
-
|
17
|
+
Tebako is an advanced executable packager designed for applications written in
|
18
|
+
interpretive languages.
|
16
19
|
|
17
|
-
|
18
|
-
|
19
|
-
packaged software from a point in the file system.
|
20
|
+
It simplifies distribution and deployment by packaging your entire project with
|
21
|
+
a bundled runtime into a single, performant, executable binary.
|
20
22
|
|
21
|
-
|
23
|
+
== Architecture
|
22
24
|
|
23
|
-
|
24
|
-
* Support signing of the binary on macOS (via notarization)
|
25
|
+
A Tebako-packaged binary is effectively a self-executing container-in-a-file.
|
25
26
|
|
26
|
-
|
27
|
+
The packaged binary contains the following components:
|
27
28
|
|
28
|
-
*
|
29
|
-
|
30
|
-
|
31
|
-
|
29
|
+
* An on-file filesystem (OFFS) containing all the project files and
|
30
|
+
dependencies in DwarFS format.
|
31
|
+
|
32
|
+
* A runtime environment that includes the necessary libraries and interpreters,
|
33
|
+
with patched filesystem calls that redirect access of project files to the
|
34
|
+
on-file filesystem.
|
32
35
|
|
33
|
-
|
36
|
+
* An executable loader that loads the on-file filesystem in memory and executes
|
37
|
+
the project.
|
38
|
+
|
39
|
+
|
40
|
+
== Supported runtimes, platforms and architectures
|
41
|
+
|
42
|
+
Tebako artifacts can be built and executed on the following platforms and
|
43
|
+
architectures.
|
44
|
+
|
45
|
+
.Supported platforms and architectures
|
46
|
+
[cols="3", options="header"]
|
47
|
+
|===
|
48
|
+
| Platform and version | Architectures | Build system
|
34
49
|
|
35
|
-
|
50
|
+
3+| **Linux**
|
51
|
+
| Ubuntu 20.04 | amd64, aarch64 | gcc/g+\+: 10; clang/clang++: 12
|
52
|
+
| Alpine 3.17 | amd64 | gcc/g+\+: default; clang/clang++: default
|
36
53
|
|
37
|
-
|
38
|
-
|
39
|
-
|
54
|
+
3+| **macOS**
|
55
|
+
| macOS 12 (Monterey) | amd64 | xcode: [13.1, 14.3.1]
|
56
|
+
| macOS 13 (Ventura) | amd64, aarch64 | xcode: [13.1, 14.3.1]
|
57
|
+
| macOS 14 (Sonoma) | amd64, aarch64 | xcode: [13.1, 14.3.1, 15.4]
|
40
58
|
|
41
|
-
|
59
|
+
3+| **Windows**
|
60
|
+
| Windows 10 | amd64 | MinGW ucrt64
|
61
|
+
| Windows 11 | amd64 | MinGW ucrt64
|
62
|
+
| Windows Server 2019 | amd64 | MinGW ucrt64
|
63
|
+
| Windows Server 2022 | amd64 | MinGW ucrt64
|
42
64
|
|
43
|
-
|
65
|
+
|===
|
66
|
+
|
67
|
+
[NOTE]
|
68
|
+
====
|
69
|
+
Windows build caveats:
|
70
|
+
|
71
|
+
* Tebako may face errors related to CMake path length limitations (https://gitlab.kitware.com/cmake/cmake/-/issues/25936).
|
44
72
|
This error may affect not tebako itself but the gems that need to be package and use CMake to build native extensions.
|
45
73
|
There is no workaround for this issue as it looks like is a limitation of the manifest used to build CMake executable.
|
74
|
+
|
46
75
|
* MSys strip utility creates broken executable when tebako image is processed. Linking with '-s' flag produces unusable
|
47
76
|
executables as well.
|
48
77
|
Until this issue (https://github.com/tamatebako/tebako/issues/172) is resolved we plan to produce an Windows executable
|
49
78
|
with debug information unstripped. You can opt to run 'strip -S' manually, it most cases it works.
|
79
|
+
====
|
80
|
+
|
81
|
+
|
82
|
+
.Supported Ruby versions
|
83
|
+
[cols="2", options="header"]
|
84
|
+
|===
|
85
|
+
| Ruby version | Supported platforms
|
86
|
+
|
87
|
+
| 2.7.8 | Linux, macOS
|
88
|
+
| 3.0.7 | Linux, macOS
|
89
|
+
| 3.1.{4,5,6} | Linux, macOS, Windows
|
90
|
+
| 3.2.{3,4} | Linux, macOS, Windows
|
91
|
+
| 3.3.{3,4} | Linux, macOS, Windows
|
92
|
+
|
93
|
+
|===
|
94
|
+
|
95
|
+
NOTE: Our goal is to support all maintained Ruby releases, including minor versions.
|
96
|
+
|
97
|
+
== Future plans
|
98
|
+
|
99
|
+
* Downloading new DwarFS images to be stored in the local home directory
|
100
|
+
* Allowing loading multiple DwarFS images in a stacked way
|
101
|
+
* Supporting a COW mechanism that the newly written files are stored
|
102
|
+
in a separate image that can be loaded on top of the read-only file systems.
|
103
|
+
|
104
|
+
|
105
|
+
== FAQ
|
106
|
+
|
107
|
+
=== Why use Tebako?
|
108
|
+
|
109
|
+
Tebako is particularly useful for developers who need to:
|
110
|
+
|
111
|
+
* Distribute applications without requiring users to have specific runtimes installed.
|
112
|
+
* Simplify the deployment process by packaging all dependencies into one binary.
|
113
|
+
* Ensure consistency across different environments by using a single executable.
|
114
|
+
* Flexibility to support different runtime versions on the user's machine.
|
115
|
+
|
116
|
+
|
117
|
+
=== How do I know I need Tebako?
|
118
|
+
|
119
|
+
You might need Tebako if you:
|
120
|
+
|
121
|
+
* Want to package your application into a single, self-contained binary.
|
122
|
+
* Want to avoid the complexities of managing runtime environments on target machines.
|
123
|
+
* Distribute software to environments where installing runtimes and their dependencies is challenging.
|
124
|
+
* Require a streamlined way to deliver applications to end-users.
|
125
|
+
* Need to ensure that your application runs consistently across different environments and architectures.
|
126
|
+
|
127
|
+
|
128
|
+
=== What is DwarFS?
|
129
|
+
|
130
|
+
https://github.com/mhx/dwarfs[DwarFS] is a fast, high compression read-only
|
131
|
+
user-land file system designed for efficient storage and access of large
|
132
|
+
collections of files.
|
133
|
+
|
134
|
+
It is used by Tebako to package applications into a compact and efficient format.
|
135
|
+
|
136
|
+
Tebako uses https://github.com/tamatebako/libdwarfs[libdwarfs], the library
|
137
|
+
form of https://github.com/mhx/dwarfs[DwarFS], developed for the Tebako project.
|
138
|
+
|
139
|
+
=== When is Tebako better than comparable solutions?
|
140
|
+
|
141
|
+
Tebako offers several advantages over comparable solutions for supported
|
142
|
+
interpretive languages.
|
143
|
+
|
144
|
+
They are listed in order of the degree of virtualization below.
|
145
|
+
|
146
|
+
Tebako stands out by providing a lightweight runtime bundling approach that
|
147
|
+
simplifies distribution and deployment while offering flexibility and
|
148
|
+
efficiency.
|
149
|
+
|
150
|
+
It eliminates the need for users to have specific runtimes installed and ensures
|
151
|
+
consistency across different environments.
|
152
|
+
|
153
|
+
With Tebako, you can package your entire project with a bundled runtime into a
|
154
|
+
single, performant, executable binary.
|
155
|
+
|
156
|
+
[cols="a,3a,3a"]
|
157
|
+
|===
|
158
|
+
| Solution | Pros | Cons
|
159
|
+
|
160
|
+
| Virtual machines (VMs)
|
161
|
+
|
|
162
|
+
- Provides full isolation and compatibility across environments
|
163
|
+
|
|
164
|
+
- Requires a separate VM installation for each application
|
165
|
+
- Heavy resource consumption for virtualization
|
166
|
+
|
167
|
+
| Docker
|
168
|
+
|
|
169
|
+
- Provides portable containers
|
170
|
+
- Isolates entire applications and their dependencies
|
171
|
+
- Supports easy deployment and scalability
|
172
|
+
|
|
173
|
+
- Requires Docker installation and management
|
174
|
+
- Requires administrative rights on machine
|
175
|
+
- Containerization overhead
|
176
|
+
|
177
|
+
| *Tebako*
|
178
|
+
|
|
179
|
+
- Packages all files and dependencies into a single binary
|
180
|
+
- Supports multiple operating systems and architectures
|
181
|
+
- Provides efficient packaging and execution with DwarFS
|
182
|
+
- Offers security features like signing on macOS
|
183
|
+
- Simplifies distribution and deployment
|
184
|
+
- Native running speed
|
185
|
+
|
|
186
|
+
- Initial packaging time longer than Ruby gems
|
187
|
+
- Minor runtime overhead
|
188
|
+
|
189
|
+
| Ruby Gems
|
190
|
+
|
|
191
|
+
- Easy installation of Ruby libraries
|
192
|
+
- Provides user-side version control and dependency management
|
193
|
+
|
|
194
|
+
- Requires Ruby installation and gem management
|
195
|
+
- Runtime execution dependent on the user's installed Ruby version and gems
|
196
|
+
|
197
|
+
|===
|
198
|
+
|
199
|
+
|
200
|
+
== Usage
|
201
|
+
|
202
|
+
=== Command-line interface
|
203
|
+
|
204
|
+
Tebako works by packaging your project into a single executable binary that
|
205
|
+
includes all the necessary dependencies.
|
206
|
+
|
207
|
+
The way to work with Tebako is through its command-line interface (CLI).
|
208
|
+
It provides the following commands:
|
209
|
+
|
210
|
+
`setup`::
|
211
|
+
Prepares the Tebako packaging environment.
|
212
|
+
|
213
|
+
`press`::
|
214
|
+
Packages a project into a single executable binary.
|
215
|
+
|
216
|
+
`clean`::
|
217
|
+
Removes Tebako artifacts.
|
218
|
+
|
219
|
+
`clean_ruby`::
|
220
|
+
Removes Tebako Ruby artifacts.
|
221
|
+
|
222
|
+
`hash`::
|
223
|
+
Calculates the Tebako script hash for use as a cache key in CI/CD environments.
|
224
|
+
|
225
|
+
`extract`::
|
226
|
+
Extracts the filesystem from a Tebako package.
|
227
|
+
|
228
|
+
`version`::
|
229
|
+
Displays the Tebako version.
|
230
|
+
|
231
|
+
`help`::
|
232
|
+
Displays the help message.
|
233
|
+
|
234
|
+
|
235
|
+
|
236
|
+
|
237
|
+
== Usage
|
238
|
+
|
239
|
+
=== General
|
240
|
+
|
241
|
+
Tebako can be used in two ways:
|
242
|
+
|
243
|
+
* Through the Tebako container
|
244
|
+
* Local installation
|
245
|
+
|
246
|
+
Please refer to the <<installation>> section on how to install Tebako.
|
247
|
+
|
248
|
+
|
249
|
+
[[installation]]
|
250
|
+
== Installation
|
251
|
+
|
252
|
+
=== General
|
253
|
+
|
254
|
+
Installation of Tebako is only needed in order to package an application.
|
255
|
+
|
256
|
+
There is no need to install anything for users who run the packaged application.
|
257
|
+
|
258
|
+
|
259
|
+
=== Using Docker
|
260
|
+
|
261
|
+
==== General
|
262
|
+
|
263
|
+
If you have Docker installed and available, the easiest way to run Tebako is
|
264
|
+
through the official Docker containers.
|
265
|
+
|
266
|
+
Docker containers with preinstalled Tebako packaging environments for Ubuntu and
|
267
|
+
Alpine Linux are available at
|
268
|
+
https://github.com/tamatebako/tebako-ci-containers[tebako-ci-containers].
|
269
|
+
|
270
|
+
|
271
|
+
==== Pull the container
|
272
|
+
|
273
|
+
Pull the Tebako container image.
|
274
|
+
|
275
|
+
[source,sh]
|
276
|
+
----
|
277
|
+
docker pull ghcr.io/tamatebako/tebako-<container_tag>:latest
|
278
|
+
----
|
279
|
+
|
280
|
+
`<container_tag>`:: is the desired image tag (e.g., `ubuntu-20.04` or `alpine-3.17`).
|
281
|
+
|
282
|
+
|
283
|
+
==== Running Tebako commands in the container
|
284
|
+
|
285
|
+
Simply prefix the Tebako command with `docker run` and the container image.
|
286
|
+
|
287
|
+
[source,sh]
|
288
|
+
----
|
289
|
+
docker run -v <application_folder>:/mnt/w \
|
290
|
+
-t ghcr.io/tamatebako/tebako-<container_tag>:latest \
|
291
|
+
tebako {command} {parameters}
|
292
|
+
----
|
293
|
+
|
294
|
+
|
295
|
+
|
296
|
+
|
297
|
+
==== Packaging from outside the container
|
298
|
+
|
299
|
+
To package your application from outside the container, just run a single Docker
|
300
|
+
command.
|
301
|
+
|
302
|
+
This command mounts the application folder into the container and runs the
|
303
|
+
`tebako press` command, specifying the application root, entry point, output
|
304
|
+
location, and Ruby version.
|
305
|
+
|
306
|
+
[source,sh]
|
307
|
+
----
|
308
|
+
docker run -v <application_folder>:/mnt/w \
|
309
|
+
-t ghcr.io/tamatebako/tebako-<container_tag>:latest \
|
310
|
+
tebako press <tebako-press-parameters>
|
311
|
+
----
|
312
|
+
|
313
|
+
`<application_folder>`:: is the path to your application folder.
|
314
|
+
|
315
|
+
`<container_tag>`:: is the desired image tag (e.g., `ubuntu-20.04` or `alpine-3.17`).
|
316
|
+
|
317
|
+
|
318
|
+
[example]
|
319
|
+
====
|
320
|
+
Assume that you have a Ruby application in the `fontist` folder of the current
|
321
|
+
directory.
|
322
|
+
|
323
|
+
You can package it to `./fontist-package` using the following command:
|
324
|
+
|
325
|
+
[source,sh]
|
326
|
+
----
|
327
|
+
docker run -v $PWD:/mnt/w \
|
328
|
+
-t ghcr.io/tamatebako/tebako-ubuntu-20.04:latest \
|
329
|
+
tebako press --root=/mnt/w/fontist --entry-point=fontist --output=/mnt/w/fontist-package --Ruby=3.2.4
|
330
|
+
----
|
331
|
+
====
|
332
|
+
|
333
|
+
==== Packaging from inside the container
|
334
|
+
|
335
|
+
It is also possible to package an application from inside the Tebako container.
|
336
|
+
|
337
|
+
Start and enter the container interactively.
|
338
|
+
|
339
|
+
[source,sh]
|
340
|
+
----
|
341
|
+
docker run -it --rm -v <application_folder>:/mnt/w \
|
342
|
+
ghcr.io/tamatebako/tebako-<container_tag>:latest bash
|
343
|
+
----
|
344
|
+
|
345
|
+
`<application_folder>`:: is the path to your application folder.
|
346
|
+
|
347
|
+
`<container_tag>`:: is the desired image tag (e.g., `ubuntu-20.04` or `alpine-3.17`).
|
348
|
+
|
349
|
+
|
350
|
+
Once inside, run the `tebako press` command:
|
351
|
+
|
352
|
+
[source,sh]
|
353
|
+
----
|
354
|
+
tebako press <tebako press parameters>
|
355
|
+
----
|
356
|
+
|
357
|
+
[example]
|
358
|
+
====
|
359
|
+
Assume that you have a Ruby application in the `fontist` folder of the current
|
360
|
+
directory.
|
361
|
+
|
362
|
+
You can package it to `./fontist-package` using the following command:
|
363
|
+
|
364
|
+
[source,sh]
|
365
|
+
----
|
366
|
+
$ docker run -it --rm -v $PWD:/mnt/w ghcr.io/tamatebako/tebako-<container_tag>:latest bash
|
367
|
+
|
368
|
+
# Inside the container:
|
369
|
+
$ tebako press --root=/mnt/w/fontist --entry-point=fontist --output=/mnt/w/fontist-package --Ruby=3.2.4
|
370
|
+
----
|
371
|
+
====
|
372
|
+
|
373
|
+
|
374
|
+
=== Local installation
|
375
|
+
|
376
|
+
==== General
|
377
|
+
|
378
|
+
There are cases where Docker may not be suitable for your needs, such as:
|
379
|
+
|
380
|
+
. Admin privileges: Running Docker requires administrative privileges, which
|
381
|
+
means Docker may not be available to users on their machines.
|
382
|
+
|
383
|
+
. Performance penalty: Docker introduces a performance penalty due to the
|
384
|
+
overhead of running containers. This can be a concern when packaging complex
|
385
|
+
applications that require heavy memory usage.
|
50
386
|
|
51
|
-
|
387
|
+
In such cases, you can choose to install Tebako locally.
|
52
388
|
|
53
|
-
|
389
|
+
Tebako is distributed as a Ruby gem. A Ruby environment is necessary.
|
54
390
|
|
55
|
-
* 2.7.8 (Linux, MacOS)
|
56
|
-
* 3.0.7 (Linux, MacOS)
|
57
|
-
* 3.1.4, 3.1.5, 3.1.6 (Linux, MacOS, Windows)
|
58
|
-
* 3.2.3, 3.2.4 (Linux, MacOS, Windows)
|
59
391
|
|
60
|
-
|
61
|
-
|
392
|
+
[source,sh]
|
393
|
+
----
|
394
|
+
$ gem install tebako
|
395
|
+
----
|
396
|
+
|
397
|
+
|
398
|
+
==== Prerequisites
|
62
399
|
|
63
|
-
|
400
|
+
These prerequisites are needed only for users who want to install Tebako on
|
401
|
+
their machine and build all Tebako components locally.
|
64
402
|
|
65
|
-
|
403
|
+
If you use Docker, there is no need to set up these prerequisites.
|
66
404
|
|
67
|
-
|
405
|
+
===== Ubuntu 20.04
|
406
|
+
|
407
|
+
====== General
|
408
|
+
|
409
|
+
There are several prerequisites that need to be installed on Ubuntu 20.04 for
|
410
|
+
Tebako to work correctly.
|
411
|
+
|
412
|
+
|
413
|
+
====== GNU C/C++ 10+ or Clang C/C++ 12+
|
68
414
|
|
69
415
|
[source,sh]
|
70
416
|
----
|
@@ -82,12 +428,14 @@ update-alternatives --install /usr/bin/clang clang /usr/bin/clang-12 150
|
|
82
428
|
update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-12 150
|
83
429
|
----
|
84
430
|
|
85
|
-
|
431
|
+
====== CMake version 3.20+
|
86
432
|
|
87
|
-
Tebako
|
433
|
+
Tebako requires CMake at a version of at least 3.20+.
|
88
434
|
|
89
|
-
If
|
435
|
+
If such CMake version is not available as a default package, set it up as
|
436
|
+
follows.
|
90
437
|
|
438
|
+
.Installing CMake 3.20+
|
91
439
|
[source,sh]
|
92
440
|
----
|
93
441
|
apt-get remove --purge --auto-remove cmake
|
@@ -98,7 +446,7 @@ curl https://apt.kitware.com/kitware-archive.sh | bash
|
|
98
446
|
apt-get install cmake
|
99
447
|
----
|
100
448
|
|
101
|
-
|
449
|
+
====== Other development tools and libraries
|
102
450
|
|
103
451
|
[source,sh]
|
104
452
|
----
|
@@ -113,7 +461,16 @@ apt-get -y install sudo git curl build-essential pkg-config bison flex autoconf
|
|
113
461
|
libutfcpp-dev
|
114
462
|
----
|
115
463
|
|
116
|
-
|
464
|
+
===== Alpine 3.17
|
465
|
+
|
466
|
+
====== General
|
467
|
+
|
468
|
+
There are several prerequisites that need to be installed on Alpine 3.17 for
|
469
|
+
Tebako to work correctly.
|
470
|
+
|
471
|
+
====== APK packages
|
472
|
+
|
473
|
+
Run the following command to install all prerequisites.
|
117
474
|
|
118
475
|
[source,sh]
|
119
476
|
----
|
@@ -127,117 +484,212 @@ apk --no-cache --upgrade add build-base cmake git bash autoconf boost-static \
|
|
127
484
|
brotli-static jemalloc-dev fmt-dev xz-static
|
128
485
|
----
|
129
486
|
|
130
|
-
|
487
|
+
===== macOS
|
488
|
+
|
489
|
+
====== General
|
490
|
+
|
491
|
+
There are several prerequisites that need to be installed on macOS for Tebako to work correctly.
|
492
|
+
|
493
|
+
The following instructions work for:
|
494
|
+
|
495
|
+
* macOS 12 (Monterey) through macOS 14 (Sonoma)
|
496
|
+
|
497
|
+
|
498
|
+
====== Homebrew packages
|
499
|
+
|
500
|
+
We use Homebrew to install the necessary packages on macOS.
|
131
501
|
|
132
502
|
[source,sh]
|
133
503
|
----
|
134
504
|
brew update
|
135
|
-
brew install gnu-sed bash pkg-config bison flex binutils libffi gdbm zlib
|
136
|
-
double-conversion boost jemalloc fmt glog libevent libsodium lz4 xz
|
505
|
+
brew install gnu-sed bash pkg-config bison flex binutils libffi gdbm zlib \
|
506
|
+
ncurses double-conversion boost jemalloc fmt glog libevent libsodium lz4 xz \
|
507
|
+
libyaml openssl@3
|
137
508
|
----
|
138
509
|
|
139
|
-
|
510
|
+
====== Bison 3+
|
511
|
+
|
512
|
+
Tebako requires Bison 3+.
|
513
|
+
|
514
|
+
On macOS 14, the default Bison version is 2.3, and the Homebrew formula is keg-only,
|
515
|
+
which means that the full path to the Bison binary must be used to utilize the
|
516
|
+
correct version.
|
517
|
+
|
518
|
+
Run the following command prior to using Tebako, or add it into your shell
|
519
|
+
profile.
|
520
|
+
|
521
|
+
====== jemalloc Library Build
|
140
522
|
|
141
|
-
|
142
|
-
|
523
|
+
The `libdwarfs` build script creates an additional jemalloc installation on macOS. This is done to satisfy the magic applied by folly during linking but uses a static library.
|
524
|
+
If the library is created in an emulated environment (QEMU, Rosetta, etc.), there are known issues (link:https://github.com/jemalloc/jemalloc/issues/1997[jemalloc issue #1997]) where jemalloc incorrectly defines the number of significant virtual address bits (lg-vaddr parameter).
|
525
|
+
|
526
|
+
These issues can be fixed by explicitly setting the `--with-lg-vaddr` parameter for the jemalloc build. We decided not to automate this since we do not feel that we can provide reasonable test coverage. Instead, our build script accepts the `LG_VADDR` environment variable and passes it to the jemalloc build as `--with-lg-vaddr=${LG_VADDR}`.
|
527
|
+
|
528
|
+
The `LG_VADDR` parameter specifies the number of significant virtual address bits, which can vary based on the CPU architecture and emulation status.
|
529
|
+
|
530
|
+
Simple script to set `LG_VADDR`. Please note that it is provided for illustration only.
|
143
531
|
|
144
532
|
[source,sh]
|
145
533
|
----
|
146
|
-
|
147
|
-
----
|
534
|
+
#!/bin/bash
|
148
535
|
|
149
|
-
|
536
|
+
# Check the CPU architecture
|
537
|
+
ARCH=$(uname -m)
|
538
|
+
|
539
|
+
# Check if running under Rosetta 2 emulation
|
540
|
+
if [[ "$ARCH" == "x86_64" && $(sysctl -n sysctl.proc_translated) == "1" ]]; then
|
541
|
+
echo "Running on Apple Silicon under Rosetta 2 emulation"
|
542
|
+
export LG_VADDR=39
|
543
|
+
elif [[ "$ARCH" == "arm64" ]]; then
|
544
|
+
echo "Running on Apple Silicon"
|
545
|
+
export LG_VADDR=39
|
546
|
+
else
|
547
|
+
echo "Running on Intel Silicon"
|
548
|
+
export LG_VADDR=48
|
549
|
+
fi
|
550
|
+
|
551
|
+
echo "Setting lg-vaddr to $LG_VADDR"
|
552
|
+
----
|
150
553
|
|
151
|
-
The simplest approach is to use Ruby development environment provided by RubyInstaller, for example Ruby+Devkit 3.1.4-1.
|
152
|
-
Once it is installed use the following commands:
|
153
554
|
|
154
555
|
[source,sh]
|
155
556
|
----
|
156
|
-
|
157
|
-
pacman -S git tar bison flex toolchain make cmake
|
158
|
-
boost diffutils libevent double-conversion
|
159
|
-
fmt glog dlfcn gtest autotools ncurses libyaml
|
557
|
+
export PATH="$(brew --prefix bison)/bin:$PATH"
|
160
558
|
----
|
161
559
|
|
162
|
-
|
560
|
+
===== Windows
|
163
561
|
|
164
|
-
|
562
|
+
====== General
|
165
563
|
|
166
|
-
|
167
|
-
Additionally, Docker containers for Ubuntu and Alpine Linux with a preinstalled Tebako packaging environment
|
168
|
-
are available at https://github.com/orgs/tamatebako/packages.
|
564
|
+
There are several prerequisites that need to be installed on macOS for Tebako to work correctly.
|
169
565
|
|
170
|
-
|
566
|
+
The following instructions work for:
|
171
567
|
|
172
|
-
|
568
|
+
* Windows 10, 11
|
569
|
+
* Windows Server 2019, 2022
|
570
|
+
|
571
|
+
====== Ruby
|
173
572
|
|
174
|
-
|
573
|
+
To run Tebako you need to have Ruby installed.
|
574
|
+
It is simplest to use the Ruby development environment provided by
|
575
|
+
https://rubyinstaller.org[RubyInstaller].
|
576
|
+
|
577
|
+
For example, Ruby+Devkit 3.1.4-1.
|
578
|
+
|
579
|
+
====== MinGW ucrt64
|
580
|
+
|
581
|
+
Enable MinGW ucrt64 and install the necessary packages.
|
582
|
+
|
583
|
+
The `ridk` command originates from the RubyInstaller installation.
|
175
584
|
|
176
585
|
[source,sh]
|
177
586
|
----
|
178
|
-
|
587
|
+
$ ridk enable ucrt64
|
588
|
+
$ pacman -S git tar bison flex toolchain make cmake
|
589
|
+
boost diffutils libevent double-conversion
|
590
|
+
fmt glog dlfcn gtest autotools ncurses libyaml
|
179
591
|
----
|
180
592
|
|
181
|
-
=== Tebako Root Folder (aka Prefix) Selection
|
182
593
|
|
183
|
-
The prefix in Tebako determines the base directory for the Tebako setup. It is an essential part of configuring how Tebako operates within your system.
|
184
|
-
The selection of the prefix follows a specific order of precedence to ensure flexibility and ease of use:
|
185
594
|
|
186
|
-
|
595
|
+
== Usage
|
596
|
+
|
597
|
+
=== Tebako root folder (aka prefix) selection
|
598
|
+
|
599
|
+
The Tebako prefix determines the base directory for the Tebako setup.
|
600
|
+
|
601
|
+
It is an essential part of configuring how Tebako operates within your system.
|
602
|
+
|
603
|
+
The selection of the Tebako prefix follows a specific order of precedence to
|
604
|
+
ensure flexibility and ease of use:
|
187
605
|
|
188
|
-
. *
|
606
|
+
. *User-specified prefix*:
|
607
|
+
The most direct way to set the root folder is by specifying it through a
|
608
|
+
command-line argument.
|
189
609
|
|
190
|
-
. *
|
610
|
+
. *Current Working Directory (PWD)*:
|
611
|
+
If the prefix option is explicitly set to `PWD`, Tebako uses the current working
|
612
|
+
directory as Tebako root folder.
|
191
613
|
|
192
|
-
. *
|
614
|
+
. *Environment variable (`TEBAKO_PREFIX`)*:
|
615
|
+
In the absence of a user-specified option, Tebako looks for an environment
|
616
|
+
variable named `TEBAKO_PREFIX`. If found, its value is used as the root folder.
|
193
617
|
|
618
|
+
. *Default value*:
|
619
|
+
If no prefix is specified and the `TEBAKO_DIR` environment variable is not set,
|
620
|
+
Tebako defaults to using a directory named `.tebako` in the user's home
|
621
|
+
directory.
|
194
622
|
|
195
|
-
|
196
|
-
|
623
|
+
|
624
|
+
Path Expansion: Regardless of the method used to set the Tebako prefix, Tebako
|
625
|
+
expands the provided path to an absolute path. This expansion includes resolving
|
626
|
+
relative paths based on the current working directory and expanding user
|
627
|
+
directory shortcuts like `~`.
|
628
|
+
|
629
|
+
|
630
|
+
=== Commands
|
631
|
+
|
632
|
+
Tebako provides several commands to manage the packaging and deployment process.
|
197
633
|
|
198
634
|
==== Press
|
199
635
|
|
200
|
-
This command "presses" a Ruby project using the Tebako
|
201
|
-
folder (`<tebako-root-folder>`).
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
208
|
-
|
636
|
+
This command "presses" a Ruby project using the Tebako components built in the Tebako
|
637
|
+
root folder (`<tebako-root-folder>`).
|
638
|
+
|
639
|
+
|
640
|
+
[NOTE]
|
641
|
+
====
|
642
|
+
The first invocation of the `press` command can take up to an hour as it sets up
|
643
|
+
the packaging environment and collects the required dependencies. Subsequent
|
644
|
+
invocations are much faster.
|
645
|
+
====
|
646
|
+
|
647
|
+
Upon the next invocation tebako will use previously created packaging
|
648
|
+
environment. The press process itself takes minutes.
|
649
|
+
|
650
|
+
You can manage setup of packaging environment manually; please refer to
|
651
|
+
description of setup and clean commands below.
|
209
652
|
|
210
|
-
[source]
|
653
|
+
[source,sh]
|
211
654
|
----
|
212
|
-
tebako press
|
655
|
+
tebako press \
|
656
|
+
-e|--entry-point=<entry-point> \
|
657
|
+
-r|--root=<project-root-folder> \
|
213
658
|
[-p|--prefix=<tebako-root-folder>] \
|
214
|
-
[-R|--Ruby=<
|
215
|
-
-
|
216
|
-
-e|--entry-point=<entry-point> \
|
217
|
-
[-o|--output=<packaged file name>] \
|
659
|
+
[-R|--Ruby=<ruby-version>] \
|
660
|
+
[-o|--output=<packaged-file-name>] \
|
218
661
|
[-l|--log-level=<error|warn|debug|trace>] \
|
219
662
|
[-D|--devmode]
|
220
663
|
----
|
221
664
|
|
222
665
|
Where:
|
223
666
|
|
224
|
-
|
667
|
+
`<tebako-root-folder>`::
|
668
|
+
the Tebako root folder (see details in the Tebako Root Folder Selection section)
|
225
669
|
|
226
|
-
|
670
|
+
`Ruby`::
|
671
|
+
This parameter defines Ruby version that will be packaged (optional, defaults to
|
672
|
+
`3.1.6`)
|
227
673
|
|
228
|
-
|
229
|
-
are located
|
674
|
+
`<project-root>`::
|
675
|
+
a folder at the host source file system where project files are located
|
230
676
|
|
231
|
-
|
232
|
-
|
677
|
+
`<entry-point>`::
|
678
|
+
an executable file (binary executable or script) that shall be started when
|
679
|
+
packaged file is called
|
233
680
|
|
234
|
-
|
235
|
-
point base name
|
681
|
+
`output`::
|
682
|
+
the output file name (optional, defaults to `<current folder>/<entry point base name>`)
|
236
683
|
|
237
|
-
|
684
|
+
`log-level`::
|
685
|
+
logging level for the Tebako built-in memory filesystem driver
|
686
|
+
(optional, defaults to `error`)
|
238
687
|
|
239
|
-
|
240
|
-
|
688
|
+
`devmode`:: flag that activates development mode, in which Tebako's cache and
|
689
|
+
packaging consistency checks are relaxed.
|
690
|
+
+
|
691
|
+
NOTE: Development mode is *not intended for production use* and should only be
|
692
|
+
used during development.
|
241
693
|
|
242
694
|
[example]
|
243
695
|
====
|
@@ -252,47 +704,54 @@ tebako press \
|
|
252
704
|
|
253
705
|
==== Setup
|
254
706
|
|
255
|
-
|
256
|
-
|
707
|
+
This command sets up the Tebako packaging environment.
|
708
|
+
|
709
|
+
Collects required packages, builds the and creates packaging environment. This
|
710
|
+
is a lengthy task that can take significant time, up to 1 hour.
|
711
|
+
|
257
712
|
Tebako supports several configurations at a single system given that their root
|
258
|
-
directories differ and
|
713
|
+
directories differ and multiple Ruby versions within single configuration
|
714
|
+
|
715
|
+
This command is optional, tebako creates packaging environment automatically
|
716
|
+
upon the first invocation of press command.
|
259
717
|
|
260
|
-
|
261
|
-
|
262
|
-
|
263
|
-
based on ```tebako setup``` output. Building cache based on ```tebako press``` may create inconsistent environment upon restore.
|
718
|
+
However, if you plan to use tebako in CI/CD environment with caching it is
|
719
|
+
highly recommended to build cache based on `tebako setup` output. Building cache
|
720
|
+
based on `tebako press` may create inconsistent environment upon restore.
|
264
721
|
|
265
|
-
[source]
|
722
|
+
[source,sh]
|
266
723
|
----
|
267
|
-
tebako setup
|
268
|
-
[-p
|
269
|
-
[-R
|
270
|
-
[-D
|
724
|
+
$ tebako setup \
|
725
|
+
[-p|--prefix=<tebako-root-folder>] \
|
726
|
+
[-R|--Ruby=<ruby-version>] \
|
727
|
+
[-D|--devmode]
|
271
728
|
----
|
272
729
|
|
273
730
|
Where:
|
274
731
|
|
275
|
-
|
732
|
+
`<tebako-root-folder>`:: the Tebako root folder (see details in the Tebako Root Folder Selection section)
|
276
733
|
|
277
|
-
|
734
|
+
`Ruby` parameter defines Ruby version that will be packaged (optional, defaults to 3.1.6)
|
278
735
|
|
279
|
-
|
736
|
+
`devmode` flag activates development mode, in which Tebako's cache and packaging consistency checks are relaxed.
|
280
737
|
Please note that this mode is not intended for production use and should only be used during development.
|
281
738
|
|
282
739
|
==== Clean
|
283
740
|
|
284
|
-
This command
|
741
|
+
This command cleans up all Tebako artifacts in the specified prefix directory.
|
742
|
+
|
743
|
+
NOTE: These artifacts are created by the `setup` and `press` commands.
|
285
744
|
Normally you do not need to do it since tebako packager optimizes artifacts lifecycle on its own.
|
286
745
|
|
287
|
-
[source]
|
746
|
+
[source,sh]
|
288
747
|
----
|
289
|
-
tebako clean \
|
748
|
+
$ tebako clean \
|
290
749
|
[-p|--prefix=<tebako-root-folder>]
|
291
750
|
----
|
292
751
|
|
293
752
|
Where:
|
294
753
|
|
295
|
-
|
754
|
+
`<tebako-root-folder>`:: the Tebako root folder (see details in the Tebako Root Folder Selection section)
|
296
755
|
|
297
756
|
[example]
|
298
757
|
====
|
@@ -302,23 +761,32 @@ tebako clean --prefix='~/.tebako'
|
|
302
761
|
----
|
303
762
|
====
|
304
763
|
|
305
|
-
==== Clean ruby
|
306
764
|
|
307
|
-
|
308
|
-
|
765
|
+
==== Clean Ruby
|
766
|
+
|
767
|
+
This command cleans up only the Ruby artifacts from the specified prefix
|
768
|
+
directory.
|
309
769
|
|
310
|
-
|
770
|
+
NOTE: These artifacts are created by the `setup` and `press` commands.
|
771
|
+
Normally you do not need to do it, since Tebako packager optimizes artifacts
|
772
|
+
lifecycle on its own.
|
773
|
+
|
774
|
+
NOTE: Compiled DwarFS libraries are not cleaned.
|
775
|
+
|
776
|
+
[source,sh]
|
311
777
|
----
|
312
|
-
tebako clean_ruby
|
313
|
-
[-p|--prefix=<tebako-root-folder>]
|
314
|
-
[-R|--Ruby=<
|
778
|
+
$ tebako clean_ruby
|
779
|
+
[-p|--prefix=<tebako-root-folder>] \
|
780
|
+
[-R|--Ruby=<ruby-version>]
|
315
781
|
----
|
316
782
|
|
317
783
|
Where:
|
318
784
|
|
319
|
-
|
320
|
-
folder)
|
321
|
-
|
785
|
+
`<tebako-root-folder>`::
|
786
|
+
the Tebako setup folder (optional, defaults to current folder)
|
787
|
+
|
788
|
+
`Ruby`::
|
789
|
+
defines Ruby version that will cleaned (optional, cleans all versions by default)
|
322
790
|
|
323
791
|
[example]
|
324
792
|
====
|
@@ -329,45 +797,30 @@ tebako clean_ruby --prefix='~/.tebako'
|
|
329
797
|
====
|
330
798
|
|
331
799
|
==== Build script hash
|
332
|
-
Hash command will calculate tebako script hash that may be used as a cache key in CI/CD environment like GitHub Actions
|
333
800
|
|
334
|
-
|
801
|
+
This command outputs a hash value for the Tebako build script, which can be used
|
802
|
+
as a cache key in CI/CD pipelines.
|
803
|
+
|
804
|
+
[source,sh]
|
335
805
|
----
|
336
|
-
tebako hash
|
806
|
+
$ tebako hash
|
337
807
|
----
|
338
808
|
|
339
|
-
=== Exit codes
|
340
|
-
|
341
|
-
[cols,"a,a"]
|
342
|
-
|===
|
343
|
-
| Code | Condition
|
344
809
|
|
345
|
-
|
346
|
-
| 1 | Invalid command line
|
347
|
-
| 101 | `tebako setup` failed at configuration step
|
348
|
-
| 102 | `tebako setup` failed at build step
|
349
|
-
| 103 | `tebako press` failed at configuration step
|
350
|
-
| 104 | `tebako press` failed at build step
|
351
|
-
| 253 | Unsupported Ruby version
|
352
|
-
| 254 | Unsupported operating systems
|
353
|
-
| 255 | Internal error
|
354
|
-
|
355
|
-
|===
|
810
|
+
=== Image extraction
|
356
811
|
|
357
|
-
|
358
|
-
|
359
|
-
Tebako provides an option to an extract filesystem from a package to local
|
360
|
-
folder for verification or execution.
|
812
|
+
Tebako provides an option to an extract its DwarFS filesystem from a package to
|
813
|
+
a local folder for verification or execution.
|
361
814
|
|
362
815
|
[source,sh]
|
363
816
|
----
|
364
|
-
<tebako-packaged-executable> --tebako-extract [<root folder for extracted filesystem>]
|
817
|
+
$ <tebako-packaged-executable> --tebako-extract [<root folder for extracted filesystem>]
|
365
818
|
----
|
366
819
|
|
367
820
|
Where,
|
368
821
|
|
369
|
-
|
370
|
-
|
822
|
+
`<root folder for extracted filesystem>`::
|
823
|
+
The root folder for the extracted filesystem (optional, defaults to `source_filesystem`)
|
371
824
|
|
372
825
|
[example]
|
373
826
|
====
|
@@ -387,47 +840,67 @@ require 'fileutils'
|
|
387
840
|
FileUtils.copy_entry '<in-memory filesystem root>', ARGV[2] || 'source_filesystem'
|
388
841
|
----
|
389
842
|
|
390
|
-
|
843
|
+
=== Exit codes
|
391
844
|
|
392
|
-
|
393
|
-
|
845
|
+
The Tebako CLI exits with different exit codes to indicate the status of the
|
846
|
+
operation. The following table lists the possible exit codes and their meanings.
|
394
847
|
|
395
|
-
|
396
|
-
|
848
|
+
.Tebako CLI exit codes
|
849
|
+
[cols="a,a"]
|
850
|
+
|===
|
851
|
+
| Code | Condition
|
397
852
|
|
398
|
-
|
399
|
-
|
853
|
+
| 0 | No error
|
854
|
+
| 1 | Invalid command line
|
855
|
+
| 101 | `tebako setup` failed at configuration step
|
856
|
+
| 102 | `tebako setup` failed at build step
|
857
|
+
| 103 | `tebako press` failed at configuration step
|
858
|
+
| 104 | `tebako press` failed at build step
|
859
|
+
| 253 | Unsupported Ruby version
|
860
|
+
| 254 | Unsupported operating systems
|
861
|
+
| 255 | Internal error
|
400
862
|
|
401
|
-
[cols="a,a,a,a"]
|
402
863
|
|===
|
403
|
-
| Scenario | `*.gemspec` | `Gemfile` | `*.gem`
|
404
864
|
|
405
|
-
| 1 | No | No | No
|
406
|
-
| 2 | No | No | One
|
407
|
-
| 3 | One | No | Any
|
408
|
-
| 4 | One | One | Any
|
409
|
-
| 5 | No | One | Any
|
410
|
-
| Error | No | No |Two or more
|
411
|
-
| Error |Two or more| Any | Any
|
412
865
|
|
413
|
-
|
866
|
+
== Packaging scenarios with Ruby
|
867
|
+
|
868
|
+
Tebako for Ruby supports the following packaging scenarios.
|
869
|
+
|
870
|
+
This is high-level description of the Tebako Ruby packaging mechanism.
|
871
|
+
|
872
|
+
NOTE: These scenarios were inspired by the `ruby-packer` approach.
|
414
873
|
|
415
|
-
|
416
|
-
|
874
|
+
NOTE: Tebako Ruby is created independently from `ruby-packer`, no line of code
|
875
|
+
was copied from `ruby-packer`.
|
417
876
|
|
418
|
-
|
877
|
+
Depending on the configuration files that are present in the root project folder, the Tebako Ruby packager supports different packaging scenarios.
|
878
|
+
|
879
|
+
These scenarios differ in what files are packaged and where the entry point is located.
|
880
|
+
|
881
|
+
Here is a summary of the scenarios:
|
882
|
+
|
883
|
+
[cols="a,2a,4a,3a,a,a,a"]
|
419
884
|
|===
|
420
|
-
| Scenario | Description | Packaging | Entry point
|
885
|
+
| Scenario | Description | Packaging | Entry point | `*.gemspec` | `Gemfile` | `*.gem`
|
421
886
|
|
422
887
|
| 1
|
423
888
|
| Simple ruby script
|
424
889
|
| Copy `<project-root>` with all sub-folders to packaged filesystem
|
425
890
|
| `<mount_point>/local/<entry_point base name>`
|
891
|
+
| No
|
892
|
+
| No
|
893
|
+
| No
|
894
|
+
|
426
895
|
|
427
896
|
| 2
|
428
897
|
| Packaged gem
|
429
898
|
| Install the gem with `gem install` to packaged filesystem
|
430
899
|
| `<mount_point>/bin/<entry_point base name>` (i.e., binstub is expected)
|
900
|
+
| No
|
901
|
+
| No
|
902
|
+
| One
|
903
|
+
|
431
904
|
|
432
905
|
| 3
|
433
906
|
| Gem source, no `bundler`
|
@@ -436,6 +909,10 @@ located, as follows:
|
|
436
909
|
. Install it with `gem install` to packaged filesystem
|
437
910
|
|
438
911
|
| `<mount_point>/bin/<entry_point base name>` (i.e., binstub is expected)
|
912
|
+
| One
|
913
|
+
| No
|
914
|
+
| Any
|
915
|
+
|
439
916
|
|
440
917
|
| 4
|
441
918
|
| Gem source, `bundler`
|
@@ -445,15 +922,43 @@ located, as follows:
|
|
445
922
|
. Install it with `gem install` to packaged file system
|
446
923
|
|
447
924
|
| `<mount_point>/bin/<entry_point base name>` (i.e., binstub is expected)
|
925
|
+
| One
|
926
|
+
| One
|
927
|
+
| Any
|
928
|
+
|
448
929
|
|
449
930
|
| 5
|
450
931
|
| Rails project
|
451
932
|
| Deploy project to packaged filesystem using `bundle install`
|
452
933
|
| `<mount_point>/local/<entry_point base name>`
|
934
|
+
| No
|
935
|
+
| One
|
936
|
+
| Any
|
937
|
+
|
938
|
+
|
939
|
+
| Error
|
940
|
+
| Error: Two or more `*.gem` files present
|
941
|
+
| -
|
942
|
+
| -
|
943
|
+
| No
|
944
|
+
| No
|
945
|
+
| Two or more
|
946
|
+
|
947
|
+
|
948
|
+
| Error
|
949
|
+
| Error: Two or more `*.gemspec` files present
|
950
|
+
| -
|
951
|
+
| -
|
952
|
+
| Two or more
|
953
|
+
| Any
|
954
|
+
| Any
|
453
955
|
|
454
956
|
|===
|
455
957
|
|
456
958
|
|
959
|
+
These scenarios determine how the project is packaged and where the entry point is located within the packaged filesystem.
|
960
|
+
|
961
|
+
|
457
962
|
== Trivia: origin of name
|
458
963
|
|
459
964
|
"tamatebako" (玉手箱) is the treasure box given to Urashima Taro in the Ryugu,
|
@@ -466,3 +971,14 @@ the said treasure box (storing gems inside a treasure box).
|
|
466
971
|
|
467
972
|
Since "tamatebako" is rather long for the non-Japanese speaker, we use "tebako"
|
468
973
|
(手箱, also "tehako") instead, the generic term for a personal box.
|
974
|
+
|
975
|
+
== Contributing
|
976
|
+
|
977
|
+
We welcome contributions! Please see our contributing guidelines for more
|
978
|
+
information.
|
979
|
+
|
980
|
+
== License
|
981
|
+
|
982
|
+
Copyright Ribose. All rights reserved.
|
983
|
+
|
984
|
+
Tebako is released under the BSD 2-Clause License. See the LICENSE file for details.
|