kettle-family 1.2.89 → 1.2.91
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +37 -1
- data/README.md +44 -7
- data/lib/kettle/family/appraisal_bootstrap.rb +2 -1
- data/lib/kettle/family/cli.rb +1 -1
- data/lib/kettle/family/command_runner.rb +4 -4
- data/lib/kettle/family/concurrency.rb +41 -0
- data/lib/kettle/family/config.rb +18 -0
- data/lib/kettle/family/local_install.rb +3 -2
- data/lib/kettle/family/release_state_check.rb +3 -4
- data/lib/kettle/family/version.rb +1 -1
- data/lib/kettle/family/workflow.rb +547 -63
- data/lib/kettle/family.rb +1 -0
- data.tar.gz.sig +0 -0
- metadata +5 -4
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e63a81da6b871ee1c27d813b23f7840e7c0e0a577e3a992076922ff2750574b3
|
|
4
|
+
data.tar.gz: 859158cd1aa7b293126edb628f02eb8bc641a0fcd4797a6f097af9fe46a9ef27
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: ee75302308856eb21755bfdc3a598fe9e0d3992b4a493c84825e891dbd76f514f976549fde093de80bfe55d822567d36cba2375a666c1ca2fc6b4a9844f8ae5c
|
|
7
|
+
data.tar.gz: 3845dff951350adc5177e8ea3bb4ce9b80a155a4a6fbb2b05618acb2363e77e9e8ce158129521f03ca8d02816b8bdfd1f1c5ec80507c982a0546e95314815068
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/CHANGELOG.md
CHANGED
|
@@ -30,6 +30,38 @@ Please file a bug if you notice a violation of semantic versioning.
|
|
|
30
30
|
|
|
31
31
|
### Security
|
|
32
32
|
|
|
33
|
+
## [1.2.91] - 2026-09-06
|
|
34
|
+
|
|
35
|
+
- TAG: [v1.2.91][1.2.91t]
|
|
36
|
+
- COVERAGE: 93.58% -- 6109/6528 lines in 35 files
|
|
37
|
+
- BRANCH COVERAGE: 78.06% -- 2483/3181 branches in 35 files
|
|
38
|
+
- 32.41% documented
|
|
39
|
+
|
|
40
|
+
### Fixed
|
|
41
|
+
|
|
42
|
+
- Trust Mise configurations in disposable worktrees before member commands run.
|
|
43
|
+
|
|
44
|
+
- Remove retired appraisal declarations cleanly from files without a trailing newline during template bootstrap.
|
|
45
|
+
|
|
46
|
+
## [1.2.90] - 2026-09-06
|
|
47
|
+
|
|
48
|
+
- TAG: [v1.2.90][1.2.90t]
|
|
49
|
+
- COVERAGE: 93.13% -- 6048/6494 lines in 35 files
|
|
50
|
+
- BRANCH COVERAGE: 76.05% -- 2407/3165 branches in 35 files
|
|
51
|
+
- 32.41% documented
|
|
52
|
+
|
|
53
|
+
### Added
|
|
54
|
+
|
|
55
|
+
- Run sibling test workflows concurrently with dynamic TurboTests2 process budgets.
|
|
56
|
+
|
|
57
|
+
- Support aggregate and disposable-worktree monorepo test execution modes.
|
|
58
|
+
|
|
59
|
+
### Fixed
|
|
60
|
+
|
|
61
|
+
- Template ready monorepo sibling gems concurrently in isolated worktrees while materializing only member-scoped changes.
|
|
62
|
+
|
|
63
|
+
- Derive family command concurrency from the executing CPU count and pass template wave width to nested tools.
|
|
64
|
+
|
|
33
65
|
## [1.2.89] - 2026-09-06
|
|
34
66
|
|
|
35
67
|
- TAG: [v1.2.89][1.2.89t]
|
|
@@ -2573,7 +2605,11 @@ Please file a bug if you notice a violation of semantic versioning.
|
|
|
2573
2605
|
- Fixed CI load failures on engines without compatible `pty` support by falling back to Open3 for interactive release commands.
|
|
2574
2606
|
- Fixed Ruby 3.2 version-bump support by loading Prism lazily and wiring the Prism gem only for MRI versions that need it.
|
|
2575
2607
|
|
|
2576
|
-
[Unreleased]: https://github.com/kettle-dev/kettle-family/compare/v1.2.
|
|
2608
|
+
[Unreleased]: https://github.com/kettle-dev/kettle-family/compare/v1.2.91...HEAD
|
|
2609
|
+
[1.2.91]: https://github.com/kettle-dev/kettle-family/compare/v1.2.90...v1.2.91
|
|
2610
|
+
[1.2.91t]: https://github.com/kettle-dev/kettle-family/releases/tag/v1.2.91
|
|
2611
|
+
[1.2.90]: https://github.com/kettle-dev/kettle-family/compare/v1.2.89...v1.2.90
|
|
2612
|
+
[1.2.90t]: https://github.com/kettle-dev/kettle-family/releases/tag/v1.2.90
|
|
2577
2613
|
[1.2.89]: https://github.com/kettle-dev/kettle-family/compare/v1.2.88...v1.2.89
|
|
2578
2614
|
[1.2.89t]: https://github.com/kettle-dev/kettle-family/releases/tag/v1.2.89
|
|
2579
2615
|
[1.2.88]: https://github.com/kettle-dev/kettle-family/compare/v1.2.87...v1.2.88
|
data/README.md
CHANGED
|
@@ -281,6 +281,37 @@ canonical development lockfile and CI checkout share that path graph. In both
|
|
|
281
281
|
cases, release normalization rejects every path outside the configured allow
|
|
282
282
|
list.
|
|
283
283
|
|
|
284
|
+
### Test concurrency
|
|
285
|
+
|
|
286
|
+
`kettle-family test --execute` runs independent sibling repositories in
|
|
287
|
+
parallel. Its default outer width is half of the CPUs detected on the current
|
|
288
|
+
machine, capped by the number of selected members; `--jobs` and `test.jobs`
|
|
289
|
+
override that width. The same budget is passed to member `kettle-release`
|
|
290
|
+
commands, because their local test phase uses the same process pool.
|
|
291
|
+
|
|
292
|
+
Family sets `TURBO_TESTS2_MAX_PROCESSES` for each active wave. This is an
|
|
293
|
+
inclusive ceiling for TurboTests2: it includes the member's primary test
|
|
294
|
+
process, not merely additional workers. Given `C` detected CPUs and `W` active
|
|
295
|
+
Family wave slots, the ceiling is `min(floor(C / 2), 1 + floor((C - W) / W))`.
|
|
296
|
+
An explicit `turbo_tests2 --count`/`--workers` value, or an explicitly supplied
|
|
297
|
+
`TURBO_TESTS2_MAX_PROCESSES`, takes precedence.
|
|
298
|
+
|
|
299
|
+
Monorepos default to `test.monorepo_mode: aggregate`: one selected member runs
|
|
300
|
+
the shared root suite and the remaining selected subgems are reported as
|
|
301
|
+
skipped. Set `test.aggregate_member` to choose that owner when it is selected.
|
|
302
|
+
For monorepos with genuinely member-scoped suites, opt in to disposable
|
|
303
|
+
worktrees instead:
|
|
304
|
+
|
|
305
|
+
```yaml
|
|
306
|
+
test:
|
|
307
|
+
monorepo_mode: worktrees
|
|
308
|
+
jobs: 4
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
Those worktrees isolate coverage, temporary files, and Bundler state. Their
|
|
312
|
+
changes are always discarded; test execution never materializes or commits
|
|
313
|
+
worktree artifacts.
|
|
314
|
+
|
|
284
315
|
### Release waves and dependency refresh
|
|
285
316
|
|
|
286
317
|
Release waves are both a sequencing mechanism and a dependency propagation
|
|
@@ -533,9 +564,9 @@ stops before family changelog generation or coverage and reports the members
|
|
|
533
564
|
that need a version bump. Run `kettle-family bump --execute patch` first, then
|
|
534
565
|
rerun the release.
|
|
535
566
|
|
|
536
|
-
State checks run concurrently by default, using up to
|
|
537
|
-
|
|
538
|
-
limit; results remain in family order.
|
|
567
|
+
State checks run concurrently by default, using up to half of the available CPU
|
|
568
|
+
cores and no more than the selected member count. Use `--jobs N` to set an
|
|
569
|
+
explicit limit; results remain in family order.
|
|
539
570
|
|
|
540
571
|
Most commands accept `--json` to print a machine-readable report instead of the
|
|
541
572
|
text report, and `--report PATH` to write that JSON report while still printing
|
|
@@ -732,9 +763,15 @@ kettle-family bump --execute --only unreleased minor
|
|
|
732
763
|
```
|
|
733
764
|
|
|
734
765
|
Run `kettle-jem` templating across the selected family members. Planning shows
|
|
735
|
-
the commands that would run; `--execute` runs them. Executed templating
|
|
736
|
-
|
|
737
|
-
|
|
766
|
+
the commands that would run; `--execute` runs them. Executed templating uses a
|
|
767
|
+
rolling outer wave of up to half the available CPU cores, bounded by ready
|
|
768
|
+
members; `--jobs` overrides that width.
|
|
769
|
+
|
|
770
|
+
For a Kettle Jem member command, Family exports the active outer width as
|
|
771
|
+
`KETTLE_FAMILY_WAVE_JOBS`. Kettle Jem retains the member's base process and
|
|
772
|
+
uses an equal share of CPUs left by that wave for its recipe and file worker
|
|
773
|
+
pools, capped at half the available CPUs. Explicit `KETTLE_JEM_*_WORKERS`
|
|
774
|
+
settings take precedence over this derived budget.
|
|
738
775
|
|
|
739
776
|
```console
|
|
740
777
|
kettle-family template
|
|
@@ -1084,7 +1121,7 @@ Thanks for RTFM. ☺️
|
|
|
1084
1121
|
[📌gitmoji]: https://gitmoji.dev
|
|
1085
1122
|
[📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
|
|
1086
1123
|
[🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
|
|
1087
|
-
[🧮kloc-img]: https://img.shields.io/badge/KLOC-6.
|
|
1124
|
+
[🧮kloc-img]: https://img.shields.io/badge/KLOC-6.528-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
|
|
1088
1125
|
[🔐security]: https://github.com/kettle-dev/kettle-family/blob/main/SECURITY.md
|
|
1089
1126
|
[🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
|
|
1090
1127
|
[📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year
|
|
@@ -66,7 +66,8 @@ module Kettle
|
|
|
66
66
|
def obsolete_appraisal_edit(path, source, node)
|
|
67
67
|
return unless legacy_appraisal_call?(node)
|
|
68
68
|
|
|
69
|
-
|
|
69
|
+
preceding_newline = source.rindex("\n", node.location.start_offset - 1)
|
|
70
|
+
line_start = preceding_newline ? preceding_newline + 1 : 0
|
|
70
71
|
line_end = source.index("\n", node.location.end_offset) || source.length
|
|
71
72
|
line_end += 1 if line_end < source.length
|
|
72
73
|
if final_standalone_call?(source, node, line_start, line_end)
|
data/lib/kettle/family/cli.rb
CHANGED
|
@@ -910,7 +910,7 @@ module Kettle
|
|
|
910
910
|
io: stdout,
|
|
911
911
|
label: "release state",
|
|
912
912
|
total: members.length,
|
|
913
|
-
jobs:
|
|
913
|
+
jobs: Concurrency.wave_jobs(requested: options[:jobs], item_count: members.length),
|
|
914
914
|
members: members,
|
|
915
915
|
heading: "release state #{members.length} member#{"s" unless members.length == 1}:"
|
|
916
916
|
)
|
|
@@ -230,8 +230,8 @@ module Kettle
|
|
|
230
230
|
@otp_coordinator = otp_coordinator
|
|
231
231
|
end
|
|
232
232
|
|
|
233
|
-
def call(member:, phase:, command:, env: {}, interactive: false, stdout_line_handler: nil, log_path: nil, passthrough_output: true)
|
|
234
|
-
argv = command_argv(member: member, command: command, env: env)
|
|
233
|
+
def call(member:, phase:, command:, env: {}, interactive: false, raw: false, stdout_line_handler: nil, log_path: nil, passthrough_output: true)
|
|
234
|
+
argv = command_argv(member: member, command: command, env: env, raw: raw)
|
|
235
235
|
process_env = process_env(member: member, env: env)
|
|
236
236
|
spawn_options = process_options
|
|
237
237
|
return skipped_result(member: member, phase: phase, argv: argv) unless execute
|
|
@@ -598,9 +598,9 @@ module Kettle
|
|
|
598
598
|
nil
|
|
599
599
|
end
|
|
600
600
|
|
|
601
|
-
def command_argv(member:, command:, env: {})
|
|
601
|
+
def command_argv(member:, command:, env: {}, raw: false)
|
|
602
602
|
argv = normalize_command(command)
|
|
603
|
-
return argv
|
|
603
|
+
return argv if raw || !mise_configured?(member)
|
|
604
604
|
|
|
605
605
|
command_env = nonsensitive_env(env)
|
|
606
606
|
unset_env, set_env = command_env.partition { |_key, value| value.nil? }
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "etc"
|
|
4
|
+
|
|
5
|
+
module Kettle
|
|
6
|
+
module Family
|
|
7
|
+
# Allocates CPU capacity between concurrent family members and optional
|
|
8
|
+
# member-internal workers. A member consumes one wave slot before any
|
|
9
|
+
# command-specific workers are considered.
|
|
10
|
+
module Concurrency
|
|
11
|
+
module_function
|
|
12
|
+
|
|
13
|
+
def default_wave_jobs(cpu_count: Etc.nprocessors)
|
|
14
|
+
[cpu_count / 2, 1].max
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def wave_jobs(requested:, item_count:, cpu_count: Etc.nprocessors)
|
|
18
|
+
return 0 if item_count.zero?
|
|
19
|
+
|
|
20
|
+
count = requested.nil? ? default_wave_jobs(cpu_count: cpu_count) : requested.to_i
|
|
21
|
+
count.clamp(1, item_count)
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def internal_worker_limit(wave_jobs:, cpu_count: Etc.nprocessors)
|
|
25
|
+
raise ArgumentError, "wave jobs must be positive" unless wave_jobs.to_i.positive?
|
|
26
|
+
|
|
27
|
+
half_cores = default_wave_jobs(cpu_count: cpu_count)
|
|
28
|
+
available_per_member = (cpu_count - wave_jobs.to_i) / wave_jobs.to_i
|
|
29
|
+
available_per_member.clamp(1, half_cores)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
# Test runners count their primary worker as part of the requested
|
|
33
|
+
# process pool. Reserve one process for each wave member, then divide
|
|
34
|
+
# the remaining capacity across them. No member receives more than half
|
|
35
|
+
# of the detected CPUs, including that primary process.
|
|
36
|
+
def test_process_ceiling(wave_jobs:, cpu_count: Etc.nprocessors)
|
|
37
|
+
[default_wave_jobs(cpu_count: cpu_count), 1 + internal_worker_limit(wave_jobs: wave_jobs, cpu_count: cpu_count)].min
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
end
|
data/lib/kettle/family/config.rb
CHANGED
|
@@ -208,6 +208,24 @@ module Kettle
|
|
|
208
208
|
fetch_path("template", "jobs")
|
|
209
209
|
end
|
|
210
210
|
|
|
211
|
+
# A monorepo commonly has one root suite that covers every subgem. The
|
|
212
|
+
# configured owner is optional; an --only invocation never expands its
|
|
213
|
+
# own selection solely to reach that owner.
|
|
214
|
+
def test_aggregate_member
|
|
215
|
+
fetch_path("test", "aggregate_member")
|
|
216
|
+
end
|
|
217
|
+
|
|
218
|
+
def test_monorepo_mode
|
|
219
|
+
mode = (fetch_path("test", "monorepo_mode") || "aggregate").to_s
|
|
220
|
+
return mode if %w[aggregate worktrees].include?(mode)
|
|
221
|
+
|
|
222
|
+
raise Error, "test.monorepo_mode must be aggregate or worktrees"
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
def test_jobs
|
|
226
|
+
fetch_path("test", "jobs")
|
|
227
|
+
end
|
|
228
|
+
|
|
211
229
|
def readme_corporate_sponsors
|
|
212
230
|
raw = fetch_path("readme", "corporate_sponsors") ||
|
|
213
231
|
fetch_path("sponsorships", "corporate") ||
|
|
@@ -6,6 +6,8 @@ require "json"
|
|
|
6
6
|
require "open3"
|
|
7
7
|
require "time"
|
|
8
8
|
|
|
9
|
+
require_relative "concurrency"
|
|
10
|
+
|
|
9
11
|
module Kettle
|
|
10
12
|
module Family
|
|
11
13
|
class LocalInstall
|
|
@@ -113,8 +115,7 @@ module Kettle
|
|
|
113
115
|
end
|
|
114
116
|
|
|
115
117
|
def install_jobs(candidate_members)
|
|
116
|
-
|
|
117
|
-
count.clamp(1, candidate_members.length)
|
|
118
|
+
Concurrency.wave_jobs(requested: jobs, item_count: candidate_members.length)
|
|
118
119
|
end
|
|
119
120
|
|
|
120
121
|
def member_from_path(path)
|
|
@@ -9,6 +9,8 @@ require "rubygems"
|
|
|
9
9
|
require "securerandom"
|
|
10
10
|
require "yaml"
|
|
11
11
|
|
|
12
|
+
require_relative "concurrency"
|
|
13
|
+
|
|
12
14
|
module Kettle
|
|
13
15
|
module Family
|
|
14
16
|
class ReleaseStateCheck
|
|
@@ -574,10 +576,7 @@ module Kettle
|
|
|
574
576
|
end
|
|
575
577
|
|
|
576
578
|
def state_jobs(items)
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
requested = jobs || [Etc.nprocessors, 4].min
|
|
580
|
-
requested.to_i.clamp(1, items.length)
|
|
579
|
+
Concurrency.wave_jobs(requested: jobs, item_count: items.length)
|
|
581
580
|
end
|
|
582
581
|
|
|
583
582
|
def parallel_map(items, &block)
|
|
@@ -5,7 +5,7 @@ module Kettle
|
|
|
5
5
|
# Version namespace for this gem.
|
|
6
6
|
module Version
|
|
7
7
|
# Current gem version.
|
|
8
|
-
VERSION = "1.2.
|
|
8
|
+
VERSION = "1.2.91"
|
|
9
9
|
end
|
|
10
10
|
# Current gem version exposed at the traditional constant location.
|
|
11
11
|
VERSION = Version::VERSION # Traditional Constant Location
|
|
@@ -11,6 +11,7 @@ require "shellwords"
|
|
|
11
11
|
require "yaml"
|
|
12
12
|
require "kettle/dev"
|
|
13
13
|
|
|
14
|
+
require_relative "concurrency"
|
|
14
15
|
require_relative "workflow_progress"
|
|
15
16
|
|
|
16
17
|
module Kettle
|
|
@@ -569,6 +570,7 @@ module Kettle
|
|
|
569
570
|
|
|
570
571
|
def member_workflow_results(workflow_members)
|
|
571
572
|
return template_member_workflow_results(workflow_members) if command == "template" && execute
|
|
573
|
+
return test_member_workflow_results(workflow_members) if command == "test" && execute
|
|
572
574
|
|
|
573
575
|
runner = CommandRunner.new(execute: execute, accept: accept)
|
|
574
576
|
results = []
|
|
@@ -614,6 +616,218 @@ module Kettle
|
|
|
614
616
|
results
|
|
615
617
|
end
|
|
616
618
|
|
|
619
|
+
# Sibling repositories have isolated working trees and can run their
|
|
620
|
+
# tests concurrently. A monorepo's default kettle-test command normally
|
|
621
|
+
# executes the shared root suite, so it is represented by one selected
|
|
622
|
+
# owner instead of repeating that suite per subgem.
|
|
623
|
+
def test_member_workflow_results(workflow_members)
|
|
624
|
+
test_members, skipped_members = test_members_for(workflow_members)
|
|
625
|
+
return [] if test_members.empty?
|
|
626
|
+
|
|
627
|
+
return monorepo_test_worktree_results(test_members) if monorepo_test_worktrees?(test_members)
|
|
628
|
+
|
|
629
|
+
wave_jobs = test_jobs(test_members)
|
|
630
|
+
queue = Queue.new
|
|
631
|
+
test_members.each_with_index { |member, index| queue << [index, member] }
|
|
632
|
+
ordered_results = Array.new(test_members.length)
|
|
633
|
+
mutex = Mutex.new
|
|
634
|
+
stop = false
|
|
635
|
+
|
|
636
|
+
Array.new(wave_jobs) do
|
|
637
|
+
Thread.new do # rubocop:disable ThreadSafety/NewThread -- sibling test suites run in isolated project roots.
|
|
638
|
+
runner = CommandRunner.new(execute: execute, accept: accept)
|
|
639
|
+
loop do
|
|
640
|
+
break if mutex.synchronize { stop }
|
|
641
|
+
|
|
642
|
+
index, member = queue.pop(true)
|
|
643
|
+
result = runner.call(
|
|
644
|
+
member: member,
|
|
645
|
+
phase: command,
|
|
646
|
+
command: workflow_command(member),
|
|
647
|
+
env: test_command_env(wave_jobs: wave_jobs)
|
|
648
|
+
)
|
|
649
|
+
mutex.synchronize do
|
|
650
|
+
ordered_results[index] = result
|
|
651
|
+
stop = true unless result.ok?
|
|
652
|
+
end
|
|
653
|
+
rescue ThreadError
|
|
654
|
+
break
|
|
655
|
+
rescue => error
|
|
656
|
+
mutex.synchronize do
|
|
657
|
+
ordered_results[index] = test_worker_error_result(member, error)
|
|
658
|
+
stop = true
|
|
659
|
+
end
|
|
660
|
+
break
|
|
661
|
+
end
|
|
662
|
+
end
|
|
663
|
+
end.each(&:join)
|
|
664
|
+
|
|
665
|
+
results = ordered_results.compact
|
|
666
|
+
return results unless results.all?(&:ok?)
|
|
667
|
+
|
|
668
|
+
results + skipped_members.map { |member| aggregate_test_skip_result(member, owner: test_members.first) }
|
|
669
|
+
end
|
|
670
|
+
|
|
671
|
+
def test_members_for(workflow_members)
|
|
672
|
+
return [workflow_members, []] unless aggregate_monorepo_test?(workflow_members)
|
|
673
|
+
|
|
674
|
+
owner_name = config.test_aggregate_member.to_s.strip
|
|
675
|
+
owner = workflow_members.find { |member| member.name == owner_name } || workflow_members.first
|
|
676
|
+
[[owner], workflow_members - [owner]]
|
|
677
|
+
end
|
|
678
|
+
|
|
679
|
+
def aggregate_monorepo_test?(workflow_members)
|
|
680
|
+
config.family_mode == "monorepo" &&
|
|
681
|
+
config.test_monorepo_mode == "aggregate" &&
|
|
682
|
+
workflow_members.length > 1
|
|
683
|
+
end
|
|
684
|
+
|
|
685
|
+
def monorepo_test_worktrees?(test_members)
|
|
686
|
+
config.family_mode == "monorepo" &&
|
|
687
|
+
config.test_monorepo_mode == "worktrees" &&
|
|
688
|
+
test_members.length > 1 &&
|
|
689
|
+
test_jobs(test_members) > 1
|
|
690
|
+
end
|
|
691
|
+
|
|
692
|
+
# Member-scoped monorepo suites cannot share the primary checkout: test
|
|
693
|
+
# coverage, temporary files, and Bundler state are all mutable. Unlike
|
|
694
|
+
# template worktrees, test worktrees intentionally discard every change.
|
|
695
|
+
def monorepo_test_worktree_results(test_members)
|
|
696
|
+
entries, setup_results = monorepo_test_worktree_entries(test_members)
|
|
697
|
+
return setup_results unless setup_results.all?(&:ok?)
|
|
698
|
+
|
|
699
|
+
run_monorepo_test_worktree_entries(entries).flatten
|
|
700
|
+
ensure
|
|
701
|
+
cleanup_monorepo_test_worktrees(entries || [])
|
|
702
|
+
end
|
|
703
|
+
|
|
704
|
+
def monorepo_test_worktree_entries(test_members)
|
|
705
|
+
runner = command_runner
|
|
706
|
+
control_member = monorepo_template_control_member
|
|
707
|
+
entries = []
|
|
708
|
+
results = []
|
|
709
|
+
test_members.each do |member|
|
|
710
|
+
worktree_root = monorepo_test_worktree_path(member)
|
|
711
|
+
result = runner.call(
|
|
712
|
+
member: control_member,
|
|
713
|
+
phase: "test_member_worktree_add",
|
|
714
|
+
command: ["git", "worktree", "add", "--detach", worktree_root, "HEAD"]
|
|
715
|
+
)
|
|
716
|
+
result.member_name = member.name
|
|
717
|
+
results << result
|
|
718
|
+
break unless result.ok?
|
|
719
|
+
|
|
720
|
+
entry = {
|
|
721
|
+
member: relocate_member_to_worktree(member, worktree_root, source_root: config.root),
|
|
722
|
+
original_member: member,
|
|
723
|
+
worktree_root: worktree_root
|
|
724
|
+
}
|
|
725
|
+
entries << entry
|
|
726
|
+
results.concat(worktree_mise_trust_results(entry, runner: runner, phase: "test_member_worktree_mise_trust"))
|
|
727
|
+
break unless results.last.ok?
|
|
728
|
+
end
|
|
729
|
+
[entries, results]
|
|
730
|
+
end
|
|
731
|
+
|
|
732
|
+
def monorepo_test_worktree_path(member)
|
|
733
|
+
safe_member = member.name.to_s.gsub(/[^A-Za-z0-9_.-]+/, "_")
|
|
734
|
+
File.join(config.root, "tmp", "kettle-family", "test-member-worktrees", safe_member, "#{Process.pid}-#{Time.now.to_i}")
|
|
735
|
+
end
|
|
736
|
+
|
|
737
|
+
def run_monorepo_test_worktree_entries(entries)
|
|
738
|
+
wave_jobs = test_jobs(entries)
|
|
739
|
+
queue = Queue.new
|
|
740
|
+
entries.each_with_index { |entry, index| queue << [index, entry] }
|
|
741
|
+
ordered_results = Array.new(entries.length)
|
|
742
|
+
mutex = Mutex.new
|
|
743
|
+
stop = false
|
|
744
|
+
|
|
745
|
+
Array.new(wave_jobs) do
|
|
746
|
+
Thread.new do # rubocop:disable ThreadSafety/NewThread -- detached worktrees isolate concurrent monorepo test suites.
|
|
747
|
+
loop do
|
|
748
|
+
break if mutex.synchronize { stop }
|
|
749
|
+
|
|
750
|
+
index, entry = queue.pop(true)
|
|
751
|
+
member = entry.fetch(:member)
|
|
752
|
+
result = CommandRunner.new(execute: execute, accept: accept).call(
|
|
753
|
+
member: member,
|
|
754
|
+
phase: command,
|
|
755
|
+
command: workflow_command(member),
|
|
756
|
+
env: test_command_env(wave_jobs: wave_jobs)
|
|
757
|
+
)
|
|
758
|
+
result.member_name = entry.fetch(:original_member).name
|
|
759
|
+
mutex.synchronize do
|
|
760
|
+
ordered_results[index] = [result]
|
|
761
|
+
stop = true unless result.ok?
|
|
762
|
+
end
|
|
763
|
+
rescue ThreadError
|
|
764
|
+
break
|
|
765
|
+
rescue => error
|
|
766
|
+
mutex.synchronize do
|
|
767
|
+
ordered_results[index] = [test_worker_error_result(entry.fetch(:original_member), error)]
|
|
768
|
+
stop = true
|
|
769
|
+
end
|
|
770
|
+
break
|
|
771
|
+
end
|
|
772
|
+
end
|
|
773
|
+
end.each(&:join)
|
|
774
|
+
ordered_results.compact
|
|
775
|
+
end
|
|
776
|
+
|
|
777
|
+
def cleanup_monorepo_test_worktrees(entries)
|
|
778
|
+
runner = command_runner
|
|
779
|
+
control_member = monorepo_template_control_member
|
|
780
|
+
entries.each do |entry|
|
|
781
|
+
root = entry.fetch(:worktree_root)
|
|
782
|
+
next unless Dir.exist?(root)
|
|
783
|
+
|
|
784
|
+
runner.call(member: control_member, phase: "test_member_worktree_remove", command: ["git", "worktree", "remove", "--force", root])
|
|
785
|
+
end
|
|
786
|
+
end
|
|
787
|
+
|
|
788
|
+
def aggregate_test_skip_result(member, owner:)
|
|
789
|
+
CommandResult.new(
|
|
790
|
+
member_name: member.name,
|
|
791
|
+
phase: command,
|
|
792
|
+
command: workflow_command(member),
|
|
793
|
+
workdir: member.root,
|
|
794
|
+
status: 0,
|
|
795
|
+
success: true,
|
|
796
|
+
stdout: "",
|
|
797
|
+
stderr: "",
|
|
798
|
+
elapsed_seconds: 0.0,
|
|
799
|
+
skipped: true,
|
|
800
|
+
reason: "shared monorepo suite owned by #{owner.name}"
|
|
801
|
+
)
|
|
802
|
+
end
|
|
803
|
+
|
|
804
|
+
def test_worker_error_result(member, error)
|
|
805
|
+
CommandResult.new(
|
|
806
|
+
member_name: member.name,
|
|
807
|
+
phase: command,
|
|
808
|
+
command: ["internal", "test-worker"],
|
|
809
|
+
workdir: member.root,
|
|
810
|
+
status: 1,
|
|
811
|
+
success: false,
|
|
812
|
+
stdout: "",
|
|
813
|
+
stderr: "#{error.class}: #{error.message}\n",
|
|
814
|
+
elapsed_seconds: 0.0,
|
|
815
|
+
skipped: false,
|
|
816
|
+
reason: error.message
|
|
817
|
+
)
|
|
818
|
+
end
|
|
819
|
+
|
|
820
|
+
def test_jobs(workflow_members)
|
|
821
|
+
requested = jobs || config.test_jobs
|
|
822
|
+
Concurrency.wave_jobs(requested: requested, item_count: workflow_members.length)
|
|
823
|
+
end
|
|
824
|
+
|
|
825
|
+
def test_command_env(wave_jobs:)
|
|
826
|
+
env = command_env
|
|
827
|
+
env["TURBO_TESTS2_MAX_PROCESSES"] ||= Concurrency.test_process_ceiling(wave_jobs: wave_jobs).to_s
|
|
828
|
+
env
|
|
829
|
+
end
|
|
830
|
+
|
|
617
831
|
def review_gha_sha_pins(workflow_members, runner:, memo:, env: command_env)
|
|
618
832
|
return true if workflow_members.empty?
|
|
619
833
|
|
|
@@ -719,17 +933,20 @@ module Kettle
|
|
|
719
933
|
end
|
|
720
934
|
|
|
721
935
|
def template_results_for_wave(wave, progress:)
|
|
936
|
+
return monorepo_template_worktree_wave_results(wave, progress: progress) if monorepo_template_worktrees?(wave)
|
|
937
|
+
|
|
938
|
+
wave_jobs = template_jobs(wave)
|
|
722
939
|
queue = Queue.new
|
|
723
940
|
wave.each_with_index { |member, index| queue << [index, member] }
|
|
724
941
|
ordered_results = Array.new(wave.length)
|
|
725
942
|
mutex = Mutex.new
|
|
726
943
|
stop = false
|
|
727
|
-
Array.new(
|
|
944
|
+
Array.new(wave_jobs) do
|
|
728
945
|
Thread.new do # rubocop:disable ThreadSafety/NewThread -- family templating intentionally runs independent members concurrently.
|
|
729
946
|
loop do
|
|
730
947
|
break if mutex.synchronize { stop }
|
|
731
948
|
index, member = queue.pop(true)
|
|
732
|
-
member_results = template_results_for_member(member, progress: progress)
|
|
949
|
+
member_results = template_results_for_member(member, progress: progress, wave_jobs: wave_jobs)
|
|
733
950
|
mutex.synchronize do
|
|
734
951
|
ordered_results[index] = member_results
|
|
735
952
|
stop = true unless member_results.all?(&:ok?)
|
|
@@ -751,10 +968,15 @@ module Kettle
|
|
|
751
968
|
ready = remaining.select do |member|
|
|
752
969
|
(member.dependencies & by_name.keys).all? { |name| completed.include?(name) }
|
|
753
970
|
end
|
|
754
|
-
# A
|
|
755
|
-
#
|
|
756
|
-
#
|
|
757
|
-
|
|
971
|
+
# A shared checkout remains a transaction boundary unless each
|
|
972
|
+
# ready monorepo member will receive an isolated worktree. The
|
|
973
|
+
# latter keeps the normal dependency graph intact while allowing
|
|
974
|
+
# independent sibling gems to use the requested worker budget.
|
|
975
|
+
wave = if monorepo_template_worktrees_enabled?
|
|
976
|
+
ready
|
|
977
|
+
else
|
|
978
|
+
ready.group_by { |member| git_root_for(member) }.values.map(&:first)
|
|
979
|
+
end
|
|
758
980
|
raise Error, "template dependency wave could not resolve: #{remaining.map(&:name).join(", ")}" if wave.empty?
|
|
759
981
|
|
|
760
982
|
waves << wave
|
|
@@ -764,6 +986,221 @@ module Kettle
|
|
|
764
986
|
end
|
|
765
987
|
end
|
|
766
988
|
|
|
989
|
+
# A monorepo has one mutable primary checkout, but its ready sibling
|
|
990
|
+
# gems can template independently. Run them in detached worktrees and
|
|
991
|
+
# materialize only changes beneath each member root before the existing
|
|
992
|
+
# serialized commit step runs in the primary checkout.
|
|
993
|
+
def monorepo_template_worktree_wave_results(wave, progress:)
|
|
994
|
+
entries, setup_results = monorepo_template_worktree_entries(wave)
|
|
995
|
+
return setup_results unless setup_results.all?(&:ok?)
|
|
996
|
+
|
|
997
|
+
worker_results = run_monorepo_template_worktree_entries(entries, progress: progress)
|
|
998
|
+
results = []
|
|
999
|
+
worker_results.each_with_index do |member_results, index|
|
|
1000
|
+
next unless member_results
|
|
1001
|
+
|
|
1002
|
+
results.concat(member_results)
|
|
1003
|
+
next unless member_results.all?(&:ok?)
|
|
1004
|
+
|
|
1005
|
+
entry = entries.fetch(index)
|
|
1006
|
+
materialized = materialize_monorepo_template_changes(entry)
|
|
1007
|
+
results << materialized
|
|
1008
|
+
next unless materialized.ok?
|
|
1009
|
+
|
|
1010
|
+
commit_results = []
|
|
1011
|
+
commit_template_changes(member: entry.fetch(:original_member), runner: command_runner, memo: commit_results)
|
|
1012
|
+
results.concat(commit_results)
|
|
1013
|
+
end
|
|
1014
|
+
results
|
|
1015
|
+
ensure
|
|
1016
|
+
cleanup_monorepo_template_worktrees(entries || [])
|
|
1017
|
+
end
|
|
1018
|
+
|
|
1019
|
+
def monorepo_template_worktrees?(wave)
|
|
1020
|
+
monorepo_template_worktrees_enabled? && wave.length > 1
|
|
1021
|
+
end
|
|
1022
|
+
|
|
1023
|
+
def monorepo_template_worktrees_enabled?
|
|
1024
|
+
command_text = config.template_command || DEFAULT_COMMANDS.fetch("template")
|
|
1025
|
+
execute && monorepo_template? && kettle_jem_template_command?(command_text) && members.length > 1 && template_jobs(members) > 1
|
|
1026
|
+
end
|
|
1027
|
+
|
|
1028
|
+
def monorepo_template_worktree_entries(wave)
|
|
1029
|
+
runner = command_runner
|
|
1030
|
+
control_member = monorepo_template_control_member
|
|
1031
|
+
entries = []
|
|
1032
|
+
results = []
|
|
1033
|
+
wave.each do |member|
|
|
1034
|
+
worktree_root = monorepo_template_worktree_path(member)
|
|
1035
|
+
result = runner.call(
|
|
1036
|
+
member: control_member,
|
|
1037
|
+
phase: "template_member_worktree_add",
|
|
1038
|
+
command: ["git", "worktree", "add", "--detach", worktree_root, "HEAD"]
|
|
1039
|
+
)
|
|
1040
|
+
result.member_name = member.name
|
|
1041
|
+
results << result
|
|
1042
|
+
break unless result.ok?
|
|
1043
|
+
|
|
1044
|
+
entry = {
|
|
1045
|
+
member: relocate_member_to_worktree(member, worktree_root, source_root: config.root),
|
|
1046
|
+
original_member: member,
|
|
1047
|
+
worktree_root: worktree_root
|
|
1048
|
+
}
|
|
1049
|
+
entries << entry
|
|
1050
|
+
results.concat(worktree_mise_trust_results(entry, runner: runner, phase: "template_member_worktree_mise_trust"))
|
|
1051
|
+
break unless results.last.ok?
|
|
1052
|
+
end
|
|
1053
|
+
[entries, results]
|
|
1054
|
+
end
|
|
1055
|
+
|
|
1056
|
+
def monorepo_template_worktree_path(member)
|
|
1057
|
+
safe_member = member.name.to_s.gsub(/[^A-Za-z0-9_.-]+/, "_")
|
|
1058
|
+
File.join(config.root, "tmp", "kettle-family", "template-member-worktrees", safe_member, "#{Process.pid}-#{Time.now.to_i}")
|
|
1059
|
+
end
|
|
1060
|
+
|
|
1061
|
+
def monorepo_template_control_member
|
|
1062
|
+
family_member.dup.tap { |member| member.root = config.root }
|
|
1063
|
+
end
|
|
1064
|
+
|
|
1065
|
+
def run_monorepo_template_worktree_entries(entries, progress:)
|
|
1066
|
+
wave_jobs = template_jobs(entries)
|
|
1067
|
+
queue = Queue.new
|
|
1068
|
+
entries.each_with_index { |entry, index| queue << [index, entry] }
|
|
1069
|
+
ordered_results = Array.new(entries.length)
|
|
1070
|
+
mutex = Mutex.new
|
|
1071
|
+
stop = false
|
|
1072
|
+
Array.new(wave_jobs) do
|
|
1073
|
+
Thread.new do # rubocop:disable ThreadSafety/NewThread -- detached worktrees isolate concurrent monorepo template members.
|
|
1074
|
+
loop do
|
|
1075
|
+
break if mutex.synchronize { stop }
|
|
1076
|
+
|
|
1077
|
+
index, entry = queue.pop(true)
|
|
1078
|
+
member_results = template_results_for_member(
|
|
1079
|
+
entry.fetch(:member),
|
|
1080
|
+
progress: progress,
|
|
1081
|
+
commit_changes: false,
|
|
1082
|
+
wave_jobs: wave_jobs
|
|
1083
|
+
)
|
|
1084
|
+
mutex.synchronize do
|
|
1085
|
+
ordered_results[index] = member_results
|
|
1086
|
+
stop = true unless member_results.all?(&:ok?)
|
|
1087
|
+
end
|
|
1088
|
+
rescue ThreadError
|
|
1089
|
+
break
|
|
1090
|
+
rescue => error
|
|
1091
|
+
mutex.synchronize do
|
|
1092
|
+
ordered_results[index] = [template_worktree_failure_result(entry.fetch(:original_member), error)]
|
|
1093
|
+
stop = true
|
|
1094
|
+
end
|
|
1095
|
+
break
|
|
1096
|
+
end
|
|
1097
|
+
end
|
|
1098
|
+
end.each(&:join)
|
|
1099
|
+
ordered_results
|
|
1100
|
+
end
|
|
1101
|
+
|
|
1102
|
+
def materialize_monorepo_template_changes(entry)
|
|
1103
|
+
original_member = entry.fetch(:original_member)
|
|
1104
|
+
worktree_root = entry.fetch(:worktree_root)
|
|
1105
|
+
relative_root = Pathname.new(original_member.root).relative_path_from(Pathname.new(config.root)).to_s
|
|
1106
|
+
paths = monorepo_template_worktree_changed_paths(worktree_root)
|
|
1107
|
+
return template_worktree_failure_result(original_member, paths) if paths.is_a?(String)
|
|
1108
|
+
|
|
1109
|
+
invalid_paths = paths.reject { |path| path == relative_root || path.start_with?("#{relative_root}/") }
|
|
1110
|
+
unless invalid_paths.empty?
|
|
1111
|
+
return template_worktree_failure_result(
|
|
1112
|
+
original_member,
|
|
1113
|
+
"isolated template worker modified shared path(s): #{invalid_paths.sort.join(", ")}; only #{relative_root}/ may be materialized"
|
|
1114
|
+
)
|
|
1115
|
+
end
|
|
1116
|
+
|
|
1117
|
+
tracked_paths = monorepo_template_worktree_tracked_paths(worktree_root)
|
|
1118
|
+
return template_worktree_failure_result(original_member, tracked_paths) if tracked_paths.is_a?(String)
|
|
1119
|
+
|
|
1120
|
+
runner = command_runner
|
|
1121
|
+
control_member = monorepo_template_control_member.dup
|
|
1122
|
+
control_member.name = original_member.name
|
|
1123
|
+
command = if tracked_paths.empty?
|
|
1124
|
+
["true"]
|
|
1125
|
+
else
|
|
1126
|
+
source = Shellwords.escape(worktree_root)
|
|
1127
|
+
scope = Shellwords.escape(relative_root)
|
|
1128
|
+
["sh", "-lc", "git -C #{source} diff --binary HEAD -- #{scope} | git apply --whitespace=nowarn"]
|
|
1129
|
+
end
|
|
1130
|
+
result = runner.call(member: control_member, phase: "template_worktree_materialize", command: command)
|
|
1131
|
+
return result unless result.ok?
|
|
1132
|
+
|
|
1133
|
+
copy_monorepo_template_untracked_paths(worktree_root, config.root, relative_root)
|
|
1134
|
+
result.stdout = "#{paths.length} path(s) materialized from isolated worktree"
|
|
1135
|
+
result
|
|
1136
|
+
rescue => error
|
|
1137
|
+
template_worktree_failure_result(original_member, error)
|
|
1138
|
+
end
|
|
1139
|
+
|
|
1140
|
+
def monorepo_template_worktree_changed_paths(worktree_root)
|
|
1141
|
+
tracked_paths = monorepo_template_worktree_tracked_paths(worktree_root)
|
|
1142
|
+
return tracked_paths if tracked_paths.is_a?(String)
|
|
1143
|
+
|
|
1144
|
+
untracked_paths = monorepo_template_worktree_untracked_paths(worktree_root)
|
|
1145
|
+
return untracked_paths if untracked_paths.is_a?(String)
|
|
1146
|
+
|
|
1147
|
+
(tracked_paths + untracked_paths).uniq
|
|
1148
|
+
end
|
|
1149
|
+
|
|
1150
|
+
def monorepo_template_worktree_tracked_paths(worktree_root)
|
|
1151
|
+
git_worktree_paths(worktree_root, %w[diff --name-only -z HEAD])
|
|
1152
|
+
end
|
|
1153
|
+
|
|
1154
|
+
def monorepo_template_worktree_untracked_paths(worktree_root)
|
|
1155
|
+
git_worktree_paths(worktree_root, %w[ls-files --others --exclude-standard -z])
|
|
1156
|
+
end
|
|
1157
|
+
|
|
1158
|
+
def git_worktree_paths(worktree_root, arguments)
|
|
1159
|
+
stdout, stderr, status = Open3.capture3("git", *arguments, chdir: worktree_root)
|
|
1160
|
+
return stdout.split("\0").reject(&:empty?) if status.success?
|
|
1161
|
+
|
|
1162
|
+
"git #{arguments.join(" ")} failed in #{worktree_root}: #{stderr.strip}"
|
|
1163
|
+
end
|
|
1164
|
+
|
|
1165
|
+
def copy_monorepo_template_untracked_paths(worktree_root, primary_root, relative_root)
|
|
1166
|
+
monorepo_template_worktree_untracked_paths(worktree_root).each do |path|
|
|
1167
|
+
next unless path == relative_root || path.start_with?("#{relative_root}/")
|
|
1168
|
+
|
|
1169
|
+
source = File.join(worktree_root, path)
|
|
1170
|
+
destination = File.join(primary_root, path)
|
|
1171
|
+
FileUtils.mkdir_p(File.dirname(destination))
|
|
1172
|
+
FileUtils.cp_r(source, destination, preserve: true)
|
|
1173
|
+
end
|
|
1174
|
+
end
|
|
1175
|
+
|
|
1176
|
+
def cleanup_monorepo_template_worktrees(entries)
|
|
1177
|
+
runner = command_runner
|
|
1178
|
+
control_member = monorepo_template_control_member
|
|
1179
|
+
entries.each do |entry|
|
|
1180
|
+
root = entry.fetch(:worktree_root)
|
|
1181
|
+
next unless Dir.exist?(root)
|
|
1182
|
+
|
|
1183
|
+
runner.call(member: control_member, phase: "template_member_worktree_remove", command: ["git", "worktree", "remove", "--force", root])
|
|
1184
|
+
end
|
|
1185
|
+
end
|
|
1186
|
+
|
|
1187
|
+
def template_worktree_failure_result(member, error)
|
|
1188
|
+
message = error.is_a?(Exception) ? "#{error.class}: #{error.message}" : error.to_s
|
|
1189
|
+
CommandResult.new(
|
|
1190
|
+
member_name: member.name,
|
|
1191
|
+
phase: "template_member_worktree",
|
|
1192
|
+
command: ["internal", "template-member-worktree"],
|
|
1193
|
+
workdir: member.root,
|
|
1194
|
+
status: 1,
|
|
1195
|
+
success: false,
|
|
1196
|
+
stdout: "",
|
|
1197
|
+
stderr: message,
|
|
1198
|
+
elapsed_seconds: 0.0,
|
|
1199
|
+
skipped: false,
|
|
1200
|
+
reason: "template member worktree failed"
|
|
1201
|
+
)
|
|
1202
|
+
end
|
|
1203
|
+
|
|
767
1204
|
def template_bootstrap_dependency_results(workflow_members)
|
|
768
1205
|
return [] if workflow_members.empty?
|
|
769
1206
|
|
|
@@ -856,17 +1293,23 @@ module Kettle
|
|
|
856
1293
|
)
|
|
857
1294
|
end
|
|
858
1295
|
|
|
859
|
-
def template_results_for_member(member, progress: nil)
|
|
1296
|
+
def template_results_for_member(member, progress: nil, commit_changes: true, wave_jobs: 1)
|
|
860
1297
|
progress&.start_member(member, total: template_phase_total(member), status: template_initial_status(member))
|
|
861
1298
|
runner = CommandRunner.new(execute: execute, accept: accept)
|
|
862
1299
|
[].tap do |memo|
|
|
863
1300
|
if config.normalize_lockfiles?
|
|
864
|
-
normalize_lockfiles(member: member, runner: runner, memo: memo, phase: "prepare_lockfiles")
|
|
1301
|
+
normalize_lockfiles(member: member, runner: runner, memo: memo, phase: "prepare_lockfiles", commit_changes: commit_changes)
|
|
865
1302
|
emit_member_result_progress(member, memo.last, progress: progress)
|
|
866
1303
|
return memo unless memo.last.ok?
|
|
867
1304
|
end
|
|
868
1305
|
|
|
869
|
-
prepared = prepare_template_dependencies(
|
|
1306
|
+
prepared = prepare_template_dependencies(
|
|
1307
|
+
member: member,
|
|
1308
|
+
runner: runner,
|
|
1309
|
+
memo: memo,
|
|
1310
|
+
commit_changes: commit_changes,
|
|
1311
|
+
wave_jobs: wave_jobs
|
|
1312
|
+
)
|
|
870
1313
|
emit_member_result_progress(member, memo.last, progress: progress) if memo.last&.phase == "prepare_template_dependencies"
|
|
871
1314
|
return memo if prepared == false
|
|
872
1315
|
|
|
@@ -874,15 +1317,15 @@ module Kettle
|
|
|
874
1317
|
member: member,
|
|
875
1318
|
phase: command,
|
|
876
1319
|
command: workflow_command(member),
|
|
877
|
-
env: template_command_env,
|
|
1320
|
+
env: template_command_env(wave_jobs: wave_jobs),
|
|
878
1321
|
stdout_line_handler: template_event_line_handler(member, progress: progress)
|
|
879
1322
|
)
|
|
880
1323
|
emit_member_result_progress(member, memo.last, progress: progress)
|
|
881
1324
|
return memo unless memo.last.ok?
|
|
882
1325
|
|
|
883
|
-
normalize_lockfiles(member: member, runner: runner, memo: memo, phase: "normalize_lockfiles")
|
|
1326
|
+
normalize_lockfiles(member: member, runner: runner, memo: memo, phase: "normalize_lockfiles", commit_changes: commit_changes)
|
|
884
1327
|
emit_member_result_progress(member, memo.last, progress: progress)
|
|
885
|
-
commit_template_changes(member: member, runner: runner, memo: memo)
|
|
1328
|
+
commit_template_changes(member: member, runner: runner, memo: memo) if commit_changes
|
|
886
1329
|
emit_member_result_progress(member, memo.last, progress: progress) if memo.last&.phase == "commit_template"
|
|
887
1330
|
ensure
|
|
888
1331
|
template_result = memo.find { |result| result.phase == "template" } || memo.last
|
|
@@ -898,8 +1341,7 @@ module Kettle
|
|
|
898
1341
|
|
|
899
1342
|
def template_jobs(workflow_members)
|
|
900
1343
|
requested = jobs || config.template_jobs
|
|
901
|
-
|
|
902
|
-
count.clamp(1, workflow_members.length)
|
|
1344
|
+
Concurrency.wave_jobs(requested: requested, item_count: workflow_members.length)
|
|
903
1345
|
end
|
|
904
1346
|
|
|
905
1347
|
def check_results(workflow_members)
|
|
@@ -999,7 +1441,11 @@ module Kettle
|
|
|
999
1441
|
outcomes << add
|
|
1000
1442
|
break unless add.ok?
|
|
1001
1443
|
|
|
1002
|
-
|
|
1444
|
+
entry = {branch: branch, member: relocate_member_to_worktree(member, worktree_root), worktree_root: worktree_root}
|
|
1445
|
+
entries << entry
|
|
1446
|
+
trust_results = worktree_mise_trust_results(entry.merge(original_member: member), runner: runner, phase: "template_worktree_mise_trust")
|
|
1447
|
+
outcomes.concat(trust_results)
|
|
1448
|
+
break unless trust_results.all?(&:ok?)
|
|
1003
1449
|
end
|
|
1004
1450
|
|
|
1005
1451
|
if outcomes.all?(&:ok?)
|
|
@@ -1055,18 +1501,58 @@ module Kettle
|
|
|
1055
1501
|
end
|
|
1056
1502
|
end
|
|
1057
1503
|
|
|
1504
|
+
# A git worktree has a distinct absolute path, and Mise deliberately
|
|
1505
|
+
# tracks trust by config path. Trust every active config between the
|
|
1506
|
+
# worktree root and the member command directory before invoking a
|
|
1507
|
+
# command there. This bootstrap must run raw: wrapping it in `mise exec`
|
|
1508
|
+
# would require the trust it is establishing.
|
|
1509
|
+
def worktree_mise_trust_results(entry, runner:, phase:)
|
|
1510
|
+
member = entry.fetch(:member)
|
|
1511
|
+
original_member = entry.fetch(:original_member, member)
|
|
1512
|
+
worktree_root = entry.fetch(:worktree_root)
|
|
1513
|
+
mise_config_roots_in_worktree(worktree_root, member.root).map do |root|
|
|
1514
|
+
result = runner.call(
|
|
1515
|
+
member: member,
|
|
1516
|
+
phase: phase,
|
|
1517
|
+
command: ["mise", "trust", "-C", root],
|
|
1518
|
+
raw: true
|
|
1519
|
+
)
|
|
1520
|
+
result.member_name = original_member.name
|
|
1521
|
+
result
|
|
1522
|
+
end
|
|
1523
|
+
end
|
|
1524
|
+
|
|
1525
|
+
def mise_config_roots_in_worktree(worktree_root, member_root)
|
|
1526
|
+
root_path = Pathname.new(worktree_root).realpath
|
|
1527
|
+
member_path = Pathname.new(member_root).realpath
|
|
1528
|
+
return [] unless member_path.to_s.start_with?("#{root_path}#{File::SEPARATOR}") || member_path == root_path
|
|
1529
|
+
|
|
1530
|
+
current = member_path
|
|
1531
|
+
roots = []
|
|
1532
|
+
loop do
|
|
1533
|
+
roots << current.to_s if %w[mise.toml .mise.toml].any? { |name| File.file?(current.join(name)) }
|
|
1534
|
+
break if current == root_path
|
|
1535
|
+
|
|
1536
|
+
current = current.parent
|
|
1537
|
+
end
|
|
1538
|
+
roots.reverse
|
|
1539
|
+
rescue Errno::ENOENT
|
|
1540
|
+
[]
|
|
1541
|
+
end
|
|
1542
|
+
|
|
1058
1543
|
def template_branch_worktree_entries_results(entries)
|
|
1544
|
+
wave_jobs = template_jobs(entries)
|
|
1059
1545
|
queue = Queue.new
|
|
1060
1546
|
entries.each_with_index { |entry, index| queue << [index, entry] }
|
|
1061
1547
|
ordered_results = Array.new(entries.length)
|
|
1062
1548
|
mutex = Mutex.new
|
|
1063
1549
|
stop = false
|
|
1064
|
-
Array.new(
|
|
1550
|
+
Array.new(wave_jobs) do
|
|
1065
1551
|
Thread.new do # rubocop:disable ThreadSafety/NewThread -- independent branch worktrees are safe to template concurrently.
|
|
1066
1552
|
loop do
|
|
1067
1553
|
break if mutex.synchronize { stop }
|
|
1068
1554
|
index, entry = queue.pop(true)
|
|
1069
|
-
branch_results = template_branch_worktree_entry_results(entry)
|
|
1555
|
+
branch_results = template_branch_worktree_entry_results(entry, wave_jobs: wave_jobs)
|
|
1070
1556
|
mutex.synchronize do
|
|
1071
1557
|
ordered_results[index] = branch_results
|
|
1072
1558
|
stop = true unless branch_results.all?(&:ok?)
|
|
@@ -1079,7 +1565,7 @@ module Kettle
|
|
|
1079
1565
|
ordered_results.compact.flatten
|
|
1080
1566
|
end
|
|
1081
1567
|
|
|
1082
|
-
def template_branch_worktree_entry_results(entry)
|
|
1568
|
+
def template_branch_worktree_entry_results(entry, wave_jobs: 1)
|
|
1083
1569
|
member = entry.fetch(:member)
|
|
1084
1570
|
branch = entry.fetch(:branch)
|
|
1085
1571
|
runner = command_runner
|
|
@@ -1087,7 +1573,7 @@ module Kettle
|
|
|
1087
1573
|
if git_upstream_for(member)
|
|
1088
1574
|
results << runner.call(member: member, phase: "template_branch_sync", command: ["git", "rebase", "@{upstream}"])
|
|
1089
1575
|
end
|
|
1090
|
-
results.concat(template_results_for_member(member)) if results.all?(&:ok?)
|
|
1576
|
+
results.concat(template_results_for_member(member, wave_jobs: wave_jobs)) if results.all?(&:ok?)
|
|
1091
1577
|
tag_branch_results(results, branch)
|
|
1092
1578
|
emit_template_event_line(member, results.all?(&:ok?) ? "." : "F", "branch #{branch}")
|
|
1093
1579
|
results
|
|
@@ -1452,7 +1938,7 @@ module Kettle
|
|
|
1452
1938
|
waves.each_with_index do |wave, index|
|
|
1453
1939
|
results << release_wave_result(wave, index: index, total: waves.length, jobs: 1) if show_wave_markers
|
|
1454
1940
|
wave.each do |member|
|
|
1455
|
-
results.concat(release_results_for_member(member, runner: runner))
|
|
1941
|
+
results.concat(release_results_for_member(member, runner: runner, wave_jobs: 1))
|
|
1456
1942
|
break unless results.last.ok?
|
|
1457
1943
|
|
|
1458
1944
|
remaining_members = ordered_members.drop(ordered_members.index(member) + 1)
|
|
@@ -1481,7 +1967,7 @@ module Kettle
|
|
|
1481
1967
|
begin
|
|
1482
1968
|
waves.each_with_index do |wave, index|
|
|
1483
1969
|
results << release_wave_result(wave, index: index, total: waves.length)
|
|
1484
|
-
wave_results = run_release_wave(wave)
|
|
1970
|
+
wave_results = run_release_wave(wave, wave_jobs: release_jobs(wave))
|
|
1485
1971
|
results.concat(wave_results.flatten)
|
|
1486
1972
|
break unless wave_results.all? { |member_results| member_results.all?(&:ok?) }
|
|
1487
1973
|
|
|
@@ -1498,11 +1984,10 @@ module Kettle
|
|
|
1498
1984
|
results
|
|
1499
1985
|
end
|
|
1500
1986
|
|
|
1501
|
-
def run_release_wave(wave)
|
|
1987
|
+
def run_release_wave(wave, wave_jobs: release_jobs(wave))
|
|
1502
1988
|
queue = Queue.new
|
|
1503
1989
|
wave.each_with_index { |member, index| queue << [index, member] }
|
|
1504
1990
|
ordered_results = Array.new(wave.length)
|
|
1505
|
-
wave_jobs = release_jobs(wave)
|
|
1506
1991
|
mutex = Mutex.new
|
|
1507
1992
|
stop = false
|
|
1508
1993
|
Array.new(wave_jobs) do
|
|
@@ -1512,7 +1997,7 @@ module Kettle
|
|
|
1512
1997
|
break if mutex.synchronize { stop }
|
|
1513
1998
|
|
|
1514
1999
|
index, member = queue.pop(true)
|
|
1515
|
-
member_results = release_results_for_member(member, runner: runner)
|
|
2000
|
+
member_results = release_results_for_member(member, runner: runner, wave_jobs: wave_jobs)
|
|
1516
2001
|
mutex.synchronize do
|
|
1517
2002
|
ordered_results[index] = member_results
|
|
1518
2003
|
stop = true unless member_results.all?(&:ok?)
|
|
@@ -1543,7 +2028,7 @@ module Kettle
|
|
|
1543
2028
|
aggregate_members, independent_members = wave.partition { |member| aggregate_release_member?(member) }
|
|
1544
2029
|
|
|
1545
2030
|
unless aggregate_members.empty?
|
|
1546
|
-
worker_results = run_monorepo_release_wave(aggregate_members)
|
|
2031
|
+
worker_results = run_monorepo_release_wave(aggregate_members, wave_jobs: release_jobs(aggregate_members))
|
|
1547
2032
|
results.concat(worker_results.flatten)
|
|
1548
2033
|
break unless worker_results.all? { |member_results| member_results.all?(&:ok?) }
|
|
1549
2034
|
|
|
@@ -1553,7 +2038,7 @@ module Kettle
|
|
|
1553
2038
|
end
|
|
1554
2039
|
|
|
1555
2040
|
independent_members.each do |member|
|
|
1556
|
-
results.concat(release_results_for_member(member, runner: release_command_runner))
|
|
2041
|
+
results.concat(release_results_for_member(member, runner: release_command_runner, wave_jobs: 1))
|
|
1557
2042
|
break unless results.last.ok?
|
|
1558
2043
|
end
|
|
1559
2044
|
break unless results.last&.ok?
|
|
@@ -1571,13 +2056,13 @@ module Kettle
|
|
|
1571
2056
|
results
|
|
1572
2057
|
end
|
|
1573
2058
|
|
|
1574
|
-
def run_monorepo_release_wave(members)
|
|
2059
|
+
def run_monorepo_release_wave(members, wave_jobs: release_jobs(members))
|
|
1575
2060
|
entries, setup_results = monorepo_release_worktree_entries(members)
|
|
1576
2061
|
unless setup_results.all?(&:ok?)
|
|
1577
2062
|
return setup_results.zip
|
|
1578
2063
|
end
|
|
1579
2064
|
|
|
1580
|
-
worker_results = run_monorepo_release_worktree_entries(entries)
|
|
2065
|
+
worker_results = run_monorepo_release_worktree_entries(entries, wave_jobs: wave_jobs)
|
|
1581
2066
|
materialization_results = materialize_monorepo_release_artifacts(entries, worker_results)
|
|
1582
2067
|
worker_results.each_with_index.map do |member_results, index|
|
|
1583
2068
|
artifact_result = materialization_results[index]
|
|
@@ -1602,7 +2087,10 @@ module Kettle
|
|
|
1602
2087
|
results << result
|
|
1603
2088
|
break unless result.ok?
|
|
1604
2089
|
|
|
1605
|
-
|
|
2090
|
+
entry = {member: relocate_member_to_worktree(member, root, source_root: config.root), original_member: member, worktree_root: root}
|
|
2091
|
+
entries << entry
|
|
2092
|
+
results.concat(worktree_mise_trust_results(entry, runner: runner, phase: "release_worktree_mise_trust"))
|
|
2093
|
+
break unless results.last.ok?
|
|
1606
2094
|
end
|
|
1607
2095
|
[entries, results]
|
|
1608
2096
|
end
|
|
@@ -1616,19 +2104,19 @@ module Kettle
|
|
|
1616
2104
|
family_member.dup.tap { |member| member.root = config.root }
|
|
1617
2105
|
end
|
|
1618
2106
|
|
|
1619
|
-
def run_monorepo_release_worktree_entries(entries)
|
|
2107
|
+
def run_monorepo_release_worktree_entries(entries, wave_jobs: release_jobs(entries))
|
|
1620
2108
|
queue = Queue.new
|
|
1621
2109
|
entries.each_with_index { |entry, index| queue << [index, entry] }
|
|
1622
2110
|
ordered_results = Array.new(entries.length)
|
|
1623
2111
|
mutex = Mutex.new
|
|
1624
2112
|
stop = false
|
|
1625
|
-
Array.new(
|
|
2113
|
+
Array.new(wave_jobs) do
|
|
1626
2114
|
Thread.new do # rubocop:disable ThreadSafety/NewThread -- each member has an isolated detached monorepo worktree.
|
|
1627
2115
|
loop do
|
|
1628
2116
|
break if mutex.synchronize { stop }
|
|
1629
2117
|
|
|
1630
2118
|
index, entry = queue.pop(true)
|
|
1631
|
-
member_results = release_results_for_monorepo_worktree(entry)
|
|
2119
|
+
member_results = release_results_for_monorepo_worktree(entry, wave_jobs: wave_jobs)
|
|
1632
2120
|
mutex.synchronize do
|
|
1633
2121
|
ordered_results[index] = member_results
|
|
1634
2122
|
stop = true unless member_results.all?(&:ok?)
|
|
@@ -1647,7 +2135,7 @@ module Kettle
|
|
|
1647
2135
|
ordered_results.compact
|
|
1648
2136
|
end
|
|
1649
2137
|
|
|
1650
|
-
def release_results_for_monorepo_worktree(entry)
|
|
2138
|
+
def release_results_for_monorepo_worktree(entry, wave_jobs: 1)
|
|
1651
2139
|
member = entry.fetch(:member)
|
|
1652
2140
|
original_member = entry.fetch(:original_member)
|
|
1653
2141
|
progress = @release_progress
|
|
@@ -1668,7 +2156,7 @@ module Kettle
|
|
|
1668
2156
|
member: member,
|
|
1669
2157
|
phase: release_phase,
|
|
1670
2158
|
command: release_command_for(member),
|
|
1671
|
-
env: release_env_for_member(member, family_root: entry.fetch(:worktree_root)).merge("KETTLE_RELEASE_FAMILY_MEMBER_PUBLISH" => "true"),
|
|
2159
|
+
env: release_env_for_member(member, family_root: entry.fetch(:worktree_root), wave_jobs: wave_jobs).merge("KETTLE_RELEASE_FAMILY_MEMBER_PUBLISH" => "true"),
|
|
1672
2160
|
interactive: release_command_interactive?,
|
|
1673
2161
|
stdout_line_handler: release_event_line_handler(original_member, progress: progress, state: release_event_state),
|
|
1674
2162
|
log_path: release_command_log_path(original_member, release_phase),
|
|
@@ -1805,7 +2293,7 @@ module Kettle
|
|
|
1805
2293
|
)
|
|
1806
2294
|
end
|
|
1807
2295
|
|
|
1808
|
-
def release_results_for_member(member, runner:)
|
|
2296
|
+
def release_results_for_member(member, runner:, wave_jobs: 1)
|
|
1809
2297
|
progress = @release_progress
|
|
1810
2298
|
progress&.start_member(member, total: release_phase_total(member), status: "check")
|
|
1811
2299
|
[].tap do |memo|
|
|
@@ -1840,7 +2328,7 @@ module Kettle
|
|
|
1840
2328
|
member: member,
|
|
1841
2329
|
phase: release_phase,
|
|
1842
2330
|
command: release_command_for(member),
|
|
1843
|
-
env: release_env_for_member(member),
|
|
2331
|
+
env: release_env_for_member(member, wave_jobs: wave_jobs),
|
|
1844
2332
|
interactive: release_command_interactive?,
|
|
1845
2333
|
stdout_line_handler: release_event_line_handler(member, progress: progress, state: release_event_state),
|
|
1846
2334
|
log_path: release_command_log_path(member, release_phase),
|
|
@@ -1917,8 +2405,7 @@ module Kettle
|
|
|
1917
2405
|
return 1 if truffleruby?
|
|
1918
2406
|
|
|
1919
2407
|
requested = jobs || config.release_jobs
|
|
1920
|
-
|
|
1921
|
-
count.clamp(1, release_members.length)
|
|
2408
|
+
Concurrency.wave_jobs(requested: requested, item_count: release_members.length)
|
|
1922
2409
|
end
|
|
1923
2410
|
|
|
1924
2411
|
def truffleruby?
|
|
@@ -2963,7 +3450,7 @@ module Kettle
|
|
|
2963
3450
|
# A monorepo member runs kettle-release from its own checkout, while its
|
|
2964
3451
|
# changelog may live at the family root. Pass the shared paths to the
|
|
2965
3452
|
# member release phase as well as to the separate family phase.
|
|
2966
|
-
def release_env_for_member(member, family_root: config.root)
|
|
3453
|
+
def release_env_for_member(member, family_root: config.root, wave_jobs: 1)
|
|
2967
3454
|
env = release_env
|
|
2968
3455
|
env.merge!(release_wave_local_path_env_for(member, family_root: family_root))
|
|
2969
3456
|
env.merge!(release_local_path_policy_env(family_root: family_root))
|
|
@@ -2997,6 +3484,10 @@ module Kettle
|
|
|
2997
3484
|
if aggregate_validation_member?(member)
|
|
2998
3485
|
env["KETTLE_RELEASE_FAMILY_CI_MODE"] = aggregate_validation_member_name?(member) ? "validation" : "member"
|
|
2999
3486
|
end
|
|
3487
|
+
# kettle-release invokes kettle-test as part of its member lifecycle.
|
|
3488
|
+
# Preserve an operator-supplied ceiling, otherwise budget the complete
|
|
3489
|
+
# TurboTests2 process pool for this active Family wave.
|
|
3490
|
+
env["TURBO_TESTS2_MAX_PROCESSES"] ||= Concurrency.test_process_ceiling(wave_jobs: wave_jobs).to_s
|
|
3000
3491
|
return env unless config.shared_changelog?
|
|
3001
3492
|
if config.member_local_changelog?(member)
|
|
3002
3493
|
return env.merge(
|
|
@@ -3592,7 +4083,6 @@ module Kettle
|
|
|
3592
4083
|
env["BUNDLE_DISABLE_CHECKSUM_VALIDATION"] = "true"
|
|
3593
4084
|
env["KETTLE_JEM_TEMPLATE_PROFILE"] = config.template_profile if config.template_profile
|
|
3594
4085
|
env["KJ_REPOSITORY_TOPOLOGY"] = config.template_repository_topology if config.template_repository_topology
|
|
3595
|
-
env["KETTLE_JEM_THREAD_WORKERS"] ||= template_thread_worker_budget.to_s
|
|
3596
4086
|
if monorepo_template?
|
|
3597
4087
|
template_git_lock_path = template_git_commit_lock_path
|
|
3598
4088
|
env["KETTLE_JEM_GIT_LOCK"] = template_git_lock_path
|
|
@@ -4348,7 +4838,7 @@ module Kettle
|
|
|
4348
4838
|
"#{outcomes.fetch(:checksum_hits)} checksum hits#{protected_label}, #{outcomes.fetch(:unchanged)} unchanged, #{changed_label}"
|
|
4349
4839
|
end
|
|
4350
4840
|
|
|
4351
|
-
def normalize_lockfiles(member:, runner:, memo:, phase:)
|
|
4841
|
+
def normalize_lockfiles(member:, runner:, memo:, phase:, commit_changes: true)
|
|
4352
4842
|
return unless config.normalize_lockfiles?
|
|
4353
4843
|
|
|
4354
4844
|
env = template_lockfile_phase?(phase) ? template_prepare_env : workflow_env
|
|
@@ -4384,7 +4874,7 @@ module Kettle
|
|
|
4384
4874
|
)
|
|
4385
4875
|
end
|
|
4386
4876
|
if template_lockfile_phase?(phase) && recoverable_bundle_failure?(result)
|
|
4387
|
-
recover_template_lockfiles(member: member, runner: runner, memo: memo, phase: "#{phase}_recovery")
|
|
4877
|
+
recover_template_lockfiles(member: member, runner: runner, memo: memo, phase: "#{phase}_recovery", commit_changes: commit_changes)
|
|
4388
4878
|
return unless memo.last&.ok?
|
|
4389
4879
|
|
|
4390
4880
|
result = runner.call(
|
|
@@ -4397,7 +4887,7 @@ module Kettle
|
|
|
4397
4887
|
memo << result
|
|
4398
4888
|
end
|
|
4399
4889
|
|
|
4400
|
-
def recover_template_lockfiles(member:, runner:, memo:, phase:)
|
|
4890
|
+
def recover_template_lockfiles(member:, runner:, memo:, phase:, commit_changes: true)
|
|
4401
4891
|
result = runner.call(
|
|
4402
4892
|
member: member,
|
|
4403
4893
|
phase: phase,
|
|
@@ -4407,13 +4897,9 @@ module Kettle
|
|
|
4407
4897
|
memo << result
|
|
4408
4898
|
return unless memo.last&.ok?
|
|
4409
4899
|
|
|
4410
|
-
|
|
4411
|
-
|
|
4412
|
-
|
|
4413
|
-
memo: memo,
|
|
4414
|
-
reason: "reset",
|
|
4415
|
-
force: true
|
|
4416
|
-
)
|
|
4900
|
+
return unless commit_changes
|
|
4901
|
+
|
|
4902
|
+
commit_normalized_lockfiles(branch_members: [member], runner: runner, memo: memo, reason: "reset", force: true)
|
|
4417
4903
|
end
|
|
4418
4904
|
|
|
4419
4905
|
def recoverable_bundle_failure?(result)
|
|
@@ -4555,7 +5041,7 @@ module Kettle
|
|
|
4555
5041
|
output.to_s.scan(/from the lockfile CHECKSUMS at Gemfile\.lock:(\d+):\d+/).flatten.map(&:to_i).uniq
|
|
4556
5042
|
end
|
|
4557
5043
|
|
|
4558
|
-
def prepare_template_dependencies(member:, runner:, memo:)
|
|
5044
|
+
def prepare_template_dependencies(member:, runner:, memo:, commit_changes: true, wave_jobs: 1)
|
|
4559
5045
|
command_text = config.template_command || default_template_command(member)
|
|
4560
5046
|
return true unless kettle_jem_template_command?(command_text)
|
|
4561
5047
|
|
|
@@ -4563,14 +5049,15 @@ module Kettle
|
|
|
4563
5049
|
member: member,
|
|
4564
5050
|
phase: "prepare_template_dependencies",
|
|
4565
5051
|
command: template_prepare_command(member),
|
|
4566
|
-
env: template_prepare_env
|
|
5052
|
+
env: template_prepare_env(wave_jobs: wave_jobs)
|
|
4567
5053
|
)
|
|
4568
5054
|
if recoverable_bundle_failure?(result)
|
|
4569
5055
|
recover_template_lockfiles(
|
|
4570
5056
|
member: member,
|
|
4571
5057
|
runner: runner,
|
|
4572
5058
|
memo: memo,
|
|
4573
|
-
phase: "prepare_template_dependencies_recovery"
|
|
5059
|
+
phase: "prepare_template_dependencies_recovery",
|
|
5060
|
+
commit_changes: commit_changes
|
|
4574
5061
|
)
|
|
4575
5062
|
if no_release_lockfiles_failure?(memo.last)
|
|
4576
5063
|
memo.pop
|
|
@@ -4583,7 +5070,7 @@ module Kettle
|
|
|
4583
5070
|
member: member,
|
|
4584
5071
|
phase: "prepare_template_dependencies",
|
|
4585
5072
|
command: template_prepare_command(member),
|
|
4586
|
-
env: template_prepare_env
|
|
5073
|
+
env: template_prepare_env(wave_jobs: wave_jobs)
|
|
4587
5074
|
)
|
|
4588
5075
|
end
|
|
4589
5076
|
memo << result
|
|
@@ -4635,15 +5122,17 @@ module Kettle
|
|
|
4635
5122
|
end
|
|
4636
5123
|
end
|
|
4637
5124
|
|
|
4638
|
-
def template_prepare_env
|
|
5125
|
+
def template_prepare_env(wave_jobs: 1)
|
|
4639
5126
|
# Templating is development work. It must resolve the configured
|
|
4640
5127
|
# family graph exactly as template application does; release-only
|
|
4641
5128
|
# lockfile cleanup owns disabling local path sources.
|
|
4642
|
-
execution_profile(template_execution_profile, workflow_env)
|
|
5129
|
+
execution_profile(template_execution_profile, workflow_env).merge(
|
|
5130
|
+
"KETTLE_FAMILY_WAVE_JOBS" => wave_jobs.to_s
|
|
5131
|
+
)
|
|
4643
5132
|
end
|
|
4644
5133
|
|
|
4645
|
-
def template_command_env
|
|
4646
|
-
(command == "template") ? template_prepare_env : command_env
|
|
5134
|
+
def template_command_env(wave_jobs: 1)
|
|
5135
|
+
(command == "template") ? template_prepare_env(wave_jobs: wave_jobs) : command_env
|
|
4647
5136
|
end
|
|
4648
5137
|
|
|
4649
5138
|
def template_execution_profile
|
|
@@ -4680,11 +5169,6 @@ module Kettle
|
|
|
4680
5169
|
env_overrides.key?(name) || ENV.key?(name)
|
|
4681
5170
|
end
|
|
4682
5171
|
|
|
4683
|
-
def template_thread_worker_budget
|
|
4684
|
-
member_jobs = template_jobs(members)
|
|
4685
|
-
[1, (Etc.nprocessors / member_jobs) - 1].max
|
|
4686
|
-
end
|
|
4687
|
-
|
|
4688
5172
|
def normalize_release_lockfiles(member:, runner:, memo:)
|
|
4689
5173
|
result = runner.call(
|
|
4690
5174
|
member: member,
|
data/lib/kettle/family.rb
CHANGED
|
@@ -13,6 +13,7 @@ require_relative "family/secrets"
|
|
|
13
13
|
require_relative "family/secrets_broker"
|
|
14
14
|
require_relative "family/branch_target_config"
|
|
15
15
|
require_relative "family/release_state_event_tape"
|
|
16
|
+
require_relative "family/concurrency"
|
|
16
17
|
require_relative "family/release_state_check"
|
|
17
18
|
require_relative "family/release_reconciler"
|
|
18
19
|
require_relative "family/unreleased_gem_cleanup"
|
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kettle-family
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.2.
|
|
4
|
+
version: 1.2.91
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Peter H. Boling
|
|
@@ -410,6 +410,7 @@ files:
|
|
|
410
410
|
- lib/kettle/family/cli.rb
|
|
411
411
|
- lib/kettle/family/command_result.rb
|
|
412
412
|
- lib/kettle/family/command_runner.rb
|
|
413
|
+
- lib/kettle/family/concurrency.rb
|
|
413
414
|
- lib/kettle/family/config.rb
|
|
414
415
|
- lib/kettle/family/debugger_bootstrap.rb
|
|
415
416
|
- lib/kettle/family/dependency_floor.rb
|
|
@@ -442,10 +443,10 @@ licenses:
|
|
|
442
443
|
- AGPL-3.0-only
|
|
443
444
|
metadata:
|
|
444
445
|
homepage_uri: https://kettle-family.galtzo.com
|
|
445
|
-
source_code_uri: https://github.com/kettle-dev/kettle-family/tree/v1.2.
|
|
446
|
-
changelog_uri: https://github.com/kettle-dev/kettle-family/blob/v1.2.
|
|
446
|
+
source_code_uri: https://github.com/kettle-dev/kettle-family/tree/v1.2.91
|
|
447
|
+
changelog_uri: https://github.com/kettle-dev/kettle-family/blob/v1.2.91/CHANGELOG.md
|
|
447
448
|
bug_tracker_uri: https://github.com/kettle-dev/kettle-family/issues
|
|
448
|
-
documentation_uri: https://www.rubydoc.info/gems/kettle-family/1.2.
|
|
449
|
+
documentation_uri: https://www.rubydoc.info/gems/kettle-family/1.2.91
|
|
449
450
|
funding_uri: https://github.com/sponsors/pboling
|
|
450
451
|
wiki_uri: https://github.com/kettle-dev/kettle-family/wiki
|
|
451
452
|
news_uri: https://www.railsbling.com/tags/kettle-family
|
metadata.gz.sig
CHANGED
|
Binary file
|