kettle-family 1.2.91 → 1.2.93

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e63a81da6b871ee1c27d813b23f7840e7c0e0a577e3a992076922ff2750574b3
-  data.tar.gz: 859158cd1aa7b293126edb628f02eb8bc641a0fcd4797a6f097af9fe46a9ef27
+  metadata.gz: a1a24d4a7f560730b58dea502aa74c1f7c2b6b5821c92bc565ee5d0eff79d8be
+  data.tar.gz: 58f47810263c6279e727419acb896abb959ccac76bff444c83b19c88c85f64f5
 SHA512:
-  metadata.gz: ee75302308856eb21755bfdc3a598fe9e0d3992b4a493c84825e891dbd76f514f976549fde093de80bfe55d822567d36cba2375a666c1ca2fc6b4a9844f8ae5c
-  data.tar.gz: 3845dff951350adc5177e8ea3bb4ce9b80a155a4a6fbb2b05618acb2363e77e9e8ce158129521f03ca8d02816b8bdfd1f1c5ec80507c982a0546e95314815068
+  metadata.gz: 7bb4085cb68b1d1ee1592ad1ab43d55afbc08a4c12a341ffffb6f1b3d712f43e51e0b8b4f59095fdd8402e1fa7c21822d6b43da8c8bd4caf340e102418d2974c
+  data.tar.gz: c12db5ebfea5489500b221fbc39f8d638fc6f4ad119eb73a210e195b779804325be2effad8970a78b65f7a5f0b837a9d324bb12242ab33c61b782877de5d6343

data/CHANGELOG.md

@@ -30,6 +30,28 @@ Please file a bug if you notice a violation of semantic versioning.
 
 ### Security
 
+## [1.2.93] - 2026-09-07
+
+- TAG: [v1.2.93][1.2.93t]
+- COVERAGE: 93.54% -- 6165/6591 lines in 35 files
+- BRANCH COVERAGE: 78.08% -- 2507/3211 branches in 35 files
+- 32.41% documented
+
+### Fixed
+
+- Update family-root bundles before member bundles for bup and bupb, committing the root lockfile only when the family is a monorepo.
+
+## [1.2.92] - 2026-09-07
+
+- TAG: [v1.2.92][1.2.92t]
+- COVERAGE: 93.52% -- 6144/6570 lines in 35 files
+- BRANCH COVERAGE: 77.98% -- 2490/3193 branches in 35 files
+- 32.41% documented
+
+### Fixed
+
+- Preserve pre-existing monorepo member changes while templating in parallel worktrees, reject concurrent member edits without overwriting them, and count only fully materialized members as successful.
+
 ## [1.2.91] - 2026-09-06
 
 - TAG: [v1.2.91][1.2.91t]
@@ -2605,7 +2627,11 @@ Please file a bug if you notice a violation of semantic versioning.
 - Fixed CI load failures on engines without compatible `pty` support by falling back to Open3 for interactive release commands.
 - Fixed Ruby 3.2 version-bump support by loading Prism lazily and wiring the Prism gem only for MRI versions that need it.
 
-[Unreleased]: https://github.com/kettle-dev/kettle-family/compare/v1.2.91...HEAD
+[Unreleased]: https://github.com/kettle-dev/kettle-family/compare/v1.2.93...HEAD
+[1.2.93]: https://github.com/kettle-dev/kettle-family/compare/v1.2.92...v1.2.93
+[1.2.93t]: https://github.com/kettle-dev/kettle-family/releases/tag/v1.2.93
+[1.2.92]: https://github.com/kettle-dev/kettle-family/compare/v1.2.91...v1.2.92
+[1.2.92t]: https://github.com/kettle-dev/kettle-family/releases/tag/v1.2.92
 [1.2.91]: https://github.com/kettle-dev/kettle-family/compare/v1.2.90...v1.2.91
 [1.2.91t]: https://github.com/kettle-dev/kettle-family/releases/tag/v1.2.91
 [1.2.90]: https://github.com/kettle-dev/kettle-family/compare/v1.2.89...v1.2.90

data/README.md

@@ -281,6 +281,22 @@ canonical development lockfile and CI checkout share that path graph. In both
 cases, release normalization rejects every path outside the configured allow
 list.
 
+### Template concurrency
+
+Executed `kettle-jem` templating for monorepo members uses detached worktrees
+when a dependency wave contains multiple ready members and the worker budget is
+greater than one. Before a worker starts, Family copies that member's current
+tracked and untracked state into its worktree. This preserves development
+lockfile changes produced by local dependency updates instead of assuming that
+the primary checkout matches `HEAD`.
+
+After the worker succeeds, Family verifies that the same member in the primary
+checkout has not changed since the snapshot. It then materializes the worker's
+final member-scoped state and serializes the normal template commit. Changes
+outside the member root are rejected, as are concurrent edits to that member;
+in either case the primary member is left untouched. Template summaries count
+a member only after worktree materialization and any required commit succeed.
+
 ### Test concurrency
 
 `kettle-family test --execute` runs independent sibling repositories in
@@ -480,8 +496,8 @@ The command names below are available from `kettle-family --help`:
 | `docs` | Run the configured documentation workflow for each selected member. |
 | `template` | Apply the configured `kettle-jem` template for each selected member. |
 | `gha-sha-pins` | Inspect or update GitHub Actions SHA pins. |
-| `bup` | Run `bundle update --all`, or update a named dependency, per member. |
-| `bupb` | Run `bundle update --bundler` per member. |
+| `bup` | Run `bundle update --all`, or update a named dependency, at the family root and per member. |
+| `bupb` | Run `bundle update --bundler` at the family root and per member. |
 | `bex` | Run an arbitrary `bundle exec` command per member. |
 | `install` | Build and install selected local family gems. |
 | `bump` | Check, plan, or execute family version alignment. |
@@ -580,10 +596,17 @@ kettle-family bup --execute
 kettle-family template --execute
 ```
 
-`bup` runs `bundle update --all` for every selected member. A gem name may be
-passed to update only that dependency, for example `kettle-family bup --execute
-kettle-test`. Executed updates commit successful lockfile changes by default;
-use `--no-commit` for a working-tree-only refresh, or `--commit` to state the
+`bup` first runs `bundle update --all` at the family root when that root has a
+`Gemfile`, then runs it for every selected member. `bupb` follows the same
+root-first order with `bundle update --bundler`. A root that is itself a member
+is updated only once. A gem name may be passed to `bup` to update only that
+dependency, for example `kettle-family bup --execute kettle-test`.
+
+Executed member updates commit successful lockfile changes by default. The
+family-root lockfile is also validated and committed for monorepos, where the
+root and members share one Git repository. A sibling-repository family root is
+updated for its orchestration bundle but is not included in any member commit.
+Use `--no-commit` for a working-tree-only refresh, or `--commit` to state the
 default explicitly. Local dependency environment variables are preserved when
 explicitly supplied, so use the family's local or released dependency mode
 deliberately before running it.
@@ -1121,7 +1144,7 @@ Thanks for RTFM. ☺️
 [📌gitmoji]: https://gitmoji.dev
 [📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
 [🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
-[🧮kloc-img]: https://img.shields.io/badge/KLOC-6.528-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
+[🧮kloc-img]: https://img.shields.io/badge/KLOC-6.591-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
 [🔐security]: https://github.com/kettle-dev/kettle-family/blob/main/SECURITY.md
 [🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
 [📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year

data/lib/kettle/family/report.rb

@@ -332,11 +332,16 @@ module Kettle
         template_results = results.select { |result| result.phase == "template" }
         return if template_results.empty?
 
+        successful_names = summary_succeeded
+        template_results.select! do |result|
+          !result.ok? || successful_names.include?(result.member_name)
+        end
+
         changed_files = template_results.sum { |result| template_changed_file_count(result) }
         outcome_counts = template_results.map { |result| template_file_outcomes(result) }
         lines << "template summary:"
-        lines << "  #{template_results.count(&:ok?)}/#{selected_members.length} members ok"
-        if outcome_counts.all?
+        lines << "  #{successful_names.length}/#{selected_members.length} members ok"
+        if outcome_counts.any? && outcome_counts.all?
           lines << "  #{outcome_counts.sum { |outcomes| outcomes.fetch(:checksum_hits) }} checksum hits"
           protected = outcome_counts.sum { |outcomes| outcomes.fetch(:checksum_protected) }
           lines << "  #{protected} checksum-protected changes" if protected.positive?
@@ -697,6 +702,7 @@ module Kettle
 
       def resume_hint_for(result)
         return release_resume_hint(result) if command == "release"
+        return "kettle-family #{command}" if result.phase.start_with?("family_root_") || result.phase == "commit_family_root_bundle_update"
 
         "kettle-family #{command} --start-at #{result.member_name}"
       end

data/lib/kettle/family/version.rb

@@ -5,7 +5,7 @@ module Kettle
     # Version namespace for this gem.
     module Version
       # Current gem version.
-      VERSION = "1.2.91"
+      VERSION = "1.2.93"
     end
     # Current gem version exposed at the traditional constant location.
     VERSION = Version::VERSION # Traditional Constant Location

data/lib/kettle/family/workflow.rb

@@ -2,6 +2,7 @@
 
 require "io/console"
 require "fileutils"
+require "digest"
 require "json"
 require "etc"
 require "open3"
@@ -574,6 +575,9 @@ module Kettle
 
         runner = CommandRunner.new(execute: execute, accept: accept)
         results = []
+        update_family_root_bundle(runner: runner, memo: results) if %w[bup bupb].include?(command)
+        return results unless results.all?(&:ok?)
+
         if command == "gha-sha-pins" && execute
           return results unless review_gha_sha_pins(workflow_members, runner: runner, memo: results)
         end
@@ -616,6 +620,39 @@ module Kettle
         results
       end
 
+      def update_family_root_bundle(runner:, memo:)
+        return unless family_root_bundle_update_required?
+
+        root_member = family_member
+        memo << runner.call(
+          member: root_member,
+          phase: "family_root_#{command}",
+          command: workflow_command,
+          env: command_env
+        )
+        return unless memo.last.ok?
+        return unless family_root_bundle_commit?
+        return unless validate_bundle_update_lockfile(
+          member: root_member,
+          memo: memo,
+          phase: "family_root_bundle_update_readiness"
+        )
+
+        commit_family_root_bundle_update(member: root_member, runner: runner, memo: memo)
+      end
+
+      def family_root_bundle_update_required?
+        return false unless File.file?(File.join(config.root, "Gemfile"))
+
+        family_members.none? do |member|
+          File.expand_path(member.root) == File.expand_path(config.root)
+        end
+      end
+
+      def family_root_bundle_commit?
+        commit && config.family_mode == "monorepo"
+      end
+
       # Sibling repositories have isolated working trees and can run their
       # tests concurrently. A monorepo's default kettle-test command normally
       # executes the shared root suite, so it is represented by one selected
@@ -1047,8 +1084,12 @@ module Kettle
             worktree_root: worktree_root
           }
           entries << entry
+          prepare_monorepo_template_worktree(entry)
           results.concat(worktree_mise_trust_results(entry, runner: runner, phase: "template_member_worktree_mise_trust"))
           break unless results.last.ok?
+        rescue => error
+          results << template_worktree_failure_result(member, error, phase: "template_member_worktree_seed")
+          break
         end
         [entries, results]
       end
@@ -1102,39 +1143,64 @@ module Kettle
       def materialize_monorepo_template_changes(entry)
         original_member = entry.fetch(:original_member)
         worktree_root = entry.fetch(:worktree_root)
-        relative_root = Pathname.new(original_member.root).relative_path_from(Pathname.new(config.root)).to_s
+        relative_root = monorepo_template_relative_root(original_member)
         paths = monorepo_template_worktree_changed_paths(worktree_root)
-        return template_worktree_failure_result(original_member, paths) if paths.is_a?(String)
+        return template_worktree_failure_result(original_member, paths, phase: "template_worktree_materialize") if paths.is_a?(String)
 
         invalid_paths = paths.reject { |path| path == relative_root || path.start_with?("#{relative_root}/") }
         unless invalid_paths.empty?
           return template_worktree_failure_result(
             original_member,
-            "isolated template worker modified shared path(s): #{invalid_paths.sort.join(", ")}; only #{relative_root}/ may be materialized"
+            "isolated template worker modified shared path(s): #{invalid_paths.sort.join(", ")}; only #{relative_root}/ may be materialized",
+            phase: "template_worktree_materialize"
           )
         end
 
-        tracked_paths = monorepo_template_worktree_tracked_paths(worktree_root)
-        return template_worktree_failure_result(original_member, tracked_paths) if tracked_paths.is_a?(String)
-
-        runner = command_runner
-        control_member = monorepo_template_control_member.dup
-        control_member.name = original_member.name
-        command = if tracked_paths.empty?
-          ["true"]
-        else
-          source = Shellwords.escape(worktree_root)
-          scope = Shellwords.escape(relative_root)
-          ["sh", "-lc", "git -C #{source} diff --binary HEAD -- #{scope} | git apply --whitespace=nowarn"]
+        initial_state = entry.fetch(:primary_member_state)
+        current_state = monorepo_template_member_state(config.root, relative_root)
+        changed_primary_paths = changed_monorepo_template_state_paths(initial_state, current_state)
+        unless changed_primary_paths.empty?
+          return template_worktree_failure_result(
+            original_member,
+            "primary member changed while isolated template worker ran: #{changed_primary_paths.join(", ")}",
+            phase: "template_worktree_materialize"
+          )
         end
-        result = runner.call(member: control_member, phase: "template_worktree_materialize", command: command)
-        return result unless result.ok?
 
-        copy_monorepo_template_untracked_paths(worktree_root, config.root, relative_root)
-        result.stdout = "#{paths.length} path(s) materialized from isolated worktree"
-        result
+        materialized_paths = (entry.fetch(:primary_changed_paths) + paths).uniq.sort
+        sync_monorepo_template_paths(worktree_root, config.root, materialized_paths)
+        template_worktree_success_result(
+          original_member,
+          phase: "template_worktree_materialize",
+          stdout: "#{materialized_paths.length} path(s) materialized from isolated worktree"
+        )
       rescue => error
-        template_worktree_failure_result(original_member, error)
+        template_worktree_failure_result(original_member, error, phase: "template_worktree_materialize")
+      end
+
+      def prepare_monorepo_template_worktree(entry)
+        original_member = entry.fetch(:original_member)
+        relative_root = monorepo_template_relative_root(original_member)
+        changed_paths = monorepo_template_worktree_changed_paths(config.root)
+        raise Error, changed_paths if changed_paths.is_a?(String)
+
+        member_changed_paths = changed_paths.select do |path|
+          path == relative_root || path.start_with?("#{relative_root}/")
+        end
+        sync_monorepo_template_paths(config.root, entry.fetch(:worktree_root), member_changed_paths)
+        primary_state = monorepo_template_member_state(config.root, relative_root)
+        worktree_state = monorepo_template_member_state(entry.fetch(:worktree_root), relative_root)
+        unless primary_state == worktree_state
+          changed = changed_monorepo_template_state_paths(primary_state, worktree_state)
+          raise Error, "isolated template worker seed differs from primary member: #{changed.join(", ")}"
+        end
+
+        entry[:primary_changed_paths] = member_changed_paths
+        entry[:primary_member_state] = primary_state
+      end
+
+      def monorepo_template_relative_root(member)
+        Pathname.new(member.root).relative_path_from(Pathname.new(config.root)).to_s
       end
 
       def monorepo_template_worktree_changed_paths(worktree_root)
@@ -1155,6 +1221,28 @@ module Kettle
         git_worktree_paths(worktree_root, %w[ls-files --others --exclude-standard -z])
       end
 
+      def monorepo_template_member_state(root, relative_root)
+        paths = git_worktree_paths(root, ["ls-files", "--cached", "--others", "--exclude-standard", "-z", "--", relative_root])
+        raise Error, paths if paths.is_a?(String)
+
+        paths.to_h { |path| [path, monorepo_template_path_signature(root, path)] }
+      end
+
+      def monorepo_template_path_signature(root, path)
+        full_path = File.join(root, path)
+        stat = File.lstat(full_path)
+        return ["symlink", File.readlink(full_path)] if stat.symlink?
+        return ["file", stat.mode & 0o777, Digest::SHA256.file(full_path).hexdigest] if stat.file?
+
+        [stat.ftype, stat.mode & 0o777]
+      rescue Errno::ENOENT
+        ["missing"]
+      end
+
+      def changed_monorepo_template_state_paths(before, after)
+        (before.keys | after.keys).select { |path| before[path] != after[path] }.sort
+      end
+
       def git_worktree_paths(worktree_root, arguments)
         stdout, stderr, status = Open3.capture3("git", *arguments, chdir: worktree_root)
         return stdout.split("\0").reject(&:empty?) if status.success?
@@ -1162,14 +1250,19 @@ module Kettle
         "git #{arguments.join(" ")} failed in #{worktree_root}: #{stderr.strip}"
       end
 
-      def copy_monorepo_template_untracked_paths(worktree_root, primary_root, relative_root)
-        monorepo_template_worktree_untracked_paths(worktree_root).each do |path|
-          next unless path == relative_root || path.start_with?("#{relative_root}/")
+      def sync_monorepo_template_paths(source_root, destination_root, paths)
+        paths.each do |path|
+          source = File.join(source_root, path)
+          destination = File.join(destination_root, path)
+          FileUtils.rm_rf(destination)
+          next unless File.exist?(source) || File.symlink?(source)
 
-          source = File.join(worktree_root, path)
-          destination = File.join(primary_root, path)
           FileUtils.mkdir_p(File.dirname(destination))
-          FileUtils.cp_r(source, destination, preserve: true)
+          if File.symlink?(source)
+            File.symlink(File.readlink(source), destination)
+          else
+            FileUtils.cp_r(source, destination, preserve: true)
+          end
         end
       end
 
@@ -1184,12 +1277,28 @@ module Kettle
         end
       end
 
-      def template_worktree_failure_result(member, error)
+      def template_worktree_success_result(member, phase:, stdout:)
+        CommandResult.new(
+          member_name: member.name,
+          phase: phase,
+          command: ["internal", phase.tr("_", "-")],
+          workdir: member.root,
+          status: 0,
+          success: true,
+          stdout: stdout,
+          stderr: "",
+          elapsed_seconds: 0.0,
+          skipped: false,
+          reason: nil
+        )
+      end
+
+      def template_worktree_failure_result(member, error, phase: "template_member_worktree")
         message = error.is_a?(Exception) ? "#{error.class}: #{error.message}" : error.to_s
         CommandResult.new(
           member_name: member.name,
-          phase: "template_member_worktree",
-          command: ["internal", "template-member-worktree"],
+          phase: phase,
+          command: ["internal", phase.tr("_", "-")],
           workdir: member.root,
           status: 1,
           success: false,
@@ -4348,13 +4457,25 @@ module Kettle
       def emit_template_progress_summary(results, progress:)
         return unless progress
 
-        template_results = results.select { |result| result.phase == "template" }
+        successful_names = successful_template_member_names(results)
+        template_results = results.select do |result|
+          result.phase == "template" && (!result.ok? || successful_names.include?(result.member_name))
+        end
         changed_files = template_results.sum { |result| template_changed_file_count(result) }
         outcome_counts = template_results.map { |result| template_file_outcomes(result) }
         progress.stop
         progress.summary(template_progress_summary_label(template_results, changed_files, outcome_counts))
       end
 
+      def successful_template_member_names(results)
+        results.group_by(&:member_name).filter_map do |member_name, member_results|
+          next unless member_results.any? { |result| result.phase == "template" }
+          next unless member_results.all?(&:ok?)
+
+          member_name
+        end
+      end
+
       def emit_release_progress_summary(results, progress:)
         return unless progress
 
@@ -4815,7 +4936,7 @@ module Kettle
 
       def template_progress_summary_label(template_results, changed_files, outcome_counts)
         label = "template summary: #{template_results.count(&:ok?)}/#{members.length} members ok"
-        return "#{label}, #{template_summary_label(changed_files)}" unless outcome_counts.all?
+        return "#{label}, #{template_summary_label(changed_files)}" if outcome_counts.empty? || !outcome_counts.all?
 
         totals = outcome_counts.each_with_object({checksum_hits: 0, checksum_protected: 0, unchanged: 0}) do |outcomes, memo|
           memo[:checksum_hits] += outcomes.fetch(:checksum_hits)
@@ -5444,13 +5565,13 @@ module Kettle
         memo << result
       end
 
-      def validate_bundle_update_lockfile(member:, memo:)
+      def validate_bundle_update_lockfile(member:, memo:, phase: "bundle_update_readiness")
         return true unless execute && commit
 
         diagnostics = bundle_update_lockfile_diagnostics(member)
         result = CommandResult.new(
           member.name,
-          "bundle_update_readiness",
+          phase,
           ["internal", "bundle-update-readiness"],
           member.root,
           diagnostics.empty? ? 0 : 1,
@@ -5465,6 +5586,22 @@ module Kettle
         result.ok?
       end
 
+      def commit_family_root_bundle_update(member:, runner:, memo:)
+        return unless family_root_bundle_commit?
+
+        result = runner.call(
+          member: member,
+          phase: "commit_family_root_bundle_update",
+          command: [
+            "sh",
+            "-lc",
+            "files=$(git ls-files --modified --others --exclude-standard -- Gemfile.lock); " \
+              "if [ -n \"$files\" ]; then git add -- Gemfile.lock && git commit --only -m '🔒️ Update family root bundle' -- Gemfile.lock; fi"
+          ]
+        )
+        memo << result
+      end
+
       def bundle_update_lockfile_diagnostics(member)
         ReadinessCheck.call(
           member: member,

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: kettle-family
 version: !ruby/object:Gem::Version
-  version: 1.2.91
+  version: 1.2.93
 platform: ruby
 authors:
 - Peter H. Boling
@@ -443,10 +443,10 @@ licenses:
 - AGPL-3.0-only
 metadata:
   homepage_uri: https://kettle-family.galtzo.com
-  source_code_uri: https://github.com/kettle-dev/kettle-family/tree/v1.2.91
-  changelog_uri: https://github.com/kettle-dev/kettle-family/blob/v1.2.91/CHANGELOG.md
+  source_code_uri: https://github.com/kettle-dev/kettle-family/tree/v1.2.93
+  changelog_uri: https://github.com/kettle-dev/kettle-family/blob/v1.2.93/CHANGELOG.md
   bug_tracker_uri: https://github.com/kettle-dev/kettle-family/issues
-  documentation_uri: https://www.rubydoc.info/gems/kettle-family/1.2.91
+  documentation_uri: https://www.rubydoc.info/gems/kettle-family/1.2.93
   funding_uri: https://github.com/sponsors/pboling
   wiki_uri: https://github.com/kettle-dev/kettle-family/wiki
   news_uri: https://www.railsbling.com/tags/kettle-family