rdkafka 0.29.2 → 0.31.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f2acf73116281ea22d96af51b56fecc615cb32616d25834a5a990fae1213a1ba
-  data.tar.gz: 04c06f93561b22719261fab54fc0191f1a6edba01926596124f430adcc11d7d5
+  metadata.gz: 3c5788b76bdace5f4d9ecf3f4b2357c9b0b9d5165be2a2253d1e256dc26140bc
+  data.tar.gz: 0bf98ff4dd410fe2c7baccfc0b0a0d37ef0b2a33d515b07033003f75722568dd
 SHA512:
-  metadata.gz: 7ea74276532d8cd48ee913a46166497ace6426bd8859eb41d72f5407bc5a1eb4273bbe2d82a793618e3420a956b95a86919e416c46e0ac657d8b8056167dcdc0
-  data.tar.gz: 20e3c8ac1bb1113673f5af90e977dc3e59e6e6b9f9b0b550e1f6b1a9796bd19b1915d8ba8e345e5cfb0e574d61e8fe354788e0f3445ee0333a8634c7f81e4d97
+  metadata.gz: 0ee7727535b52e6a8e75381ba884a3a19b068636ab5738aa792b46b1779347c66908508e0a03ae9a510f21ea6a918c40610f2fbdb0dd05353a586c0310a7ba9e
+  data.tar.gz: 6af92011110706a80d16867a91891906280006620aaf6e804053e8a25c8d53f55e5fafbd511c15dc843c0a21aacada24a90571a43e97b3f64f4b7f6de99b8aa7

data/CHANGELOG.md

@@ -1,11 +1,38 @@
 # Rdkafka Changelog
 
+## 0.31.0 (2026-10-09)
+- **[EOL]** Drop Ruby 3.2 support; require Ruby 3.3+.
+- [Feature] Add `status`, `latency` and `broker_id` to `Producer::DeliveryReport`, with `#persisted?`, `#possibly_persisted?` and `#not_persisted?`, so a failed delivery tells whether a retry may duplicate the message.
+- [Feature] Add `Admin#describe_cluster` to get the cluster id, controller and broker nodes (with racks) without a full metadata request.
+- [Feature] Add `Admin#describe_topics` to get topic ids, partition leaders and replicas, and per-topic errors. Both methods can also return the operations the client is authorized to perform.
+- [Feature] Add `Admin#describe_consumer_groups` to read the state, type, coordinator, authorized operations and members (with their current and target partition assignments) of consumer groups.
+- [Enhancement] Bump librdkafka to `2.15.1` for OpenSSL and libcurl security fixes. It changes how IPv6 addresses are formatted and validated against broker certificates.
+- [Enhancement] Bump the bundled OpenSSL to `3.5.9` (LTS) for its security fixes.
+- [Enhancement] Read each pointer field once in `Message#initialize`, saving 3 allocations per consumed message.
+- [Change] Build the precompiled `macos_arm64` library on macOS 26 with the minimum macOS pinned to 14.0, instead of inheriting it from the build host.
+- [Change] Build the precompiled `aarch64-linux-musl` gem on Alpine `3.24` (was `3.23`), matching `x86_64-linux-musl`.
+- [Maintenance] Split `lib/rdkafka/bindings.rb` into per-area files under `lib/rdkafka/bindings/`. `Rdkafka::Bindings` is unchanged.
+
+## 0.30.0 (2026-09-25)
+- [Feature] Add `Admin#alter_consumer_group_offsets` and `Admin#delete_consumer_group_offsets` to set or clear a consumer group's committed offsets from the admin client.
+- [Enhancement] Bump the bundled zlib to `1.3.2`.
+- [Enhancement] Bump librdkafka to `2.15.0`.
+- [Enhancement] Bump the bundled MIT Kerberos (krb5) to `1.22.2`.
+- [Enhancement] Add `Rdkafka::Config.partitioner_key_uses_bytesize` to hash partition keys by byte length, matching other Kafka clients for multibyte keys. Defaults to `false`.
+- [Maintenance] Derive the zlib `CHECKSUMS` entry from `ZLIB_VERSION`.
+- [Maintenance] Drop the unused `dist/openssl-3.0.16.tar.gz` build cache.
+- [Fix] Honor the `isolation_level:` argument to `Admin#list_offsets`, which was silently ignored.
+- [Fix] Return correct results for every item of a multi-item topic, partition, group or ACL admin request, not just the first.
+- [Maintenance] Fix a use-after-free in the multi-item admin integration test that read result-name pointers after librdkafka had destroyed the background event, which made it flaky on newer glibc (e.g. Debian trixie).
+- [Maintenance] Stabilize consumer specs on slow CI runners: wait longer for a partition assignment, raise the `TestTopics.create` admin timeout and drop the fixed 60s limit of the long-running consumption spec.
+- [Maintenance] Raise the `TestTopics.create` admin timeout to 30s so a loaded CI runner no longer hits `create topic timed out` (which also tripped the leaked-handle guard).
+
 ## 0.29.2 (2026-09-11)
 - [Fix] Close live clients from an `at_exit` hook before Ruby's shutdown finalization, so librdkafka is no longer `dlclose`d while its native threads are still running (which could segfault on exit).
 - [Fix] Make client construction exception-safe and destroy the native handle when setup fails after `rd_kafka_new`, so a later error no longer orphans the native client.
-- [Fix] Destroy each polled message exactly once in `Consumer#poll_batch`/`#poll_batch_nb` when building a message raises a non-`RdkafkaError`, fixing a double-free that could abort the process and closing the matching leak window.
-- [Fix] Free the background queue, `AdminOptions` and any already-built `ConfigResource`s (and remove the handle) when `Admin#describe_configs`/`#incremental_alter_configs` raise while building native resources (e.g. a non-String resource name).
-- [Fix] Raise a clear `ConfigError` instead of segfaulting when `Admin#describe_configs`/`#incremental_alter_configs` are given an empty resource name or a negative resource type (`rd_kafka_ConfigResource_new` returns NULL), or when `rd_kafka_AdminOptions_new` returns NULL.
+- [Fix] Fix a double-free in `Consumer#poll_batch`/`#poll_batch_nb` that could abort the process when building a message raised a non-`RdkafkaError`.
+- [Fix] Free native resources when `Admin#describe_configs`/`#incremental_alter_configs` raise while building their request (e.g. a non-String resource name).
+- [Fix] Raise `ConfigError` instead of segfaulting when `Admin#describe_configs`/`#incremental_alter_configs` get an empty resource name or a negative resource type.
 
 ## 0.29.1 (2026-09-04)
 - [Enhancement] Add `Admin#delete_records` to delete all messages in a partition up to a given offset.
@@ -19,8 +46,8 @@
 
 ## 0.29.0 (2026-07-10)
 - [Enhancement] Bump librdkafka to `2.14.2`
-- [Enhancement] Add `Consumer#metadata` and `Producer#metadata`, mirroring `Admin#metadata`, so cluster/topic metadata can be fetched from an existing consumer or producer handle without opening a dedicated admin connection.
-- [Enhancement] Support offset-commit metadata: `Consumer#store_offset(message, metadata)` stores an optional metadata string alongside the offset, `Consumer::Partition#metadata` exposes it, and `TopicPartitionList` marshals it to and from the native list (read back via `Consumer#committed`).
+- [Enhancement] Add `Consumer#metadata` and `Producer#metadata`, mirroring `Admin#metadata`, to fetch cluster metadata without a separate admin connection.
+- [Enhancement] Support offset-commit metadata via `Consumer#store_offset(message, metadata)`, readable through `Consumer::Partition#metadata` and `Consumer#committed`.
 - [Enhancement] Expose `replicas` and `isrs` (in-sync replica broker ids) on each partition in topic metadata; both were previously dropped from the `Metadata#topics` partition hashes.
 - [Enhancement] Extract the admin background-event result handlers into one class per operation under `lib/rdkafka/callbacks/`. Internal reorganization with no API or behavior change.
 - [Enhancement] Reuse per-thread scratch pointers in `Consumer::Headers.from_native`, removing the per-message native allocations from the consumer hot path.
@@ -28,32 +55,32 @@
 - [Fix] Cache the partition count for a missing topic, so `produce` with a `partition_key` to a missing topic no longer re-runs a blocking metadata query on every message.
 - [Fix] Let `PartitionsCountCache` adopt a lower partition count once the cached entry has expired, so a topic recreated with fewer partitions no longer fails `produce` until process restart.
 - [Fix] Surface per-message build errors inline in `Consumer#poll_batch`/`#poll_batch_nb` instead of discarding the batch, preserving the messages that built successfully.
-- [Fix] Bound the `Metadata` retry loop by a ~5s wall-clock budget with a floor of 3 attempts, cap the backoff at 1s, and stop overwriting the request timeout with the backoff value, which could block a synchronous fetch for minutes.
+- [Fix] Bound the `Metadata` retry loop to about 5 seconds, so a synchronous metadata fetch can no longer block for minutes.
 - [Fix] Synchronize `AbstractHandle::REGISTRY` mutations with a mutex, which could otherwise lose a write on JRuby and leave a handle unregistered or leaked.
-- [Fix] Fix the NULL background-queue cleanup branches in `Admin#delete_group`, `Admin#delete_acl` and `Admin#describe_acl`, which raised a `NameError` and leaked the native request object instead of raising `ConfigError`.
+- [Fix] Raise `ConfigError` instead of `NameError` in `Admin#delete_group`, `#delete_acl` and `#describe_acl` when the background queue is unavailable.
 - [Fix] Actually populate `ConfigBindingResult#synonyms`. The synonym loop built the synonym binding objects and discarded them, so `#synonyms` was always an empty array; they are now appended.
 - [Fix] Stop leaking the native `rd_kafka_topic_conf_t` in `Producer#set_topic_config` when a per-topic config value is rejected.
-- [Fix] Add the missing `closed_consumer_check` to `Consumer#position`, the only public consumer method without one; calling it on a closed consumer now raises a consistent `ClosedConsumerError` instead of failing deeper in the call.
+- [Fix] Raise `ClosedConsumerError` from `Consumer#position` on a closed consumer, like the other consumer methods.
 - [Fix] Guard the OAuthBearer token refresh callback against a NULL client pointer before calling `rd_kafka_name` on it.
-- [Fix] Make the `Consumer` GC finalizer close the consumer and destroy its consumer queue, not just the native client, preventing a hang or leak when a consumer is collected without an explicit `close`.
+- [Fix] Fully close a consumer collected without an explicit `close`, preventing a hang or leak.
 - [Fix] Free the librdkafka-allocated string in `Consumer#cluster_id` and `Consumer#member_id`, and fix the `rd_kafka_clusterid` arity. `Consumer#cluster_id` now accepts a `timeout_ms`.
 - [Fix] Attach `rd_kafka_query_watermark_offsets` with `blocking: true` so it releases the GVL; it previously froze every other Ruby thread for up to `timeout_ms`.
 - [Fix] Stop leaking the native `rd_kafka_conf_t` when client creation fails, a multi-KB leak per failed attempt for supervisors retrying on transient SASL/SSL misconfiguration.
 - [Fix] Stop `Metadata` from leaking the native metadata struct on every retried fetch; each attempt now frees its own native resources.
 - [Fix] Stop `Producer#produce` from orphaning the delivery handle in the process-global registry when it fails after registering it.
 - [Fix] Guard the message delivery callback so a raising user `delivery_callback` can no longer leave the handle locked or crash the producer; exceptions are now logged and swallowed.
-- [Fix] Stop `describe_configs`, `incremental_alter_configs` and `list_offsets` from leaking handles and native resources when their arguments are rejected; all three now validate input before allocating.
-- [Fix] Destroy the native topic-partition list in `TopicPartitionList#to_native_tpl` when population fails partway (e.g. an invalid partition). The caller-owned list was leaked on any exception mid-build; it is now destroyed before the error propagates.
+- [Fix] Stop `describe_configs`, `incremental_alter_configs` and `list_offsets` from leaking native resources when their arguments are rejected.
+- [Fix] Stop `TopicPartitionList#to_native_tpl` from leaking the native list when building it fails partway.
 - [Fix] Raise instead of silently dropping a rejected `incremental_alter_configs` entry, which previously left the alter request reporting success.
 - [Fix] Allocate the admin result-count out-parameter as `:size_t` instead of `:int32`, fixing a 4-byte overflow on every admin result parse.
 - [Fix] Stabilize the flaky `Consumer#lag` "calculates the consumer lag" spec on overloaded CI.
-- [Fix] Resolve admin operation handles from the event error when an operation fails at the operation level (e.g. brokers unreachable), instead of blocking until `wait` timed out and discarding the real error.
-- [Fix] Destroy admin API background events after processing; they were never destroyed, so every admin operation leaked its whole result event. The internal FFI struct fields on admin handles were removed as part of this - use `handle.wait` and the returned report objects, whose interfaces are unchanged.
+- [Fix] Return the real error from an admin operation that fails at the operation level (e.g. brokers unreachable) instead of blocking until `wait` times out.
+- [Fix] Stop every admin operation from leaking its result event. The internal FFI struct fields on admin handles are removed; use `handle.wait` and the returned report objects, which are unchanged.
 
 ## 0.28.0 (2026-06-03)
 - [Enhancement] Bump librdkafka to `2.14.1`
 - [Enhancement] Replace version-gated `Warning[:performance]` with dynamic `Warning.categories` opt-in in spec helper so new Ruby warning categories are automatically enabled without manual updates.
-- [Enhancement] `poll_batch` and `poll_batch_nb` now return error events inline as `RdkafkaError` objects rather than raising on the first error. The return type is `Array<Message, RdkafkaError>` and callers are responsible for handling errors in the result.
+- [Enhancement] `poll_batch` and `poll_batch_nb` now return errors inline as `RdkafkaError` objects instead of raising on the first one. The return type is `Array<Message, RdkafkaError>`, so callers must handle errors in the result.
 
 ## 0.27.0 (2026-05-07)
 - [Feature] Add `Consumer#poll_batch(timeout_ms, max_items:)` and `Consumer#poll_batch_nb(timeout_ms, max_items:)` for batch message polling via `rd_kafka_consume_batch_queue`.
@@ -75,7 +102,7 @@
 - [Feature] Support `rd_kafka_ListOffsets` admin API for querying partition offsets by specification (earliest, latest, max_timestamp, or by timestamp) without requiring a consumer group (#613).
 - [Enhancement] Use native ARM64 runners instead of QEMU emulation for Alpine musl aarch64 builds, improving build performance and reliability.
 - [Enhancement] Enable parallel compilation (`make -j$(nproc)`) for ARM64 Alpine musl builds.
-- [Enhancement] Add file descriptor API for fiber scheduler integration. Expose `enable_queue_io_events` and `enable_background_queue_io_events` on `Consumer`, `Producer`, and `Admin` to enable non-blocking monitoring with select/poll/epoll for integration with Ruby fiber schedulers (Falcon, Async) and custom event loops.
+- [Enhancement] Add `enable_queue_io_events` and `enable_background_queue_io_events` to `Consumer`, `Producer` and `Admin` for non-blocking file descriptor monitoring with fiber schedulers (Falcon, Async) and custom event loops.
 - [Enhancement] Add non-blocking poll methods (`poll_nb`, `events_poll_nb`) on `Consumer` that skip GVL release for efficient fiber scheduler integration when using `poll(0)`.
 - [Enhancement] Add `events_poll_nb_each` method on `Producer`, `Consumer`, and `Admin` for polling events in a single GVL/mutex session. Yields count after each iteration, caller returns `:stop` to break.
 - [Enhancement] Add `poll_nb_each` method on `Consumer` for non-blocking message polling with proper resource cleanup, yielding each message and supporting early termination via `:stop` return value.

data/README.md

@@ -163,6 +163,8 @@ bundle exec rake produce_messages
 
 | rdkafka-ruby | librdkafka | patches |
 |-|-|-|
+| 0.31.x (2026-10-09) | 2.15.1 (2026-09-09) | yes |
+| 0.30.x (2026-09-25) | 2.15.0 (2026-06-30) | yes |
 | 0.29.x (2026-07-10) | 2.14.2 (2026-06-03) | yes |
 | 0.28.x (2026-06-03) | 2.14.1 (2026-04-15) | yes |
 | 0.27.x (2026-05-07) | 2.14.0 (2026-04-01) | yes |

data/docker-compose-ssl.yml

@@ -17,6 +17,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,SSL:SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://127.0.0.1:9092,SSL://127.0.0.1:9093
       KAFKA_BROKER_ID: 1
+      KAFKA_BROKER_RACK: rack-1
       KAFKA_CONTROLLER_QUORUM_VOTERS: 1@127.0.0.1:9094
       ALLOW_PLAINTEXT_LISTENER: 'yes'
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'

data/docker-compose.yml

@@ -16,6 +16,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://127.0.0.1:9092
       KAFKA_BROKER_ID: 1
+      KAFKA_BROKER_RACK: rack-1
       KAFKA_CONTROLLER_QUORUM_VOTERS: 1@127.0.0.1:9093
       ALLOW_PLAINTEXT_LISTENER: 'yes'
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'

data/lib/rdkafka/admin/alter_consumer_group_offsets_handle.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Handle for alter consumer group offsets operation
+    class AlterConsumerGroupOffsetsHandle < AbstractHandle
+      layout :pending, :bool,
+        :response, :int
+
+      # @return [String] the name of the operation
+      def operation_name
+        "alter consumer group offsets"
+      end
+
+      # Creates the result report
+      # @return [AlterConsumerGroupOffsetsReport]
+      def create_result
+        prepared_result
+      end
+
+      # Raises an error if the operation failed
+      # @raise [RdkafkaError]
+      def raise_error
+        raise RdkafkaError.new(
+          self[:response],
+          broker_message: broker_message
+        )
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/alter_consumer_group_offsets_report.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Report for alter consumer group offsets operation result
+    class AlterConsumerGroupOffsetsReport
+      # @return [String, nil] the group the operation ran against
+      attr_reader :group_name
+
+      # @return [String, nil] group level error message, if any
+      attr_reader :error_string
+
+      # Per partition outcomes, each with `:topic`, `:partition`, `:offset` and `:error`
+      # (an `RdkafkaError` or nil). Partitions are reported individually because Kafka can
+      # fail some and accept others within a single request.
+      # @return [Array<Hash>]
+      attr_reader :partitions
+
+      # @param error_string [FFI::Pointer] pointer to the group level error string
+      # @param result_name [FFI::Pointer] pointer to the group name
+      # @param partitions_ptr [FFI::Pointer] pointer to the per partition result list
+      def initialize(error_string, result_name, partitions_ptr = FFI::Pointer::NULL)
+        @error_string = error_string.read_string if error_string != FFI::Pointer::NULL
+        @group_name = result_name.read_string if result_name != FFI::Pointer::NULL
+        @partitions = extract_partitions(partitions_ptr)
+      end
+
+      private
+
+      # @param partitions_ptr [FFI::Pointer] pointer to a native topic partition list
+      # @return [Array<Hash>]
+      def extract_partitions(partitions_ptr)
+        return [] if partitions_ptr.nil? || partitions_ptr.null?
+
+        native = Bindings::TopicPartitionList.new(partitions_ptr)
+
+        Array.new(native[:cnt]) do |i|
+          tp = Bindings::TopicPartition.new(native[:elems] + (i * Bindings::TopicPartition.size))
+
+          {
+            topic: tp[:topic],
+            partition: tp[:partition],
+            offset: tp[:offset],
+            error: tp[:err].zero? ? nil : RdkafkaError.new(tp[:err])
+          }
+        end
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/delete_consumer_group_offsets_handle.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Handle for delete consumer group offsets operation
+    class DeleteConsumerGroupOffsetsHandle < AbstractHandle
+      layout :pending, :bool,
+        :response, :int
+
+      # @return [String] the name of the operation
+      def operation_name
+        "delete consumer group offsets"
+      end
+
+      # Creates the result report
+      # @return [DeleteConsumerGroupOffsetsReport]
+      def create_result
+        prepared_result
+      end
+
+      # Raises an error if the operation failed
+      # @raise [RdkafkaError]
+      def raise_error
+        raise RdkafkaError.new(
+          self[:response],
+          broker_message: broker_message
+        )
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/delete_consumer_group_offsets_report.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Report for delete consumer group offsets operation result
+    class DeleteConsumerGroupOffsetsReport
+      # @return [String, nil] the group the operation ran against
+      attr_reader :group_name
+
+      # @return [String, nil] group level error message, if any
+      attr_reader :error_string
+
+      # Per partition outcomes, each with `:topic`, `:partition`, `:offset` and `:error`
+      # (an `RdkafkaError` or nil). Partitions are reported individually because Kafka can
+      # fail some and accept others within a single request.
+      # @return [Array<Hash>]
+      attr_reader :partitions
+
+      # @param error_string [FFI::Pointer] pointer to the group level error string
+      # @param result_name [FFI::Pointer] pointer to the group name
+      # @param partitions_ptr [FFI::Pointer] pointer to the per partition result list
+      def initialize(error_string, result_name, partitions_ptr = FFI::Pointer::NULL)
+        @error_string = error_string.read_string if error_string != FFI::Pointer::NULL
+        @group_name = result_name.read_string if result_name != FFI::Pointer::NULL
+        @partitions = extract_partitions(partitions_ptr)
+      end
+
+      private
+
+      # @param partitions_ptr [FFI::Pointer] pointer to a native topic partition list
+      # @return [Array<Hash>]
+      def extract_partitions(partitions_ptr)
+        return [] if partitions_ptr.nil? || partitions_ptr.null?
+
+        native = Bindings::TopicPartitionList.new(partitions_ptr)
+
+        Array.new(native[:cnt]) do |i|
+          tp = Bindings::TopicPartition.new(native[:elems] + (i * Bindings::TopicPartition.size))
+
+          {
+            topic: tp[:topic],
+            partition: tp[:partition],
+            offset: tp[:offset],
+            error: tp[:err].zero? ? nil : RdkafkaError.new(tp[:err])
+          }
+        end
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/describe_cluster_handle.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Handle for describe cluster operation
+    class DescribeClusterHandle < AbstractHandle
+      layout :pending, :bool,
+        :response, :int
+
+      # @return [String] the name of the operation
+      def operation_name
+        "describe cluster"
+      end
+
+      # @return [DescribeClusterReport] report prepared by the background event callback, with
+      #   the cluster id, controller and broker nodes.
+      def create_result
+        prepared_result
+      end
+
+      # Raises an error if the operation failed
+      # @raise [RdkafkaError]
+      def raise_error
+        raise RdkafkaError.new(
+          self[:response],
+          broker_message: broker_message
+        )
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/describe_cluster_report.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Report for describe cluster operation result
+    class DescribeClusterReport
+      include DescribeResultParsing
+
+      # @return [String, nil] the cluster id
+      attr_reader :cluster_id
+
+      # The current controller node, as a hash with `:id` [Integer], `:host` [String],
+      # `:port` [Integer] and `:rack` [String, nil]
+      # @return [Hash, nil]
+      attr_reader :controller
+
+      # Broker nodes of the cluster, each in the same shape as {#controller}
+      # @return [Array<Hash>]
+      attr_reader :nodes
+
+      # Operations the client is authorized to perform on the cluster, as
+      # `Bindings::RD_KAFKA_ACL_OPERATION_*` codes. `nil` unless requested with
+      # `include_authorized_operations: true`.
+      # @return [Array<Integer>, nil]
+      attr_reader :authorized_operations
+
+      # @param result_ptr [FFI::Pointer] pointer to the `rd_kafka_DescribeCluster_result_t`
+      def initialize(result_ptr)
+        @nodes = []
+
+        return if result_ptr.null?
+
+        cluster_id_ptr = Bindings.rd_kafka_DescribeCluster_result_cluster_id(result_ptr)
+        @cluster_id = cluster_id_ptr.null? ? nil : cluster_id_ptr.read_string
+        @controller = extract_node(Bindings.rd_kafka_DescribeCluster_result_controller(result_ptr))
+
+        count_ptr = FFI::MemoryPointer.new(:size_t)
+        @nodes = extract_nodes(Bindings.rd_kafka_DescribeCluster_result_nodes(result_ptr, count_ptr), count_ptr)
+
+        count_ptr = FFI::MemoryPointer.new(:size_t)
+        @authorized_operations = extract_authorized_operations(
+          Bindings.rd_kafka_DescribeCluster_result_authorized_operations(result_ptr, count_ptr),
+          count_ptr
+        )
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/describe_consumer_groups_handle.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Handle for describe consumer groups operation
+    class DescribeConsumerGroupsHandle < AbstractHandle
+      layout :pending, :bool,
+        :response, :int
+
+      # @return [String] the name of the operation
+      def operation_name
+        "describe consumer groups"
+      end
+
+      # @return [DescribeConsumerGroupsReport] report prepared by the background event callback,
+      #   with the described consumer groups.
+      def create_result
+        prepared_result
+      end
+
+      # Raises an error if the operation failed
+      # @raise [RdkafkaError]
+      def raise_error
+        raise RdkafkaError.new(
+          self[:response],
+          broker_message: broker_message
+        )
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/describe_consumer_groups_report.rb

@@ -0,0 +1,151 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Report for describe consumer groups operation result
+    class DescribeConsumerGroupsReport
+      include DescribeResultParsing
+
+      # Described consumer groups, one per requested group id. Each entry is a hash with:
+      #   - `:group_id` [String] the consumer group id
+      #   - `:error` [RdkafkaError, nil] error describing this group, or `nil` on success. The
+      #     other fields are not meaningful when it is set.
+      #   - `:is_simple_consumer_group` [Boolean] `true` for a group that assigns partitions
+      #     manually and uses Kafka only for offset storage
+      #   - `:state` [Integer] the group state as a `Bindings::RD_KAFKA_CONSUMER_GROUP_STATE_*`
+      #     code. A group id the cluster does not know is reported as
+      #     `RD_KAFKA_CONSUMER_GROUP_STATE_DEAD` with no members.
+      #   - `:state_name` [String] human-readable name of that state (e.g. `"Stable"`, `"Empty"`)
+      #   - `:type` [Integer] the group protocol type as a
+      #     `Bindings::RD_KAFKA_CONSUMER_GROUP_TYPE_*` code, one of:
+      #     `RD_KAFKA_CONSUMER_GROUP_TYPE_CLASSIC`,
+      #     `RD_KAFKA_CONSUMER_GROUP_TYPE_CONSUMER` (KIP-848),
+      #     `RD_KAFKA_CONSUMER_GROUP_TYPE_UNKNOWN`
+      #   - `:type_name` [String] human-readable name of that type (e.g. `"Classic"`, `"Consumer"`)
+      #   - `:partition_assignor` [String, nil] partition assignor in use (e.g. `"range"`)
+      #   - `:coordinator` [Hash, nil] the group coordinator broker as
+      #     `{ id:, host:, port:, rack: }`
+      #   - `:authorized_operations` [Array<Integer>, nil] ACL operations the client may perform on
+      #     the group, as `Bindings::RD_KAFKA_ACL_OPERATION_*` codes. `nil` unless requested with
+      #     `include_authorized_operations: true`.
+      #   - `:members` [Array<Hash>] current group members, each with:
+      #     - `:member_id` [String] the member (consumer) id
+      #     - `:client_id` [String] the member's `client.id`
+      #     - `:group_instance_id` [String, nil] the static membership instance id, if any
+      #     - `:host` [String] the host the member connects from
+      #     - `:assignment` [Hash{String => Array<Integer>}] assigned partitions, by topic
+      #     - `:target_assignment` [Hash{String => Array<Integer>}, nil] partitions the member is
+      #       moving to during a rebalance, by topic. Only `consumer` (KIP-848) groups have it;
+      #       `nil` for `classic` groups.
+      # @return [Array<Hash>]
+      attr_reader :groups
+
+      # @param result_ptr [FFI::Pointer] pointer to the `rd_kafka_DescribeConsumerGroups_result_t`
+      def initialize(result_ptr)
+        @groups = []
+
+        return if result_ptr.null?
+
+        count_ptr = FFI::MemoryPointer.new(:size_t)
+        array_ptr = Bindings.rd_kafka_DescribeConsumerGroups_result_groups(result_ptr, count_ptr)
+
+        return if array_ptr.null?
+
+        array_ptr.read_array_of_pointer(count_ptr.read(:size_t)).each do |group_ptr|
+          @groups << build_group(group_ptr)
+        end
+      end
+
+      private
+
+      # @param group_ptr [FFI::Pointer] pointer to the `rd_kafka_ConsumerGroupDescription_t`
+      # @return [Hash]
+      def build_group(group_ptr)
+        state = Bindings.rd_kafka_ConsumerGroupDescription_state(group_ptr)
+        type = Bindings.rd_kafka_ConsumerGroupDescription_type(group_ptr)
+
+        {
+          group_id: read_string(Bindings.rd_kafka_ConsumerGroupDescription_group_id(group_ptr)),
+          error: build_error(Bindings.rd_kafka_ConsumerGroupDescription_error(group_ptr)),
+          is_simple_consumer_group:
+            Bindings.rd_kafka_ConsumerGroupDescription_is_simple_consumer_group(group_ptr) != 0,
+          state: state,
+          state_name: read_string(Bindings.rd_kafka_consumer_group_state_name(state)),
+          type: type,
+          type_name: read_string(Bindings.rd_kafka_consumer_group_type_name(type)),
+          partition_assignor:
+            read_string(Bindings.rd_kafka_ConsumerGroupDescription_partition_assignor(group_ptr)),
+          coordinator: extract_node(Bindings.rd_kafka_ConsumerGroupDescription_coordinator(group_ptr)),
+          authorized_operations: build_authorized_operations(group_ptr),
+          members: Array.new(Bindings.rd_kafka_ConsumerGroupDescription_member_count(group_ptr)) do |index|
+            build_member(Bindings.rd_kafka_ConsumerGroupDescription_member(group_ptr, index))
+          end
+        }
+      end
+
+      # @param member_ptr [FFI::Pointer] pointer to the `rd_kafka_MemberDescription_t`
+      # @return [Hash]
+      def build_member(member_ptr)
+        {
+          member_id: read_string(Bindings.rd_kafka_MemberDescription_consumer_id(member_ptr)),
+          client_id: read_string(Bindings.rd_kafka_MemberDescription_client_id(member_ptr)),
+          group_instance_id: read_string(Bindings.rd_kafka_MemberDescription_group_instance_id(member_ptr)),
+          host: read_string(Bindings.rd_kafka_MemberDescription_host(member_ptr)),
+          assignment: build_assignment(Bindings.rd_kafka_MemberDescription_assignment(member_ptr)),
+          target_assignment: build_target_assignment(member_ptr)
+        }
+      end
+
+      # @param assignment_ptr [FFI::Pointer] pointer to the `rd_kafka_MemberAssignment_t`
+      # @return [Hash{String => Array<Integer>}]
+      def build_assignment(assignment_ptr)
+        return {} if assignment_ptr.null?
+
+        tpl_ptr = Bindings.rd_kafka_MemberAssignment_partitions(assignment_ptr)
+
+        return {} if tpl_ptr.null?
+
+        Consumer::TopicPartitionList
+          .from_native_tpl(tpl_ptr)
+          .to_h
+          .transform_values { |partitions| (partitions || []).map(&:partition) }
+      end
+
+      # @param member_ptr [FFI::Pointer] pointer to the `rd_kafka_MemberDescription_t`
+      # @return [Hash{String => Array<Integer>}, nil]
+      def build_target_assignment(member_ptr)
+        assignment_ptr = Bindings.rd_kafka_MemberDescription_target_assignment(member_ptr)
+
+        assignment_ptr.null? ? nil : build_assignment(assignment_ptr)
+      end
+
+      # @param group_ptr [FFI::Pointer] pointer to the `rd_kafka_ConsumerGroupDescription_t`
+      # @return [Array<Integer>, nil]
+      def build_authorized_operations(group_ptr)
+        count_ptr = FFI::MemoryPointer.new(:size_t)
+
+        extract_authorized_operations(
+          Bindings.rd_kafka_ConsumerGroupDescription_authorized_operations(group_ptr, count_ptr),
+          count_ptr
+        )
+      end
+
+      # @param error_ptr [FFI::Pointer] pointer to the `rd_kafka_error_t`
+      # @return [RdkafkaError, nil]
+      def build_error(error_ptr)
+        return nil if error_ptr.null?
+
+        RdkafkaError.new(
+          Bindings.rd_kafka_error_code(error_ptr),
+          broker_message: read_string(Bindings.rd_kafka_error_string(error_ptr))
+        )
+      end
+
+      # @param ptr [FFI::Pointer] pointer to a C string
+      # @return [String, nil]
+      def read_string(ptr)
+        ptr.null? ? nil : ptr.read_string
+      end
+    end
+  end
+end

data/lib/rdkafka/admin/describe_result_parsing.rb

@@ -0,0 +1,47 @@
+# frozen_string_literal: true
+
+module Rdkafka
+  class Admin
+    # Helpers shared by the describe-topics and describe-cluster reports to copy nodes and
+    # authorized operations out of event-owned memory.
+    # @private
+    module DescribeResultParsing
+      private
+
+      # @param node_ptr [FFI::Pointer] pointer to a `rd_kafka_Node_t`
+      # @return [Hash, nil] node with `:id`, `:host`, `:port` and `:rack` or nil when NULL
+      def extract_node(node_ptr)
+        return nil if node_ptr.null?
+
+        host_ptr = Bindings.rd_kafka_Node_host(node_ptr)
+        rack_ptr = Bindings.rd_kafka_Node_rack(node_ptr)
+
+        {
+          id: Bindings.rd_kafka_Node_id(node_ptr),
+          host: host_ptr.null? ? nil : host_ptr.read_string,
+          port: Bindings.rd_kafka_Node_port(node_ptr),
+          rack: rack_ptr.null? ? nil : rack_ptr.read_string
+        }
+      end
+
+      # @param array_ptr [FFI::Pointer] pointer to a `rd_kafka_Node_t*` array
+      # @param count_ptr [FFI::MemoryPointer] size_t holding the array length
+      # @return [Array<Hash>] nodes as returned by {#extract_node}
+      def extract_nodes(array_ptr, count_ptr)
+        return [] if array_ptr.null?
+
+        array_ptr.read_array_of_pointer(count_ptr.read(:size_t)).map { |node_ptr| extract_node(node_ptr) }
+      end
+
+      # @param array_ptr [FFI::Pointer] pointer to a `rd_kafka_AclOperation_t` array
+      # @param count_ptr [FFI::MemoryPointer] size_t holding the array length
+      # @return [Array<Integer>, nil] `Bindings::RD_KAFKA_ACL_OPERATION_*` codes or nil when the
+      #   authorized operations were not requested
+      def extract_authorized_operations(array_ptr, count_ptr)
+        return nil if array_ptr.null?
+
+        array_ptr.read_array_of_int(count_ptr.read(:size_t))
+      end
+    end
+  end
+end