mammoth 0.9.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6333539be1c721f39698edbb27f8285e5b26755b3838dc3f342a198f93b4ff56
-  data.tar.gz: 1c825fea3c20668a1095e69a1931abd01f95bd8eb247365ce1b702e99bb40360
+  metadata.gz: fc13868a4aa6d845e63c655af36b9bbb69604be8bfb5b7ab6beda7d1a765c145
+  data.tar.gz: 9a3c9514953f68db2d5287903e29980ffb147460691817f5de1a19a8123f1727
 SHA512:
-  metadata.gz: d2470011cc7d16c9d76afde146d529dc7c56963ade4819e77fa75a81c43b7fc535ba57d7e6291eeea94547365a096e38db9182f0c7af01cc011f05cff81ef5a6
-  data.tar.gz: b06a8a8d35914d9d123fc7843e3c1318624ec5250c3b89656537680ae0bba9b22dfdf8b4bcb278769e3d49d4fabea4571d107119b19fe824b274e8345e45e0e4
+  metadata.gz: 552aa45078fa0ed209890bbfe8eb2dc0672234d4e008c8cc0edd954a284e0b40bffafe16a2a8c2df8dd0219aee3eed742ba2a38531cd3d2acd7b13b35bb48b11
+  data.tar.gz: 69f509480ecdd46aa50a7879c3dbc1589becce9a651b1697e1f164ea0d6be781edb9f54238940a32553b59956c8cba05cd2b0caaffca57812823cb32e6fa481b

data/CHANGELOG.md

@@ -2,6 +2,72 @@
 
 ## Unreleased
 
+## 1.0.0 - 2026-07-17
+
+### Added
+
+- Added PostgreSQL slot readiness diagnostics and Prometheus gauges for
+  inspection availability, presence, activity, retained WAL bytes, safe WAL
+  size, WAL status, invalidation, inactivity time, and restart/flush LSNs.
+- `/readyz` now fails closed when the configured slot is missing, inactive,
+  invalidated, conflicting, WAL-lost, or cannot be inspected.
+- Added startup validation for every configured publication table that publishes
+  `UPDATE` or `DELETE`, accepting a usable primary key, selected replica-identity
+  index, or `REPLICA IDENTITY FULL` and reporting all invalid tables together.
+- Added catalog-derived, ordered replica-identity mappings for relation OIDs and
+  schema-qualified tables.
+- Added a self-verifying live PostgreSQL example for composite, non-`id`
+  replica identities across `INSERT`, `UPDATE`, and `DELETE`.
+- Added a live PostgreSQL example that shows fail-closed restart after slot
+  invalidation and the explicit operator reconciliation required to resume.
+- Added a PostgreSQL observability example correlating Mammoth readiness and
+  Prometheus slot gauges with native replication and publication catalogs.
+- Added a live additive schema-evolution example demonstrating consumer-first
+  compatibility, relation-metadata refresh, and the non-delivery of DDL.
+- Added a destination idempotency example proving atomic receiver-side
+  duplicate suppression across isolated Mammoth delivery ledgers.
+
+### Fixed
+
+- Added PostgreSQL slot preflight before streaming and fail-closed handling for
+  missing, active, lost, invalidated, incompatible, and
+  checkpoint-unreachable slots.
+- Disabled automatic slot recreation whenever a configured or persisted resume
+  LSN requires WAL continuity.
+- Rejected temporary-slot recovery from durable checkpoints.
+- Preserved pgoutput-client's acknowledgement-compatible transport LSN
+  separately from normalized CDC-core `commit_lsn` payload context.
+- Checkpoints and PostgreSQL feedback now use the source-owned transport
+  watermark, preventing decimal transaction commit values from reaching
+  `runner.ack`.
+- Event-mode delivery resolves an envelope's transport position for each exact
+  child event without mutating or rebuilding CDC-core work.
+- Wired publication identity mappings into
+  `Pgoutput::SourceAdapter::ReplicaIdentityResolver` so composite and non-`id`
+  keys are normalized without partial identity data.
+
+### Quality
+
+- Upgraded pgoutput-client to 0.4.0 and covered slot identity, health,
+  retained-WAL reporting, continuity boundaries, and safe first-time creation.
+- Upgraded pgoutput-source-adapter to 0.3.0 and covered catalog-to-resolver
+  composition for composite replica identities.
+- Upgraded pgoutput-decoder to 0.2.0 so key-only old tuples preserve their
+  exact decoded columns instead of synthesizing absent non-key values.
+- Added end-to-end source/coordinator coverage proving that a transaction with
+  decimal `commit_lsn` checkpoints and acknowledges its formatted transport LSN.
+- Added opt-in PostgreSQL 17 integration coverage for transactional `INSERT`,
+  `UPDATE`, and `DELETE` with composite identity, replication reconnects,
+  concurrent out-of-order completion, contiguous slot acknowledgement, and
+  fail-closed slot invalidation; CI and release gates run the suite against a
+  logical-replication service.
+- Strengthened boundary coverage for source-owned position resolution and
+  publication-catalog inspection, and synchronized PostgreSQL, runtime,
+  operational-state, configuration, and transaction-example documentation.
+- Documented PostgreSQL WAL guardrails, slot/catalog monitoring, DDL and
+  sequence limitations, destination conflict ownership, upgrade recovery,
+  troubleshooting, example caveats, and Kubernetes deployment policy.
+
 ## 0.9.0
 
 ### Changed

data/README.md

@@ -39,11 +39,12 @@ API documentation:
 https://kanutocd.github.io/mammoth/Mammoth.html
 
 
-## OSS MVP
+## v1.0 Release Scope
 
-Mammoth OSS includes:
+Mammoth 1.0 includes:
 
-- CLI foundation
+- operator CLI for validation, bootstrap, status, delivery, observability, and
+  dead-letter workflows
 - YAML configuration loading
 - JSON Schema-backed configuration validation
 - SQLite operational memory bootstrap
@@ -54,19 +55,60 @@ Mammoth OSS includes:
 - webhook fanout to multiple destinations
 - fanout route filters by schema, table, and operation
 - per-destination enable/disable and retry policy controls
-- delivery worker with retry, delivered-ledger, checkpoint, and DLQ handling
+- delivery worker with retry, delivered-ledger, and DLQ handling
+- contiguous delivery watermark for checkpoint and PostgreSQL acknowledgement
+- source-owned transport LSN preservation independent of payload `commit_lsn`
+- fail-closed PostgreSQL slot and checkpoint continuity preflight
+- fail-closed publication replica-identity preflight for `UPDATE` and `DELETE`
 - dead-letter inspection and filtered replay commands
 - CDC-core event serialization boundary
 - CDC Ecosystem source-adapter integration boundary
 - Docker image support
 - public Helm chart support
 - unit and e2e test tasks
-- health and metrics endpoints
+- health, PostgreSQL slot readiness, and retained-WAL metrics endpoints
 - canonical CDC dispatch counters through a `CDC::Core::Observer`
 - explicit extension registries for state, destination, and runtime adapters
 - node identity and local capability reporting
 - lifecycle hooks, configuration providers, and reusable local command objects
 
+## Feature Examples
+
+The runnable examples are organized around production behaviors and failure
+modes, not isolated API snippets.
+
+| Example | v1 capability demonstrated |
+| --- | --- |
+| [`live_postgres_webhook`](examples/live_postgres_webhook) | End-to-end PostgreSQL logical replication into webhook delivery. |
+| [`transaction_webhook`](examples/transaction_webhook) | TransactionEnvelope preservation through the concurrent runtime. |
+| [`webhook_fanout`](examples/webhook_fanout) | Routed multi-destination fanout, environment-backed headers, signing, and independent retry policies. |
+| [`ordering`](examples/ordering) | Ordered and throughput-oriented transaction scheduling. |
+| [`checkpoint_recovery`](examples/checkpoint_recovery) | Durable restart recovery, replay suppression, checkpointing, and acknowledgement. |
+| [`slot_invalidation_recovery`](examples/slot_invalidation_recovery) | Fail-closed slot invalidation and explicit operator reconciliation. |
+| [`composite_replica_identity`](examples/composite_replica_identity) | Composite, non-`id` replica identity across `INSERT`, `UPDATE`, and `DELETE`. |
+| [`postgres_observability`](examples/postgres_observability) | Slot readiness and Prometheus metrics correlated with PostgreSQL catalogs. |
+| [`schema_evolution`](examples/schema_evolution) | Consumer-first additive schema evolution without implying DDL delivery. |
+| [`destination_idempotency`](examples/destination_idempotency) | Atomic destination-side duplicate suppression across isolated relay ledgers. |
+| [`failing_webhook_retry`](examples/failing_webhook_retry) | Retry exhaustion and durable dead-letter persistence. |
+| [`operational_state`](examples/operational_state) | Inspectable checkpoints, delivered ledgers, and dead letters. |
+| [`kubernetes_helm`](examples/kubernetes_helm) | Single-consumer Kubernetes deployment using the public Helm chart. |
+
+See [`examples/README.md`](examples/README.md) for the complete index, boundary
+notes, and commands.
+
+## v1 Compatibility
+
+Mammoth 1.x treats its validated configuration, serialized webhook envelopes,
+documented CLI command behavior, and forward operational-state migrations as
+supported contracts. Compatible minor releases may add optional configuration
+or payload fields, but do not remove or reinterpret existing fields.
+
+Human-readable CLI formatting and PostgreSQL-derived row columns are not frozen:
+scripts should rely on documented exit behavior, while receivers must tolerate
+additive fields and coordinate source schema changes. See
+[`docs/COMPATIBILITY.md`](docs/COMPATIBILITY.md) for the complete promise and
+major-version boundaries.
+
 ## Boundary
 
 Mammoth begins at CDC-core work items and ends at webhook fanout delivery.
@@ -82,7 +124,9 @@ For the live PostgreSQL stream, `pgoutput-source-adapter` incrementally owns
 `Begin`/`Commit` buffering and emits exact `CDC::Core::ChangeEvent` or
 `CDC::Core::TransactionEnvelope` work items. Mammoth only composes the
 transport, parser, decoder, and source adapter and forwards the resulting core
-work to delivery.
+work to delivery. Mammoth's publication preflight supplies ordered,
+catalog-derived replica-identity columns to the adapter, which owns composite
+and non-`id` key extraction.
 
 At the downstream boundary, `Mammoth::DeliveryProcessor` implements
 `CDC::Core::Processor` and returns `CDC::Core::ProcessorResult`. Inline and
@@ -206,7 +250,17 @@ script/test-e2e
 ```
 
 The e2e task uses a real HTTP receiver, real SQLite database, and real
-filesystem paths.
+filesystem paths. Set `MAMMOTH_E2E_POSTGRES_URL` to include the real PostgreSQL
+logical-replication scenarios:
+
+```bash
+MAMMOTH_E2E_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:5432/mammoth_e2e \
+  bundle exec rake test:e2e
+```
+
+The PostgreSQL fixture must have `wal_level=logical`; its test role must be able
+to create publications and logical replication slots, terminate replication
+backends, and change `max_slot_wal_keep_size`.
 
 ## Kubernetes
 
@@ -225,6 +279,14 @@ helm install mammoth charts/mammoth
 The chart uses one replica and `Recreate` strategy to respect PostgreSQL's
 logical replication slot constraint: one slot, one active subscriber.
 
+Production operators should also monitor retained WAL and slot readiness,
+configure PostgreSQL retention guardrails, and alert on database disk and
+catalog health. DDL and sequence state are not replicated; coordinate schema
+changes with webhook consumers and synchronize sequences externally when
+building a writable database copy. See
+[`docs/POSTGRESQL.md`](docs/POSTGRESQL.md) and
+[`docs/TROUBLESHOOTING.md`](docs/TROUBLESHOOTING.md).
+
 ## License
 
 Mammoth OSS is licensed under the [MIT License](LICENSE.txt).

data/config/mammoth.example.yml

@@ -89,6 +89,8 @@ replication:
   #
   # Publications define which table changes PostgreSQL sends to Mammoth.
   # Create them explicitly in production so the database boundary is clear.
+  # Tables that publish UPDATE or DELETE must have a primary key, an eligible
+  # REPLICA IDENTITY USING INDEX, or REPLICA IDENTITY FULL.
   #
   # Example SQL:
   #   CREATE PUBLICATION mammoth_publication FOR TABLE users, orders;
@@ -98,7 +100,9 @@ replication:
   # Optional start LSN for advanced recovery or controlled replay.
   #
   # Leave unset for normal operation. Mammoth will use its checkpoint store and
-  # the replication slot position.
+  # the replication slot position. Persisted PostgreSQL checkpoints use the
+  # acknowledgement-compatible transport LSN, not a normalized transaction's
+  # decimal commit_lsn.
   #
   # Example:
   #   start_lsn: "0/16B6C50"
@@ -108,10 +112,15 @@ replication:
   # Automatically create the replication slot if it does not exist.
   #
   # true:
-  #   Convenient for development, demos, and first-time bootstrap.
+  #   Convenient for development, demos, and first-time bootstrap. Mammoth
+  #   creates a missing slot only when no configured or persisted resume LSN
+  #   requires continuity.
   #
   # false:
   #   Recommended for production when slots are DBA-managed infrastructure.
+  #
+  # A missing slot is never recreated while a checkpoint exists. Mammoth fails
+  # closed because a replacement slot cannot recover the lost WAL interval.
   auto_create_slot: false
 
   # Use a temporary replication slot.
@@ -127,8 +136,9 @@ replication:
 
   # Standby feedback interval in seconds.
   #
-  # Mammoth sends periodic feedback while streaming so PostgreSQL knows the
-  # consumer is alive and can advance slot state safely.
+  # Mammoth sends periodic feedback while streaming. The feedback position
+  # advances only after every earlier delivery has a durable outcome and the
+  # same contiguous position has been written to Mammoth's checkpoint store.
   #
   # Lower values:
   #   faster liveness/feedback visibility, slightly more network chatter
@@ -145,13 +155,13 @@ delivery:
   # Delivery granularity.
   #
   # event:
-  #   Deliver each change event independently. This has lower latency but weaker
-  #   transaction-level semantics.
+  #   Deliver each change event independently. This has lower latency but does
+  #   not give consumers one atomic transaction payload. Checkpoint and WAL
+  #   acknowledgement still wait for every event in the transaction.
   #
   # transaction:
-  #   Deliver a complete TransactionEnvelope as one payload. This is the safer
-  #   default because checkpointing can advance only after the whole transaction
-  #   has been delivered successfully.
+  #   Deliver a complete TransactionEnvelope as one payload. This is recommended
+  #   when downstream consumers need transaction-level context.
   #
   # Recommended:
   #   transaction

data/config/mammoth.schema.json

@@ -141,7 +141,7 @@
             "type": "string",
             "minLength": 1
           },
-          "description": "PostgreSQL publications to subscribe to. Publications define which table changes PostgreSQL sends to Mammoth."
+          "description": "PostgreSQL publications to subscribe to. Publications define which table changes PostgreSQL sends to Mammoth. Tables publishing UPDATE or DELETE require a usable primary key, selected replica-identity index, or REPLICA IDENTITY FULL."
         },
         "start_lsn": {
           "type": [
@@ -152,7 +152,7 @@
         },
         "auto_create_slot": {
           "type": "boolean",
-          "description": "When true, Mammoth creates the replication slot if it does not exist. Convenient for development; DBA-managed slots are recommended for production."
+          "description": "When true, Mammoth creates a missing replication slot only for a fresh stream with no configured or persisted resume LSN. Mammoth never recreates a missing slot while checkpoint continuity is required."
         },
         "temporary_slot": {
           "type": "boolean",

data/lib/mammoth/application.rb

@@ -9,8 +9,10 @@ module Mammoth
   # composition stays outside this class so the application runtime consumes an
   # injected CDC work source rather than owning upstream CDC source-adapter
   # lifecycle decisions.
+  # rubocop:disable Metrics/ClassLength
   class Application
-    attr_reader :config, :state_adapter, :consumer, :delivery_worker, :checkpoint_store, :lifecycle_hooks, :observer
+    attr_reader :config, :state_adapter, :consumer, :delivery_worker, :checkpoint_store, :lifecycle_hooks, :observer,
+                :progress_coordinator
 
     # @param config [Mammoth::Configuration] loaded configuration
     # @param source [#each, nil] injectable event source for tests and demos
@@ -27,6 +29,7 @@ module Mammoth
       @state_adapter = state_adapter || build_state_adapter
       @checkpoint_store = @state_adapter.checkpoint_store
       @consumer = ReplicationConsumer.new(source: source || build_source, delivery_unit: delivery_unit)
+      @progress_coordinator = build_progress_coordinator
       @delivery_worker = sink ? build_delivery_worker(sink: sink, sleeper: sleeper) : build_configured_delivery_worker(sleeper:)
     end
 
@@ -72,12 +75,14 @@ module Mammoth
     def process_consumer(runtime)
       processed = 0
 
-      consumer.start do |work|
+      consumer.start_with_boundaries do |work, group_end|
+        progress_coordinator.register(work, group_end:)
         process_work(runtime, work)
         processed += 1
       end
 
       runtime.flush
+      progress_coordinator.finalize
       processed
     end
 
@@ -88,7 +93,11 @@ module Mammoth
     def build_runtime
       Runtimes::Registry.build(
         runtime_adapter,
-        processor: DeliveryProcessor.new(delivery_worker:, delivery_unit: delivery_unit),
+        processor: DeliveryProcessor.new(
+          delivery_worker:,
+          delivery_unit: delivery_unit,
+          progress_coordinator: progress_coordinator
+        ),
         concurrency: runtime_concurrency,
         timeout: runtime_timeout,
         preserve_order: runtime_preserve_order?,
@@ -101,6 +110,27 @@ module Mammoth
       Sources::Postgres.new(config, checkpoint_store: checkpoint_store)
     end
 
+    def build_progress_coordinator
+      DeliveryProgressCoordinator.new(
+        checkpoint_store: checkpoint_store,
+        source_name: config.dig("mammoth", "name"),
+        slot_name: config.dig("replication", "slot"),
+        publication_name: Array(config.dig("replication", "publications")).join(","),
+        acknowledger: source_acknowledger,
+        position_resolver: source_position_resolver
+      )
+    end
+
+    def source_acknowledger
+      source = consumer.source
+      source.method(:acknowledge) if source.respond_to?(:acknowledge)
+    end
+
+    def source_position_resolver
+      source = consumer.source
+      source.method(:progress_position_for) if source.respond_to?(:progress_position_for)
+    end
+
     def build_delivery_worker(sink:, sleeper:, delivery_policy: {})
       DeliveryWorker.from_config(
         config,
@@ -176,4 +206,5 @@ module Mammoth
       value.nil? || value
     end
   end
+  # rubocop:enable Metrics/ClassLength
 end

data/lib/mammoth/delivery_processor.rb

@@ -32,14 +32,16 @@ module Mammoth
 
     concurrent_safe!
 
-    attr_reader :delivery_worker, :delivery_unit
+    attr_reader :delivery_worker, :delivery_unit, :progress_coordinator
 
     # @param delivery_worker [Mammoth::DeliveryWorker] relay-aware delivery worker
     # @param delivery_unit [String, Symbol] event or transaction
-    def initialize(delivery_worker:, delivery_unit: :event)
+    # @param progress_coordinator [Mammoth::DeliveryProgressCoordinator, nil] contiguous progress coordinator
+    def initialize(delivery_worker:, delivery_unit: :event, progress_coordinator: nil)
       super()
       @delivery_worker = delivery_worker
       @delivery_unit = delivery_unit.to_sym
+      @progress_coordinator = progress_coordinator
     end
 
     # @return [Boolean] true when this processor instance is safe for concurrent execution.
@@ -63,6 +65,7 @@ module Mammoth
     # @return [CDC::Core::ProcessorResult] normalized processor result
     def process(work)
       summary = deliver(work)
+      progress_coordinator&.complete(work)
       build_result(work, summary)
     rescue StandardError => e
       failure_result(work, e, retryable: e.is_a?(DeliveryError))

data/lib/mammoth/delivery_progress_coordinator.rb

@@ -0,0 +1,153 @@
+# frozen_string_literal: true
+
+module Mammoth
+  # Coordinates durable delivery progress across out-of-order workers.
+  #
+  # Work is registered in source order before dispatch. Completion may arrive
+  # in any order, but a checkpoint and upstream acknowledgement advance only
+  # when every item in the oldest closed source group has a durable outcome.
+  # A source group is normally one PostgreSQL transaction.
+  class DeliveryProgressCoordinator
+    # Mutable source transaction tracked by the coordinator.
+    class Group
+      attr_reader :items
+      attr_accessor :closed
+
+      # @param items [Array<DeliveryProgressCoordinator::Entry>] registered work
+      # @param closed [Boolean] whether the source group is complete
+      def initialize(items:, closed:)
+        @items = items
+        @closed = closed
+      end
+    end
+
+    # Mutable completion state for one registered work item.
+    class Entry
+      attr_reader :work, :source_position
+      attr_accessor :completed
+
+      # @param work [Object] registered work
+      # @param source_position [String, Integer, nil] upstream position
+      # @param completed [Boolean] whether a durable outcome was recorded
+      def initialize(work:, source_position:, completed:)
+        @work = work
+        @source_position = source_position
+        @completed = completed
+      end
+    end
+
+    attr_reader :checkpoint_store, :source_name, :slot_name, :publication_name
+
+    # @param checkpoint_store [Mammoth::CheckpointStore] durable checkpoint persistence
+    # @param source_name [String] logical source name
+    # @param slot_name [String] PostgreSQL replication slot name
+    # @param publication_name [String] publication name
+    # @param acknowledger [#call, nil] upstream durable-progress acknowledgement
+    # @param position_resolver [#call, nil] source-owned durable position resolver
+    def initialize(checkpoint_store:, source_name:, slot_name:, publication_name:, acknowledger: nil,
+                   position_resolver: nil)
+      @checkpoint_store = checkpoint_store
+      @source_name = source_name
+      @slot_name = slot_name
+      @publication_name = publication_name
+      @acknowledger = acknowledger
+      @position_resolver = position_resolver
+      # rubocop:disable Layout/LeadingCommentSpace -- Steep inline type syntax requires `#:`.
+      @groups = [] #: Array[Group]
+      @entries_by_work = {} #: Hash[untyped, Array[Entry]]
+      # rubocop:enable Layout/LeadingCommentSpace
+      @entries_by_work.compare_by_identity
+      @mutex = Mutex.new
+    end
+
+    # Register work before dispatch.
+    #
+    # @param work [CDC::Core::ChangeEvent, CDC::Core::TransactionEnvelope] work item
+    # @param group_end [Boolean] whether this is the final item in its source group
+    # @return [void]
+    def register(work, group_end:)
+      @mutex.synchronize do
+        group = current_group
+        entry = Entry.new(work: work, source_position: source_position(work), completed: false)
+        group.items << entry
+        (@entries_by_work[work] ||= []) << entry # steep:ignore
+        group.closed = true if group_end
+      end
+      nil
+    end
+
+    # Mark a work item durably resolved and advance all newly contiguous groups.
+    #
+    # @param work [CDC::Core::ChangeEvent, CDC::Core::TransactionEnvelope] work item
+    # @return [String, nil] latest source position advanced by this call
+    def complete(work)
+      @mutex.synchronize do
+        entry = pending_entry_for(work)
+        raise ReplicationError, "delivery progress completed before registration" unless entry
+
+        entry.completed = true
+        advance_contiguous_groups
+      end
+    end
+
+    # Close and advance a final source group after a clean end-of-stream.
+    #
+    # @return [String, nil] latest source position advanced by this call
+    def finalize
+      @mutex.synchronize do
+        @groups.last.closed = true if @groups.last
+        advance_contiguous_groups
+      end
+    end
+
+    private
+
+    def current_group
+      return @groups.last if @groups.last && !@groups.last.closed
+
+      Group.new(items: [], closed: false).tap { |group| @groups << group }
+    end
+
+    def pending_entry_for(work)
+      @entries_by_work.fetch(work, []).find { |entry| !entry.completed } # steep:ignore
+    end
+
+    def advance_contiguous_groups
+      advanced_position = nil
+
+      while (group = @groups.first) && group.closed && group.items.all?(&:completed)
+        position = group.items.reverse_each.lazy.map(&:source_position).find { |value| !value.nil? }
+        persist_and_acknowledge(position) if position
+        advanced_position = position || advanced_position
+        remove_group(group)
+      end
+
+      advanced_position
+    end
+
+    def persist_and_acknowledge(position)
+      checkpoint_store.write(
+        source_name: source_name,
+        slot_name: slot_name,
+        publication_name: publication_name,
+        last_lsn: position
+      )
+      @acknowledger&.call(position)
+    end
+
+    def remove_group(group)
+      @groups.shift
+      group.items.each do |entry|
+        entries = @entries_by_work[entry.work]
+        entries.delete(entry)
+        @entries_by_work.delete(entry.work) if entries.empty?
+      end
+    end
+
+    def source_position(work)
+      return @position_resolver.call(work) if @position_resolver
+
+      work.commit_lsn if work.respond_to?(:commit_lsn)
+    end
+  end
+end

data/lib/mammoth/delivery_worker.rb

@@ -1,12 +1,12 @@
 # frozen_string_literal: true
 
 module Mammoth
-  # Delivers normalized events with retry, checkpoint, and dead-letter handling.
+  # Delivers normalized events with retry, ledger, and dead-letter handling.
   #
   # DeliveryWorker is Mammoth's first reliable delivery unit. It intentionally keeps
-  # the delivery contract small: attempt webhook delivery, advance the checkpoint
-  # after success, and persist the failed event to the dead letter queue after
-  # retry exhaustion.
+  # the delivery contract small: attempt webhook delivery, record idempotent
+  # success, and persist exhausted failures to the dead letter queue. Contiguous
+  # checkpointing is owned by DeliveryProgressCoordinator.
   class DeliveryWorker
     # Default source name used when an event does not provide one.
     DEFAULT_SOURCE = "postgresql"
@@ -15,7 +15,7 @@ module Mammoth
                 :sleeper, :source_name, :slot_name, :publication_name, :route_filter, :enabled
 
     # @param sink [#deliver] destination sink
-    # @param checkpoint_store [Mammoth::CheckpointStore] checkpoint persistence
+    # @param checkpoint_store [Mammoth::CheckpointStore] retained dependency; shared progress owns writes
     # @param dead_letter_store [Mammoth::DeadLetterStore] dead letter persistence
     # @param delivered_envelope_store [Mammoth::DeliveredEnvelopeStore] downstream delivery ledger
     # @param source_name [String] logical source name
@@ -47,7 +47,7 @@ module Mammoth
     #
     # @param config [Mammoth::Configuration] loaded configuration
     # @param sink [#deliver] destination sink
-    # @param checkpoint_store [Mammoth::CheckpointStore] checkpoint persistence
+    # @param checkpoint_store [Mammoth::CheckpointStore] retained dependency; shared progress owns writes
     # @param dead_letter_store [Mammoth::DeadLetterStore] dead letter persistence
     # @param delivered_envelope_store [Mammoth::DeliveredEnvelopeStore] downstream delivery ledger
     # @param sleeper [#call] sleep strategy
@@ -70,7 +70,7 @@ module Mammoth
       )
     end
 
-    # Deliver a transaction envelope with retry, checkpoint, and DLQ handling.
+    # Deliver a transaction envelope with retry, ledger, and DLQ handling.
     #
     # @param envelope [CDC::Core::TransactionEnvelope] CDC transaction envelope
     # @return [Hash] delivery summary
@@ -78,7 +78,7 @@ module Mammoth
       deliver_work(envelope, serializer: TransactionEnvelopeSerializer, delivery_method: :deliver_transaction)
     end
 
-    # Deliver an event with retry, checkpoint, and DLQ handling.
+    # Deliver an event with retry, ledger, and DLQ handling.
     #
     # @param event [CDC::Core::ChangeEvent] normalized event
     # @return [Hash] delivery summary
@@ -99,7 +99,6 @@ module Mammoth
       return skip_result if skip_result
 
       if delivered_envelope_store.delivered?(idempotency_key)
-        checkpoint_payload(payload)
         return {
           status: "skipped",
           duplicate: true,
@@ -121,7 +120,6 @@ module Mammoth
           transaction_id: payload["transaction_id"],
           source_position: payload["source_position"]
         )
-        checkpoint_payload(payload)
         result.merge(attempts: attempts, idempotency_key: idempotency_key)
       rescue DeliveryError => e
         return dead_letter(work, e, attempts, serializer:) if attempts >= max_attempts
@@ -132,19 +130,6 @@ module Mammoth
     end
     # rubocop:enable Metrics/MethodLength
 
-    def checkpoint(work, serializer:)
-      checkpoint_payload(serializer.call(work))
-    end
-
-    def checkpoint_payload(payload)
-      checkpoint_store.write(
-        source_name: source_name,
-        slot_name: slot_name,
-        publication_name: publication_name,
-        last_lsn: payload["source_position"]
-      )
-    end
-
     def idempotency_key_for(payload:, delivery_unit:)
       [
         source_name,
@@ -170,8 +155,7 @@ module Mammoth
       skipped(payload, idempotency_key:, reason: "route_mismatch") unless route_filter.match_payload?(payload)
     end
 
-    def skipped(payload, idempotency_key:, reason:)
-      checkpoint_payload(payload)
+    def skipped(_payload, idempotency_key:, reason:)
       {
         status: "skipped",
         reason: reason,

data/lib/mammoth/observability_metrics.rb

@@ -3,6 +3,8 @@
 module Mammoth
   # Prometheus formatting helpers for operational and dispatch metrics.
   module ObservabilityMetrics
+    include PostgresObservabilityMetrics
+
     # Maps canonical CDC core metric names to Mammoth's Prometheus counters.
     DISPATCH_METRIC_NAMES = {
       CDC::Core::Observer.started_metric_name => "mammoth_dispatch_started_total",
@@ -14,7 +16,7 @@ module Mammoth
     private
 
     def metric_headers
-      operational_metric_headers + dispatch_metric_headers
+      operational_metric_headers + dispatch_metric_headers + postgres_metric_headers
     end
 
     def operational_metric_headers