pgmq-ruby 0.3.0 → 0.5.0

data/CLAUDE.md

@@ -0,0 +1,310 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+PGMQ-Ruby is a **low-level Ruby client** for PGMQ (PostgreSQL Message Queue), analogous to how rdkafka-ruby relates to Kafka. It provides direct 1:1 wrappers for PGMQ SQL functions as a thin transport layer.
+
+This is **NOT** a full job processing framework. Framework features (instrumentation, job processing, Rails ActiveJob, retry strategies, monitoring integrations) belong in the planned `pgmq-framework` gem.
+
+**Key Design Principles:**
+- Thin wrapper around PGMQ SQL functions (low-level primitives only)
+- Thread-safe connection pooling (transport layer)
+- PostgreSQL transaction support (database primitive, not framework abstraction)
+- Framework-agnostic (works with Rails, Sinatra, plain Ruby)
+- Minimal dependencies (only `pg` and `connection_pool`)
+- No instrumentation/observability layer (framework concern)
+
+## Development Commands
+
+### Testing
+
+```bash
+# Start PostgreSQL with PGMQ extension (required for tests)
+docker compose up -d
+
+# Run all tests
+bundle exec rspec
+
+# Run specific test file
+bundle exec rspec spec/unit/client_spec.rb
+
+# Run integration tests only
+bundle exec rspec spec/integration
+
+# Run single test by line number
+bundle exec rspec spec/unit/client_spec.rb:42
+```
+
+**Important:** PostgreSQL runs on port **5433** locally (see docker-compose.yml) to avoid conflicts with system PostgreSQL. Tests use this port automatically.
+
+### Code Quality
+
+```bash
+# Run tests
+bundle exec rspec
+```
+
+
+### Interactive Development
+
+```bash
+# Start IRB console with PGMQ loaded
+bundle exec bin/console
+
+# Console automatically connects to test database at localhost:5433
+```
+
+## Architecture
+
+### Core Components
+
+1. **PGMQ::Client** (`lib/pgmq/client.rb`)
+   - Main interface for all PGMQ operations (modular architecture, ~130 lines)
+   - Delegates connection management to PGMQ::Connection
+   - Validates queue names (48 char max, PostgreSQL identifier rules)
+   - Composed of 8 functional modules for separation of concerns:
+     - **Transaction** (`lib/pgmq/transaction.rb`) - PostgreSQL transaction support
+     - **QueueManagement** (`lib/pgmq/client/queue_management.rb`) - Queue lifecycle (create, drop, list)
+     - **Producer** (`lib/pgmq/client/producer.rb`) - Message sending operations
+     - **Consumer** (`lib/pgmq/client/consumer.rb`) - Single-queue reading operations
+     - **MultiQueue** (`lib/pgmq/client/multi_queue.rb`) - Multi-queue operations (UNION ALL)
+     - **MessageLifecycle** (`lib/pgmq/client/message_lifecycle.rb`) - Message state transitions (pop, delete, archive, set_vt)
+     - **Maintenance** (`lib/pgmq/client/maintenance.rb`) - Queue maintenance (purge, notifications)
+     - **Metrics** (`lib/pgmq/client/metrics.rb`) - Monitoring and metrics
+   - Benefits: Each module can be tested independently, easier maintenance, clear separation of concerns
+   - Pattern inspired by Waterdrop's modular client architecture
+
+2. **PGMQ::Connection** (`lib/pgmq/connection.rb`)
+   - Thread-safe connection pooling using `connection_pool` gem
+   - Supports multiple connection strategies:
+     - Connection strings: `postgres://user:pass@host/db`
+     - Hash parameters: `{ host:, port:, dbname:, user:, password: }`
+     - Callables (for Rails): `-> { ActiveRecord::Base.connection.raw_connection }`
+   - Auto-reconnect on connection failures (configurable)
+   - Connection health checks before use
+   - Note: Connection parameters are required - users should manage ENV variables themselves using `ENV.fetch`
+
+3. **PGMQ::Transaction** (`lib/pgmq/transaction.rb`)
+   - Low-level PostgreSQL transaction support (database primitive)
+   - Wraps PostgreSQL's native transactions (NOT a framework abstraction)
+   - Analogous to rdkafka-ruby providing Kafka transaction support
+   - Mixin providing `client.transaction do |txn|` support
+   - Enables atomic operations across multiple queues
+   - Rolls back automatically on errors
+
+4. **Models**
+   - **PGMQ::Message** - Represents queue messages with `msg_id`, `message` (raw JSONB), `read_ct`, `enqueued_at`, `vt`, `queue_name` (for multi-queue ops)
+   - **PGMQ::Metrics** - Queue metrics (length, age, total messages)
+   - **PGMQ::QueueMetadata** - Queue information (name, creation time)
+
+### Connection Pooling Details
+
+- Default pool size: 5 connections
+- Default timeout: 5 seconds
+- Fiber-aware (works with Ruby 3.0+ Fiber Scheduler)
+- Auto-reconnect enabled by default (can be disabled)
+- Connection health verified before each use (if auto-reconnect enabled)
+
+### Error Hierarchy
+
+```
+PGMQ::Errors::BaseError (StandardError)
+├── PGMQ::Errors::ConnectionError
+├── PGMQ::Errors::QueueNotFoundError
+├── PGMQ::Errors::MessageNotFoundError
+├── PGMQ::Errors::SerializationError
+├── PGMQ::Errors::ConfigurationError
+└── PGMQ::Errors::InvalidQueueNameError
+```
+
+### Queue Name Validation
+
+- Maximum 48 characters (PGMQ limitation for table prefixes)
+- Must start with letter or underscore
+- Only letters, digits, underscores allowed
+- Case-sensitive
+- Validated in `Client#validate_queue_name!`
+
+### Modular Architecture Pattern
+
+The Client class follows Waterdrop's modular architecture pattern for better maintainability:
+
+**Structure:**
+- Small core `Client` class (~130 lines) that includes functional modules
+- Each module focuses on a single domain area (queue management, producer, consumer, etc.)
+- Modules access parent class helpers (`validate_queue_name!`, `with_connection`)
+- Complete backward compatibility - no API changes
+
+**Benefits:**
+- **Testability**: Each module can be tested independently
+- **Maintainability**: Smaller files, clearer boundaries (largest file ~210 lines)
+- **Discoverability**: Logical grouping makes finding methods easier
+- **Extensibility**: New modules can be added without bloating the core class
+
+**Module Organization:**
+1. Transaction support (existing mixin)
+2. Queue lifecycle operations (create, drop, list)
+3. Message production (sending)
+4. Single-queue consumption (reading)
+5. Multi-queue operations (efficient polling across queues)
+6. Message lifecycle (pop, delete, archive, visibility timeout)
+7. Maintenance operations (purge, detach archive)
+8. Metrics and monitoring
+
+All modules are automatically loaded via Zeitwerk.
+
+## Common Patterns
+
+### Client Initialization
+
+```ruby
+# Connection string (preferred)
+client = PGMQ::Client.new('postgres://localhost:5433/pgmq_test')
+
+# Connection hash
+client = PGMQ::Client.new(
+  host: 'localhost',
+  port: 5433,
+  dbname: 'pgmq_test',
+  user: 'postgres',
+  password: 'postgres'
+)
+
+# Connection hash using ENV variables (user manages ENV themselves)
+client = PGMQ::Client.new(
+  host: ENV.fetch('PG_HOST', 'localhost'),
+  port: ENV.fetch('PG_PORT', 5432).to_i,
+  dbname: ENV.fetch('PG_DATABASE', 'pgmq'),
+  user: ENV.fetch('PG_USER', 'postgres'),
+  password: ENV.fetch('PG_PASSWORD', 'postgres')
+)
+
+# Rails integration (reuses Rails connection pool)
+client = PGMQ::Client.new(-> { ActiveRecord::Base.connection.raw_connection })
+
+# Custom pool configuration
+client = PGMQ::Client.new(
+  'postgres://localhost/db',
+  pool_size: 10,
+  pool_timeout: 10,
+  auto_reconnect: false
+)
+```
+
+### Transaction Pattern
+
+```ruby
+# Atomic operations across queues
+client.transaction do |txn|
+  msg = txn.read('pending', vt: 30)
+  if msg
+    txn.produce('processed', msg.payload)
+    txn.delete('pending', msg.msg_id)
+  end
+end
+```
+
+### Long Polling Pattern
+
+```ruby
+# Efficient message consumption
+loop do
+  msg = client.read_with_poll('orders',
+    vt: 30,
+    max_poll_seconds: 5,
+    poll_interval_ms: 100
+  )
+  break unless msg
+
+  process(msg)
+  client.delete('orders', msg.msg_id)
+end
+```
+
+## Testing Guidelines
+
+### Test Structure
+
+- **Unit tests** (`spec/unit/`) - Test classes in isolation, mock database
+- **Integration tests** (`spec/integration/`) - Test full workflow with real PostgreSQL
+
+### Test Helpers
+
+Located in `spec/support/database_helpers.rb`:
+- Database cleanup between tests
+- Connection helpers
+- Common test utilities
+
+### Coverage Requirements
+
+- Minimum overall coverage: 80%
+- Minimum per-file coverage: 70%
+- SimpleCov configured in `spec/spec_helper.rb`
+
+### Writing Tests
+
+- Use descriptive test names
+- Follow AAA pattern (Arrange, Act, Assert)
+- Mock external dependencies in unit tests
+- Use real database for integration tests
+- Clean up queues in `after` blocks
+
+## Code Style
+
+- Ruby 3.2+ syntax
+- Frozen string literals (`# frozen_string_literal: true`)
+- Max line length: 120 characters (except specs/gemspec)
+- YARD documentation for public methods
+- Single quotes for strings (unless interpolation needed)
+
+## CI/CD
+
+GitHub Actions workflows in `.github/workflows/`:
+- **ci.yml** - Runs RSpec on all Ruby versions (3.2, 3.3, 3.4, 3.5)
+- **push.yml** - Additional checks on push
+
+## Dependencies
+
+**Runtime:**
+- `pg` (~> 1.5) - PostgreSQL adapter
+- `connection_pool` (~> 2.4) - Thread-safe connection pooling
+
+**Development:**
+- `rspec` - Testing framework
+- `simplecov` - Code coverage
+- `pry` / `pry-byebug` - Debugging tools
+
+
+## Version Information
+
+- Minimum Ruby: 3.2.0
+- Supported PostgreSQL: 14-18 with PGMQ extension
+- License: LGPL-3.0
+
+## Important Files
+
+### Core Library Structure
+- `lib/pgmq/client.rb` - Main public API (~130 lines, includes all modules)
+- `lib/pgmq/client/` - Client functional modules directory:
+  - `queue_management.rb` - Queue lifecycle operations (~100 lines)
+  - `producer.rb` - Message sending (~70 lines)
+  - `consumer.rb` - Single-queue reading (~140 lines)
+  - `multi_queue.rb` - Multi-queue operations (~200 lines)
+  - `message_lifecycle.rb` - Message state transitions (~210 lines)
+  - `maintenance.rb` - Queue maintenance (~30 lines)
+  - `metrics.rb` - Monitoring (~40 lines)
+- `lib/pgmq/connection.rb` - Connection pooling and management
+- `lib/pgmq/transaction.rb` - Transaction support
+- `lib/pgmq/message.rb` - Message model (Data class)
+- `lib/pgmq/metrics.rb` - Metrics model
+- `lib/pgmq/queue_metadata.rb` - Queue metadata model
+
+### Documentation & Testing
+- `spec/spec_helper.rb` - RSpec configuration + SimpleCov setup
+- `spec/integration/` - Integration tests with real PostgreSQL
+- `spec/unit/` - Unit tests for individual components
+- `DEVELOPMENT.md` - Comprehensive development documentation
+- `README.md` - User-facing documentation with all API examples
+- `CLAUDE.md` - AI assistant guidance (this file)

data/Gemfile

@@ -1,15 +1,15 @@
 # frozen_string_literal: true
 
-source 'https://rubygems.org'
+source "https://rubygems.org"
 
 gemspec
 
 group :development, :test do
-  gem 'rake'
-  gem 'rspec'
-  gem 'yard-lint'
+  gem "rake"
+  gem "rspec"
+  gem "async", "~> 2.6" # Fiber Scheduler for concurrent I/O testing
 end
 
 group :test do
-  gem 'simplecov', require: false
+  gem "simplecov", require: false
 end

data/Gemfile.lint

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+# Documentation linting
+gem "yard-lint"
+
+# Code style (StandardRB via RuboCop)
+gem "standard"
+gem "standard-performance"
+gem "rubocop-performance"
+gem "rubocop-rspec"
+gem "standard-rspec"
+gem "rubocop-capybara"
+gem "rubocop-factory_bot"
+gem "rubocop-rspec_rails"

data/Gemfile.lint.lock

@@ -0,0 +1,120 @@
+GEM
+  remote: https://rubygems.org/
+  specs:
+    ast (2.4.3)
+    json (2.18.1)
+    language_server-protocol (3.17.0.5)
+    lint_roller (1.1.0)
+    parallel (1.27.0)
+    parser (3.3.10.2)
+      ast (~> 2.4.1)
+      racc
+    prism (1.9.0)
+    racc (1.8.1)
+    rainbow (3.1.1)
+    regexp_parser (2.11.3)
+    rubocop (1.84.2)
+      json (~> 2.3)
+      language_server-protocol (~> 3.17.0.2)
+      lint_roller (~> 1.1.0)
+      parallel (~> 1.10)
+      parser (>= 3.3.0.2)
+      rainbow (>= 2.2.2, < 4.0)
+      regexp_parser (>= 2.9.3, < 3.0)
+      rubocop-ast (>= 1.49.0, < 2.0)
+      ruby-progressbar (~> 1.7)
+      unicode-display_width (>= 2.4.0, < 4.0)
+    rubocop-ast (1.49.0)
+      parser (>= 3.3.7.2)
+      prism (~> 1.7)
+    rubocop-capybara (2.22.1)
+      lint_roller (~> 1.1)
+      rubocop (~> 1.72, >= 1.72.1)
+    rubocop-factory_bot (2.28.0)
+      lint_roller (~> 1.1)
+      rubocop (~> 1.72, >= 1.72.1)
+    rubocop-performance (1.26.1)
+      lint_roller (~> 1.1)
+      rubocop (>= 1.75.0, < 2.0)
+      rubocop-ast (>= 1.47.1, < 2.0)
+    rubocop-rspec (3.9.0)
+      lint_roller (~> 1.1)
+      rubocop (~> 1.81)
+    rubocop-rspec_rails (2.32.0)
+      lint_roller (~> 1.1)
+      rubocop (~> 1.72, >= 1.72.1)
+      rubocop-rspec (~> 3.5)
+    ruby-progressbar (1.13.0)
+    standard (1.54.0)
+      language_server-protocol (~> 3.17.0.2)
+      lint_roller (~> 1.0)
+      rubocop (~> 1.84.0)
+      standard-custom (~> 1.0.0)
+      standard-performance (~> 1.8)
+    standard-custom (1.0.2)
+      lint_roller (~> 1.0)
+      rubocop (~> 1.50)
+    standard-performance (1.9.0)
+      lint_roller (~> 1.1)
+      rubocop-performance (~> 1.26.0)
+    standard-rspec (0.4.0)
+      lint_roller (>= 1.0)
+      rubocop-capybara (~> 2.22)
+      rubocop-factory_bot (~> 2.27)
+      rubocop-rspec (~> 3.9)
+      rubocop-rspec_rails (~> 2.31)
+    unicode-display_width (3.2.0)
+      unicode-emoji (~> 4.1)
+    unicode-emoji (4.2.0)
+    yard (0.9.38)
+    yard-lint (1.4.0)
+      yard (~> 0.9)
+      zeitwerk (~> 2.6)
+    zeitwerk (2.7.4)
+
+PLATFORMS
+  ruby
+  x86_64-linux
+
+DEPENDENCIES
+  rubocop-capybara
+  rubocop-factory_bot
+  rubocop-performance
+  rubocop-rspec
+  rubocop-rspec_rails
+  standard
+  standard-performance
+  standard-rspec
+  yard-lint
+
+CHECKSUMS
+  ast (2.4.3) sha256=954615157c1d6a382bc27d690d973195e79db7f55e9765ac7c481c60bdb4d383
+  json (2.18.1) sha256=fe112755501b8d0466b5ada6cf50c8c3f41e897fa128ac5d263ec09eedc9f986
+  language_server-protocol (3.17.0.5) sha256=fd1e39a51a28bf3eec959379985a72e296e9f9acfce46f6a79d31ca8760803cc
+  lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87
+  parallel (1.27.0) sha256=4ac151e1806b755fb4e2dc2332cbf0e54f2e24ba821ff2d3dcf86bf6dc4ae130
+  parser (3.3.10.2) sha256=6f60c84aa4bdcedb6d1a2434b738fe8a8136807b6adc8f7f53b97da9bc4e9357
+  prism (1.9.0) sha256=7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85
+  racc (1.8.1) sha256=4a7f6929691dbec8b5209a0b373bc2614882b55fc5d2e447a21aaa691303d62f
+  rainbow (3.1.1) sha256=039491aa3a89f42efa1d6dec2fc4e62ede96eb6acd95e52f1ad581182b79bc6a
+  regexp_parser (2.11.3) sha256=ca13f381a173b7a93450e53459075c9b76a10433caadcb2f1180f2c741fc55a4
+  rubocop (1.84.2) sha256=5692cea54168f3dc8cb79a6fe95c5424b7ea893c707ad7a4307b0585e88dbf5f
+  rubocop-ast (1.49.0) sha256=49c3676d3123a0923d333e20c6c2dbaaae2d2287b475273fddee0c61da9f71fd
+  rubocop-capybara (2.22.1) sha256=ced88caef23efea53f46e098ff352f8fc1068c649606ca75cb74650970f51c0c
+  rubocop-factory_bot (2.28.0) sha256=4b17fc02124444173317e131759d195b0d762844a71a29fe8139c1105d92f0cb
+  rubocop-performance (1.26.1) sha256=cd19b936ff196df85829d264b522fd4f98b6c89ad271fa52744a8c11b8f71834
+  rubocop-rspec (3.9.0) sha256=8fa70a3619408237d789aeecfb9beef40576acc855173e60939d63332fdb55e2
+  rubocop-rspec_rails (2.32.0) sha256=4a0d641c72f6ebb957534f539d9d0a62c47abd8ce0d0aeee1ef4701e892a9100
+  ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33
+  standard (1.54.0) sha256=7a4b08f83d9893083c8f03bc486f0feeb6a84d48233b40829c03ef4767ea0100
+  standard-custom (1.0.2) sha256=424adc84179a074f1a2a309bb9cf7cd6bfdb2b6541f20c6bf9436c0ba22a652b
+  standard-performance (1.9.0) sha256=49483d31be448292951d80e5e67cdcb576c2502103c7b40aec6f1b6e9c88e3f2
+  standard-rspec (0.4.0) sha256=0fdf64c887cd6404f1c3a1435b14ba6fde2e9e80c0f4dafe4b04a67f673db262
+  unicode-display_width (3.2.0) sha256=0cdd96b5681a5949cdbc2c55e7b420facae74c4aaf9a9815eee1087cb1853c42
+  unicode-emoji (4.2.0) sha256=519e69150f75652e40bf736106cfbc8f0f73aa3fb6a65afe62fefa7f80b0f80f
+  yard (0.9.38) sha256=721fb82afb10532aa49860655f6cc2eaa7130889df291b052e1e6b268283010f
+  yard-lint (1.4.0) sha256=7dd88fbb08fd77cb840bea899d58812817b36d92291b5693dd0eeb3af9f91f0f
+  zeitwerk (2.7.4) sha256=2bef90f356bdafe9a6c2bd32bcd804f83a4f9b8bc27f3600fff051eb3edcec8b
+
+BUNDLED WITH
+  4.0.3

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    pgmq-ruby (0.3.0)
+    pgmq-ruby (0.5.0)
       connection_pool (~> 2.4)
       pg (~> 1.5)
       zeitwerk (~> 2.6)
@@ -9,9 +9,26 @@ PATH
 GEM
   remote: https://rubygems.org/
   specs:
+    async (2.36.0)
+      console (~> 1.29)
+      fiber-annotation
+      io-event (~> 1.11)
+      metrics (~> 0.12)
+      traces (~> 0.18)
     connection_pool (2.5.4)
+    console (1.34.3)
+      fiber-annotation
+      fiber-local (~> 1.1)
+      json
     diff-lcs (1.6.2)
     docile (1.4.1)
+    fiber-annotation (0.2.0)
+    fiber-local (1.1.0)
+      fiber-storage
+    fiber-storage (1.0.1)
+    io-event (1.14.2)
+    json (2.18.1)
+    metrics (0.15.0)
     pg (1.6.2)
     pg (1.6.2-aarch64-linux)
     pg (1.6.2-aarch64-linux-musl)
@@ -39,10 +56,7 @@ GEM
       simplecov_json_formatter (~> 0.1)
     simplecov-html (0.13.2)
     simplecov_json_formatter (0.1.4)
-    yard (0.9.37)
-    yard-lint (1.2.3)
-      yard (~> 0.9)
-      zeitwerk (~> 2.6)
+    traces (0.18.2)
     zeitwerk (2.7.3)
 
 PLATFORMS
@@ -55,11 +69,11 @@ PLATFORMS
   x86_64-linux-musl
 
 DEPENDENCIES
+  async (~> 2.6)
   pgmq-ruby!
   rake
   rspec
   simplecov
-  yard-lint
 
 BUNDLED WITH
    2.7.2