queen-mq 0.4.0 → 0.6.4

package/README.md

@@ -1,385 +1,632 @@
-# Queen MQ - PostgreSQL-backed C++ Message Queue
+# Queen MQ - JavaScript Client
 
 <div align="center">
 
-**A modern, performant message queue system built on PostgreSQL**
+**Modern, high-performance message queue client for Node.js**
 
+[![npm](https://img.shields.io/npm/v/queen-mq.svg)](https://www.npmjs.com/package/queen-mq)
 [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE.md)
 [![Node](https://img.shields.io/badge/node-%3E%3D22.0.0-brightgreen.svg)](https://nodejs.org/)
-[![C++](https://img.shields.io/badge/C%2B%2B-17-blue.svg)](https://en.cppreference.com/w/cpp/17)
 
-[Quick Start](#one-single-example) • [Complete V2 Guide](client-js/client-v2/README.md) • [Webapp](#webapp) • [Server Setup](#install-the-server-and-configure-it) • [HTTP API](#raw-http-api)
-
-<p align="center">
-  <img src="assets/queen-logo.svg" alt="Queen Logo" width="120" />
-</p>
+[Quick Start](#quick-start) • [Complete Guide](client-v2/README.md) • [Examples](#examples) • [API Reference](#api-reference)
 
 </div>
 
 ---
 
-Why "Queen"? Because years ago, when I first read the word "queue" in my mind, I read it as "queen".
+## What is Queen MQ?
+
+Queen MQ is a PostgreSQL-backed message queue system with a powerful feature set:
+
+- **FIFO Partitions** - Unlimited ordered partitions within queues
+- **Consumer Groups** - Kafka-style consumer groups for scalability
+- **Flexible Semantics** - Exactly-once, at-least-once, and at-most-once delivery
+- **Transactions** - Atomic operations across push and ack
+- **High Performance** - 200K+ messages/sec with proper batching
+- **Subscription Modes** - Process from beginning, new messages only, or from timestamp
+- **Dead Letter Queue** - Automatic failure handling and monitoring
+- **Message Tracing** - Debug distributed workflows with trace timelines
+- **Client-Side Buffering** - 10x-100x throughput boost for high-volume pushes
+- **Real-time Streaming** - Windowed aggregation and processing
+
+This client provides a fluent, promise-based API for Node.js applications.
 
 ---
 
-## Introduction
-
-QueenMQ is a queue system written in C++ and backed by PostgreSQL, born from the need to manage many FIFO partitions for Smartchat with solid guarantees around delivery and failure handling. During the initial development, I realized that with a few simple additions to the original design, I could build a very powerful and flexible queue system. This project is almost entirely written by AI, with my supervision—only the test files (or a good part of them) are manually written.
-
-Here are the main features:
-- Unlimited FIFO partitions within queues
-- Queue semantics like RabbitMQ
-- Consumer groups over queues like Kafka
-- Allows for all the patterns you need, from simple queues to complex workflows and request/response patterns
-- QoS levels: Exactly-once delivery (with transactionId), at-least-once delivery, and at-most-once delivery
-- Subscription modes for replay (new messages only or from a specific timestamp) and message history control
-- Transactions between operations (push and ack mainly) for atomicity
-- Dead letter queue for failure handling
-- Lease renewal for long-running tasks
-- Message tracing for debugging workflows
-- Encryption of messages at DB level
-- Automatic message retention and cleanup - Configurable per-queue retention policies
-- Streaming capabilities for real-time aggregation and processing of messages
-- A nice webapp for monitoring and managing the system
-- Maintenace mode that allows to continue pushing messages even when the database is down, and to drain the messages to the database when the maintenance mode is disabled
-
-The system consists of a PostgreSQL database, a replicated server that can be scaled horizontally (though it won't be the bottleneck), and a client library for interacting with the server. All client-server communication happens over HTTP, and the client library are available for JavaScript and C++ (Other languages like Python are planned). There's also a modern Vue 3 web app for monitoring and managing the system.
-
-With proper batching, the system can handle +200k **messages** per second (not req/s) on modest hardware.
-
-Main documentation:
-- [Client Guide JS](client-js/client-v2/README.md)
-- [Client Guide C++](client-cpp/README.md)
-- [Server Guide](server/README.md)
-- [Streaming Guide](docs/STREAMING_USAGE.md)
-- [API Reference](server/API.md)
-- [Message Retention & Cleanup](docs/RETENTION.md)
-- [Webapp](webapp/README.md)
-- [Expose the Webapp behind a proxy](proxy/README.md)
-
-## Concepts
-
-Although the system is designed to be simple to use (not really), there are some concepts that are important to understand to use the system effectively.
+## Installation
 
-### Queues
+```bash
+npm install queen-mq
+```
 
-Queues are a way to organize your messages into logical groups. Each queue is like a container for messages. You can have as many queues as you want, and you can configure each queue with different settings, like the lease time, retry limit, encryption, retention, priority, delayed processing, window buffer and retention.
+**Requirements:** Node.js 22+
 
-### Partitions
+---
 
-Partitions are a way to organize your messages into logical ordered groups. Each partition is like a separate queue inside a queue, and messages in the same partition are guaranteed to be processed in order. Only one consumer/client can acquire the lock for a partition at a time. You can have as many partitions as you want, and you can have as many consumers/clients as you want. If the lease in not acknowledged in time, the message is released and can be acquired by another consumer/client. Use renewLease() to renew the lease during long-running tasks. Each different consumer group can acquire the lock for a partition independently, so you can have as many consumer groups as you want.
+## Quick Start
 
-### Default partition and consumer group
+```javascript
+import { Queen } from 'queen-mq'
 
-The default partition (if not specified) is `Default`. If you don't specify a consumer group, the messages are processed in queue mode, that is creating a default consumer group named `__QUEUE_MODE__`. Each queue can be used at the same time by multiple consumer groups, and each consumer group can process messages from multiple partitions.
+// Connect to Queen server
+const queen = new Queen('http://localhost:6632')
 
-### Consumer groups
+// Create a queue
+await queen.queue('tasks').create()
 
-Consumer groups are a way to process messages for different purposes. Each consumer group is like a separate queue, and messages in the same consumer group partition are guaranteed to be processed in order. Each consumer group tracks its own position in the queue. You can start a consumer group from the beginning of the queue, from a specific timestamp, or only new messages.
+// Push messages
+await queen.queue('tasks').push([
+  { data: { task: 'send-email', to: 'alice@example.com' } }
+])
 
-### Subscription modes
+// Consume messages
+await queen.queue('tasks').consume(async (message) => {
+  console.log('Processing:', message.data)
+  // Auto-ack on success, auto-retry on error
+})
+```
 
-Subscription modes are a way to control the message history that is processed. You can choose to process all messages (including historical ones), only new messages, or messages from a specific timestamp. Subscription modes are only available when using consumer groups.
+---
 
-### Long polling (waiting for messages)
+## Core Concepts
 
-The client works with the pull model for pop operations, meaning that you need to explicitly request messages from the queue. Pop and consume mehtods can "wait" server side for messages to be available. When the method is called with wait=true, the method will block until messages are available or the timeout is reached. When the timeout is reached, the method returns an empty array. Long polling is a very efficient way to wait for messages, and it is the recommended way to consume messages.
+### Queues
 
-### Lease renewal
+Logical containers for messages with configurable settings:
+- **Lease time** - How long a consumer has to process a message
+- **Retry limit** - Number of retry attempts before DLQ
+- **Priority** - Queue priority for multi-queue consumers
+- **Encryption** - Message payload encryption at rest
+- **Retention** - Automatic cleanup policies
 
-Lease renewal is a way to keep the lock for a partition or consumer group alive. You can use lease renewal to prevent the lock from expiring and being acquired by another consumer/client.
+```javascript
+await queen.queue('orders')
+  .config({
+    leaseTime: 300,        // 5 minutes
+    retryLimit: 3,
+    priority: 5,
+    encryptionEnabled: false
+  })
+  .create()
+```
 
-### Ack and Nack
+### Partitions
 
-Ack and Nack are the way to acknowledge or not a message. Ack means that the message has been processed successfully, and Nack means that the message has not been processed successfully. Ack/Nack requires the partitionId, the transactionId and the leaseId to be specified. If the ack is coming from a consumer group, the consumer group name is also required. This logic is handled automatically by the client library, you don't need to worry about it. If the message has already been acknowledged, the ack will be ignored. Based on the configuration, the message can be automatically acknowledged or manually acknowledged.
+Ordered lanes within a queue. Messages in the same partition are processed sequentially:
 
-### Transactions
+```javascript
+// All messages for user-123 are processed in order
+await queen.queue('user-events')
+  .partition('user-123')
+  .push([
+    { data: { event: 'login' } },
+    { data: { event: 'view-page' } },
+    { data: { event: 'logout' } }
+  ])
+```
 
-Transactions are a way to ensure that a group of operations are atomic. You can use transactions to ensure that a group of operations are executed together, and if one of the operations fails, the entire transaction is rolled back. Transactions are useful to achieve the exactly-once guarantee. You can process a message on a queue, and forward to the next queue only if the consumer lease is still valid, concatenating ack and push into a single transaction.
+**Use cases:**
+- Per-user ordering
+- Per-tenant isolation
+- Sharding for parallelism
 
-### Dead letter queue
+### Consumer Groups
 
-The dead letter queue is a way to handle messages that are not processed successfully. When a message is not processed successfully, it is moved to the dead letter queue. Messages goes in the DLQ when they are NACK after the retry limit is reached. You configure the retry limit for each queue at queue config.
+Multiple consumers sharing work, with independent progress tracking:
 
-## Comparison with RabbitMQ, Kafka, and NATS
+```javascript
+// Worker 1 & 2 share the load
+await queen.queue('emails')
+  .group('processors')
+  .consume(async (message) => {
+    await sendEmail(message.data)
+  })
+
+// Separate group processes same messages independently
+await queen.queue('emails')
+  .group('analytics')
+  .consume(async (message) => {
+    await logMetrics(message.data)
+  })
+```
 
-For users familiar with existing message queue systems, here's how Queen's semantics and usage patterns compare:
+### Subscription Modes
 
-### Conceptual Mapping
+Control whether consumer groups process historical messages:
 
-| Queen Concept | RabbitMQ Equivalent | Kafka Equivalent | NATS Equivalent |
-|---------------|---------------------|------------------|-----------------|
-| Queue | Queue | Topic | Stream (JetStream) |
-| Partition | N/A (queues are single-consumer by default) | Partition | N/A |
-| Consumer Group | Competing Consumers pattern | Consumer Group | Queue Group |
-| Queue Mode (no group) | Exclusive consumer | N/A (always uses groups) | Single subscriber |
-| Lease | Message TTL / Visibility timeout | N/A (commit-based) | Ack wait / nak delay |
-| Ack/Nack | Ack/Nack | Commit offset | Ack/Nak |
-| Transaction | Publisher confirms + consumer acks | Transactional producer/consumer | N/A |
-| Dead Letter Queue | Dead Letter Exchange | N/A (manual) | N/A (manual) |
+```javascript
+// Default: Process ALL messages (including backlog)
+await queen.queue('events')
+  .group('batch-analytics')
+  .consume(async (message) => { /* all messages */ })
+
+// Skip history, only new messages
+await queen.queue('events')
+  .group('realtime-monitor')
+  .subscriptionMode('new')
+  .consume(async (message) => { /* new only */ })
+
+// Start from specific timestamp
+await queen.queue('events')
+  .group('replay')
+  .subscriptionFrom('2025-10-28T10:00:00.000Z')
+  .consume(async (message) => { /* from timestamp */ })
+```
 
+---
 
-## One single example
+## Connection Options
 
-> 📖 **[Complete Guide: client-js/client-v2/README.md](client-js/client-v2/README.md)** - Full tutorial with all features!
+### Single Server
 
 ```javascript
-import { Queen } from 'queen-mq'
-
-// Connect to Queen
 const queen = new Queen('http://localhost:6632')
+```
 
-// Create a queue
-await queen
-.queue('critical-task')
-.config({
-  leaseTime: 10, // 10 seconds to process the messages (seconds)
-})
-.create()
+### Multiple Servers (High Availability)
 
-// Push messages
-await queen
-.queue('critical-task')
-.partition('tenant-123')
-.push([
-  { 
-    transactionId: 'my-id', // This is autogenerated, but you can set your own. Unique per queue partition.
-    data: { id: 123, description: 'Critical task' }  // The message payload
-  },
+```javascript
+const queen = new Queen([
+  'http://server1:6632',
+  'http://server2:6632'
 ])
-.onSuccess(async (messages) => { // Not mandatory
-  console.log('Messages pushed successfully:', messages)
-})
-.onDuplicate(async (messages) => { // Not mandatory, triggered when a message with the same transactionId is pushed
-  console.warn('Duplicate transaction IDs detected')
-})
-.onError(async (messages, error) => {  // Without callbacks, push throws an error if some messages are not pushed
-  console.error('Error pushing messages:', error)
-})
+```
 
-// Consume messages
-await queen
-.queue('critical-task')
-// The consumer group name, without this, the messages are processed in queue mode
-.group('processor-consumer-group') 
-// 10 parallel workers
-.concurrency(10)
-// I want to manually ack/nack messages 
-.autoAck(false) 
-// 10 messages per batch, prefetch them
-.batch(10) 
-// Auto-renew the lease for the messages every 2 seconds
-.renewLease(true, 2000) 
-// Process each message individually
-.each() 
-// Do your work here
-.consume(async (message) => {
-  console.log('Processing:', message.data) 
-})
-// Ack the messages if you processed them successfully
-.onSuccess(async (message) => {
-  await queen
-  .transaction()
-  .queue('critical-task-next')
-  .partition('XXX')
-  .push([{ data: { message: 'Final', count: 3 } }])
-  .ack(message)
-  .commit()  
+### Full Configuration
+
+```javascript
+const queen = new Queen({
+  urls: ['http://server1:6632', 'http://server2:6632'],
+  timeoutMillis: 30000,
+  retryAttempts: 3,
+  loadBalancingStrategy: 'round-robin',  // or 'session'
+  enableFailover: true
 })
-// Nack the messages if you failed to process them
-.onError(async (message, error) => {
-  console.error('Error processing messages:', error)
-  await queen.ack(message, false)
+```
+
+---
+
+## Basic Usage Patterns
+
+### Push Messages
+
+```javascript
+// Simple push
+await queen.queue('tasks').push([
+  { data: { job: 'resize-image', imageId: 123 } }
+])
+
+// With partition
+await queen.queue('tasks')
+  .partition('tenant-456')
+  .push([{ data: { action: 'process' } }])
+
+// With custom transaction ID (for exactly-once)
+await queen.queue('tasks').push([
+  {
+    transactionId: 'unique-id-123',
+    data: { value: 42 }
+  }
+])
+```
+
+### Consume Messages (Long-Running Workers)
+
+```javascript
+// Runs forever, processes messages as they arrive
+await queen.queue('tasks')
+  .concurrency(10)        // 10 parallel workers
+  .batch(20)              // Fetch 20 at a time
+  .consume(async (message) => {
+    await processTask(message.data)
+    // Auto-ack on success, auto-retry on error
+  })
+
+// Process with limit and stop
+await queen.queue('tasks')
+  .limit(100)
+  .consume(async (message) => {
+    await processTask(message.data)
+  })
+```
+
+### Pop Messages (On-Demand Processing)
+
+```javascript
+// Grab messages manually
+const messages = await queen.queue('tasks')
+  .batch(10)
+  .wait(true)  // Long polling
+  .pop()
+
+// Manual acknowledgment
+for (const message of messages) {
+  try {
+    await processMessage(message.data)
+    await queen.ack(message, true)  // Success
+  } catch (error) {
+    await queen.ack(message, false)  // Retry
+  }
+}
+```
+
+### Transactions (Atomic Operations)
+
+```javascript
+// Pop from queue A
+const messages = await queen.queue('input').pop()
+
+// Atomically: ack input AND push output
+await queen.transaction()
+  .ack(messages[0])
+  .queue('output')
+  .push([{ data: processedResult }])
+  .commit()
+
+// If commit fails, nothing happens - message stays in input queue
+```
+
+### Client-Side Buffering (High Throughput)
+
+```javascript
+// Buffer messages locally, batch to server
+for (let i = 0; i < 10000; i++) {
+  await queen.queue('events')
+    .buffer({ messageCount: 500, timeMillis: 1000 })
+    .push([{ data: { id: i } }])
+}
+
+// Flush remaining buffered messages
+await queen.flushAllBuffers()
+
+// Result: 10x-100x faster than individual pushes
+```
+
+### Dead Letter Queue
+
+```javascript
+// Enable DLQ on queue
+await queen.queue('risky')
+  .config({ retryLimit: 3, dlqAfterMaxRetries: true })
+  .create()
+
+// Query failed messages
+const dlq = await queen.queue('risky')
+  .dlq()
+  .limit(10)
+  .get()
+
+console.log(`Found ${dlq.total} failed messages`)
+for (const msg of dlq.messages) {
+  console.log('Error:', msg.errorMessage)
+}
+```
+
+### Message Tracing
+
+```javascript
+await queen.queue('orders').consume(async (msg) => {
+  const orderId = msg.data.orderId
+  
+  // Record trace with name for cross-service correlation
+  await msg.trace({
+    traceName: `order-${orderId}`,
+    eventType: 'info',
+    data: { text: 'Order processing started' }
+  })
+  
+  await processOrder(msg.data)
+  
+  await msg.trace({
+    traceName: `order-${orderId}`,
+    eventType: 'processing',
+    data: { 
+      text: 'Order completed',
+      total: msg.data.total
+    }
+  })
 })
+
+// View traces in webapp: Traces → Search "order-12345"
 ```
 
-## Webapp
+---
 
-A modern Vue 3 web interface for managing and monitoring Queen MQ. 
+## Examples
 
-![Queen MQ Dashboard](./assets/dashboard.png)
+### Complete Pipeline with Consumer Groups
 
-![Queen MQ Queues](./assets/queues.png) 
+```javascript
+import { Queen } from 'queen-mq'
 
-![Queen MQ Messages](./assets/messages.png) 
+const queen = new Queen('http://localhost:6632')
 
-![Queen MQ Traces](./assets/traces.png) 
+// Stage 1: Ingest with buffering
+async function ingestEvents() {
+  for (let i = 0; i < 10000; i++) {
+    await queen.queue('raw-events')
+      .partition(`user-${i % 100}`)
+      .buffer({ messageCount: 500, timeMillis: 1000 })
+      .push([{ data: { userId: i % 100, event: 'page_view' } }])
+  }
+  await queen.flushAllBuffers()
+}
+
+// Stage 2: Process with transactions
+async function processEvents() {
+  await queen.queue('raw-events')
+    .group('processors')
+    .concurrency(5)
+    .batch(10)
+    .autoAck(false)
+    .consume(async (messages) => {
+      const results = messages.map(m => process(m.data))
+      
+      // Atomic: ack all inputs, push all outputs
+      const txn = queen.transaction()
+      for (const msg of messages) txn.ack(msg)
+      txn.queue('processed-events').push(results.map(r => ({ data: r })))
+      await txn.commit()
+    })
+}
+
+// Stage 3: Separate analytics consumer (fan-out)
+async function analytics() {
+  await queen.queue('raw-events')
+    .group('analytics')
+    .subscriptionMode('new')  // Skip backlog
+    .consume(async (message) => {
+      await logMetrics(message.data)
+    })
+}
+
+await ingestEvents()
+await Promise.all([processEvents(), analytics()])
+```
 
-![Queen MQ Analytics](./assets/analytics.png) 
+### Long-Running Tasks with Lease Renewal
 
-![Queen MQ System Metrics](./assets/systemmetrics.png) 
+```javascript
+await queen.queue('video-processing')
+  .renewLease(true, 60000)  // Renew every 60 seconds
+  .consume(async (message) => {
+    // Can take hours - lease keeps renewing automatically
+    await processVideo(message.data)
+  })
+```
 
-**Features:**
-- 📊 Real-time dashboard with system metrics
-- 📈 Message throughput visualization
-- 🔍 Queue management and monitoring
-- 👥 Consumer group tracking
-- 💬 Message browser with trace timeline
-- 🔎 **Trace explorer for debugging distributed workflows**
-- 📉 Analytics and insights
-- 🌓 Dark/light theme support
+### Error Handling with Callbacks
 
-**Quick Start:**
-```bash
-cd webapp
-npm install
-npm run dev
+```javascript
+await queen.queue('tasks')
+  .autoAck(false)
+  .consume(async (message) => {
+    return await riskyOperation(message.data)
+  })
+  .onSuccess(async (message, result) => {
+    console.log('Success:', result)
+    await queen.ack(message, true)
+  })
+  .onError(async (message, error) => {
+    console.error('Failed:', error.message)
+    
+    // Custom retry logic
+    if (error.message.includes('temporary')) {
+      await queen.ack(message, false)  // Retry
+    } else {
+      await queen.ack(message, 'failed', { error: error.message })
+    }
+  })
 ```
 
-The dashboard will be available at `http://localhost:4000` or at `http://localhost:6632` directly from the server.
+---
 
-See [webapp/README.md](webapp/README.md) for more details.
+## API Reference
 
-## Architecture
+### Queue Operations
 
-Queen uses a high-performance **acceptor/worker pattern** with uWebSockets, combining non-blocking I/O for HTTP/WebSocket with a dedicated thread pool for database operations.
+```javascript
+// Create
+await queen.queue('my-queue').create()
+await queen.queue('my-queue').config({ priority: 5 }).create()
 
-**View the interactive architecture diagram:** [architecture.svg](./assets/architecture.svg)
+// Delete
+await queen.queue('my-queue').delete()
 
-**Key Components:**
-- **UWS Acceptor**: Single thread listening on port 6632, round-robin distributes to workers
-- **UWS Workers**: N event loop threads (default: 10) handling HTTP routes and WebSocket
-- **Response Timers**: Per-worker timers (25ms tick) drain response queue back to clients
-- **DB ThreadPool**: Separate pool for blocking PostgreSQL operations
-- **Poll Workers**: 2 reserved threads for long-polling with adaptive backoff (100ms→2000ms)
-- **Poll Intention Registry**: Thread-safe store for long-poll requests
-- **Database Pool**: 150 shared PostgreSQL connections (libpq) with mutex/condition variable
-- **Response Queue**: Thread-safe queue decoupling DB results from event loop responses
+// Get info
+const info = await queen.getQueueInfo('my-queue')
+```
 
-**Request Flow:**
-1. Client → Acceptor → Worker (event loop)
-2. Worker registers response, submits job to DB ThreadPool
-3. DB thread executes query, pushes result to Response Queue
-4. Worker's response timer drains queue, sends HTTP response
+### Push
 
-**Long-Polling Flow:**
-1. No immediate messages? Register intention in Registry
-2. Poll Workers wake every 50ms, group intentions by queue/partition/consumer
-3. Rate-limited DB queries (100ms initial, exponential backoff to 2s)
-4. Messages distributed to waiting clients via Response Queue
-5. Timeouts detected by Poll Workers, send 204 No Content
+```javascript
+await queen.queue('q').push([{ data: { value: 1 } }])
+await queen.queue('q').partition('p1').push([{ data: { value: 1 } }])
+await queen.queue('q').buffer({ messageCount: 100, timeMillis: 1000 }).push([...])
+```
 
-This architecture provides high concurrency, efficient connection pooling, and minimal latency for both immediate and long-polling requests.
+### Pop
 
-### PostgreSQL Failover
+```javascript
+const msgs = await queen.queue('q').pop()
+const msgs = await queen.queue('q').batch(10).pop()
+const msgs = await queen.queue('q').batch(10).wait(true).pop()
+```
 
-Queen automatically buffers messages to disk when PostgreSQL is unavailable - **zero message loss**:
+### Consume
 
-- Normal pushes go directly to PostgreSQL (FIFO preserved)
-- If PostgreSQL is down, messages buffered to file (macOS: `/tmp/queen`, Linux: `/var/lib/queen/buffers`)
-- Automatic replay when PostgreSQL recovers
-- Survives server crashes and restarts
-- Directory auto-created on first run
+```javascript
+await queen.queue('q').consume(async (msg) => { /* process */ })
+await queen.queue('q').limit(10).consume(async (msg) => { /* process */ })
+await queen.queue('q').concurrency(5).consume(async (msg) => { /* 5 workers */ })
+await queen.queue('q').group('my-group').consume(async (msg) => { /* consumer group */ })
+```
 
-**No configuration needed** - failover is automatic!
+### Acknowledgment
 
-**Custom directory:**
-```bash
-FILE_BUFFER_DIR=/custom/path ./bin/queen-server
+```javascript
+await queen.ack(message, true)   // Success
+await queen.ack(message, false)  // Retry
+await queen.ack(message, false, { error: 'reason' })
+await queen.ack([msg1, msg2], true)  // Batch ack
 ```
 
-## Performance Benchmarks
+### Transactions
 
-**Preliminary results from C++ client benchmark** (detailed benchmarks on dedicated hardware coming soon)
+```javascript
+await queen.transaction()
+  .ack(message)
+  .queue('output')
+  .push([{ data: { result: 'processed' } }])
+  .commit()
+```
 
-### Test Environment
-- **Hardware:** Apple M4 Air (all components on same machine, 10 processors available)
-- **Server:** 1 server, 4 workers, 95 total DB connections/threads
-- **Database:** PostgreSQL in Docker
-- **Client:** C++ benchmark tool (`benchmark/bin/benchmark`)
+### Lease Renewal
 
-### Results
+```javascript
+await queen.renew(message)
+await queen.renew([msg1, msg2, msg3])
+await queen.queue('q').renewLease(true, 60000).consume(async (msg) => { /* auto-renew */ })
+```
 
-| Mode | Threads | Messages | Batch Size | Partitions | Queue Mode | Throughput | Bandwidth |
-|------|---------|----------|------------|------------|------------|------------|-----------|
-| **Producer** | 100 | 10K | 1 | 100 | single-queue | **1,431 msg/sec** | 0.41 MB/sec |
-| **Producer** | 100 | 10K | 1 | 100 | multi-queue | 527 msg/sec | 0.15 MB/sec |
-| **Producer** | 10 | 1M | 1,000 | 100 | single-queue | **62,927 msg/sec** | 18.00 MB/sec |
-| **Producer** | 10 | 1M | 1,000 | 100 | multi-queue | 49,663 msg/sec | 14.21 MB/sec |
-| **Producer** | 10 | 1M | 10,000 | 10 | single-queue | 30,351 msg/sec | 8.68 MB/sec |
-| **Consumer** | 10 | 1M | 1,000 | 100 | single-queue | **31,922 msg/sec** | 16.25 MB/sec |
-| **Consumer** | 10 | 1M | 10,000 | 10 | single-queue | **82,974 msg/sec** | 42.25 MB/sec |
+### Buffering
 
-### Key Observations
+```javascript
+await queen.flushAllBuffers()
+await queen.queue('q').flushBuffer()
+const stats = queen.getBufferStats()
+```
 
-- ✅ **Batch size matters:** Larger batches (1,000-10,000) dramatically improve throughput
-- ✅ **Single-queue mode faster:** Better partition-level parallelism than multi-queue
-- ✅ **Consumer performance:** Scales well with batch size (82K msg/sec with 10K batches)
-- ✅ **Producer peak:** 62K msg/sec with optimal batch size (1,000)
-- ⚠️ **Small batches:** Performance drops significantly with batch=1 (lock contention)
+### Dead Letter Queue
 
-**Note:** All timing metrics exclude idle timeouts and measure actual message processing time (first message → last message).
+```javascript
+const dlq = await queen.queue('q').dlq().limit(10).get()
+const dlq = await queen.queue('q').dlq('consumer-group').limit(10).get()
+const dlq = await queen.queue('q').dlq().from('2025-01-01').to('2025-01-31').get()
+```
 
-### Run Your Own Benchmarks
+### Shutdown
 
-```bash
-cd benchmark
-make
+```javascript
+await queen.close()  // Flush buffers and close connections
+```
+
+---
+
+## Configuration Defaults
+
+### Client Defaults
+
+```javascript
+{
+  timeoutMillis: 30000,
+  retryAttempts: 3,
+  retryDelayMillis: 1000,
+  loadBalancingStrategy: 'round-robin',
+  enableFailover: true
+}
+```
+
+### Queue Defaults
+
+```javascript
+{
+  leaseTime: 300,         // 5 minutes
+  retryLimit: 3,
+  priority: 0,
+  delayedProcessing: 0,
+  windowBuffer: 0,
+  maxSize: 0,            // Unlimited
+  retentionSeconds: 0,   // Keep forever
+  encryptionEnabled: false
+}
+```
 
-# Producer
-./bin/benchmark producer --threads 10 --count 1000000 --batch 1000 --partitions 100 --mode single-queue
+### Consume Defaults
 
-# Consumer
-./bin/benchmark consumer --threads 10 --batch 1000 --partitions 100 --mode single-queue
+```javascript
+{
+  concurrency: 1,
+  batch: 1,
+  autoAck: true,
+  wait: true,            // Long polling
+  timeoutMillis: 30000,
+  limit: null,           // Run forever
+  renewLease: false
+}
 ```
 
-See [benchmark/README.md](benchmark/README.md) for detailed usage.
+---
 
-## Install the server and configure it
+## Logging
 
-### Quick Start
+Enable detailed logging for debugging:
 
-```sh
-cd server
-make clean
-make deps
-make build-only
-DB_POOL_SIZE=50 ./bin/queen-server
+```bash
+export QUEEN_CLIENT_LOG=true
+node your-app.js
 ```
 
-**📖 Complete Build & Tuning Guide:** [server/README.md](server/README.md)
+Example output:
+```
+[2025-10-28T10:30:45.123Z] [INFO] [Queen.constructor] {"status":"initialized","urls":1}
+[2025-10-28T10:30:45.234Z] [INFO] [QueueBuilder.push] {"queue":"tasks","partition":"Default","count":5}
+```
+
+---
+
+## Best Practices
+
+1. ✅ **Use `consume()` for workers** - Simpler API, handles retries automatically
+2. ✅ **Use `pop()` for control** - When you need precise control over acking
+3. ✅ **Buffer for speed** - Always use buffering when pushing many messages
+4. ✅ **Partitions for order** - Use partitions when message order matters
+5. ✅ **Consumer groups for scale** - Run multiple workers in the same group
+6. ✅ **Transactions for consistency** - Use transactions for atomic operations
+7. ✅ **Enable DLQ** - Always enable DLQ in production
+8. ✅ **Renew long leases** - Use auto-renewal for long-running tasks
+9. ✅ **Graceful shutdown** - Always call `queen.close()` before exiting
+10. ✅ **Monitor DLQ** - Regularly check for failed messages
+
+---
+
+## TypeScript Support
 
-Includes:
-- Build instructions and optimization
-- Performance tuning (worker threads, database pool)
-- Production deployment (systemd, Docker, load balancing)
-- Troubleshooting common issues
-- Benchmarking guides
+Full TypeScript definitions included:
 
-### Environment Variables
+```typescript
+import { Queen, Message, QueueConfig } from 'queen-mq'
 
-[The full list of environment variables is here](server/ENV_VARIABLES.md)
+const queen: Queen = new Queen('http://localhost:6632')
 
-### With Docker
-```sh
-./build.sh
+interface OrderData {
+  orderId: number
+  amount: number
+}
+
+const messages: Message<OrderData>[] = await queen.queue('orders').pop()
 ```
 
-### Running on k8s
+---
 
-[Running in k8s](server/k8s-example.yaml)
+## Documentation
 
-## 🔌 Raw HTTP API
+- **[Complete V2 Guide](client-v2/README.md)** - Full tutorial with all features (94 test examples)
+- **[HTTP API Reference](https://github.com/smartpricing/queen/blob/master/server/API.md)** - Raw HTTP endpoints
+- **[Server Guide](https://github.com/smartpricing/queen/blob/master/server/README.md)** - Server setup and configuration
+- **[Architecture Guide](https://github.com/smartpricing/queen/blob/master/docs/ARCHITECTURE.md)** - Deep dive into internals
 
-You can use Queen directly from HTTP without the JS client.
+---
 
-[Here the complete list of API endpoints](server/API.md)
+## Support
 
-## ⚠️ Known Issues & Roadmap
+- **GitHub:** [smartpricing/queen](https://github.com/smartpricing/queen)
+- **Issues:** [GitHub Issues](https://github.com/smartpricing/queen/issues)
+- **LinkedIn:** [Smartness](https://www.linkedin.com/company/smartness-com/)
 
-### Server Startup Timing (Critical)
-**Issue:** Worker initialization timeout (30s → 3600s) now matches file buffer recovery timeout. This is a temporary fix.
+---
 
-**Better Solution Needed:**
-- Make recovery non-blocking while preserving FIFO ordering guarantees
-- Implement progressive readiness with memory-buffered queue during recovery
-- Add configurable recovery timeout with graceful degradation
-- See: `server/src/services/file_buffer.cpp:212` (MAX_STARTUP_RECOVERY_SECONDS)
-- See: `server/src/acceptor_server.cpp:1876` (worker initialization timeout)
+## License
 
+Apache 2.0 - See [LICENSE.md](../LICENSE.md)
 
-### Other TODO Items
-- Mini streaming engine
-- Proper concurrency on clients 
-- Check client failover
-- Py client