worker-que 1.0.0 → 1.1.1

package/README.md

@@ -1,107 +1,37 @@
-# que-ts
+# worker-que
 
-[![Test](https://github.com/Duke-Engineering/que-ts/actions/workflows/test.yml/badge.svg)](https://github.com/Duke-Engineering/que-ts/actions/workflows/test.yml)
-[![Coverage](https://github.com/Duke-Engineering/que-ts/actions/workflows/coverage.yml/badge.svg)](https://github.com/Duke-Engineering/que-ts/actions/workflows/coverage.yml)
-[![Security](https://github.com/Duke-Engineering/que-ts/actions/workflows/security.yml/badge.svg)](https://github.com/Duke-Engineering/que-ts/actions/workflows/security.yml)
-[![npm version](https://badge.fury.io/js/que-ts.svg)](https://badge.fury.io/js/que-ts)
+> A production-ready TypeScript job queue library for PostgreSQL with real-time monitoring dashboard
 
-A TypeScript job queue library for PostgreSQL, compatible with Ruby Que and que-go implementations.
+[![npm version](https://badge.fury.io/js/worker-que.svg)](https://www.npmjs.com/package/worker-que)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.2-blue.svg)](https://www.typescriptlang.org/)
+[![Node.js](https://img.shields.io/badge/Node.js-%3E%3D16.0.0-green.svg)](https://nodejs.org/)
 
 ## Features
 
-- **Cross-language compatibility**: Works with [Que (Ruby)](https://github.com/chanks/que) and [que-go](https://github.com/bgentry/que-go) job queues
-- **PostgreSQL advisory locks**: Reliable job processing with no duplicate execution
-- **TypeScript support**: Full type safety with comprehensive interfaces
-- **Retry logic**: Exponential backoff for failed jobs
-- **Multiple queues**: Support for named queues and priorities
-- **Transaction support**: Enqueue jobs within existing database transactions
-- **Web Dashboard**: Real-time monitoring and management UI (Express.js integration)
-- **SSL/TLS Support**: Secure connections with client certificates
-
-## Installation
-
-### From npm (when published)
-```bash
-npm install que-ts
-```
-
-### From GitHub (development)
-```bash
-npm install github:Duke-Engineering/que-ts#master
-```
-
-**Note**: When installing from GitHub, the package will automatically build from TypeScript source using the `prepare` script.
-
-### Troubleshooting GitHub Installation
-
-If you encounter "Cannot find module 'que-ts'" errors when installing from GitHub:
-
-1. **Check the installation completed successfully**:
-   ```bash
-   cd node_modules/que-ts
-   ls dist/  # Should show compiled JavaScript files
-   ```
-
-2. **Manual build if needed**:
-   ```bash
-   cd node_modules/que-ts
-   npm run build
-   ```
-
-3. **Verify installation**:
-   ```bash
-   cd node_modules/que-ts
-   node test-install.js
-   ```
-
-4. **Alternative: Use specific tag**:
-   ```bash
-   npm install github:Duke-Engineering/que-ts#v1.0.0
-   ```
+- ⚡ **High Performance** - PostgreSQL-backed with advisory locks for reliable job processing
+- 📊 **Real-time Dashboard** - Beautiful web UI for monitoring and managing jobs
+- 🔐 **Enterprise Security** - SSL/TLS support with client certificates
+- 🎯 **Priority Queues** - Multiple queues with configurable priorities
+- 🔄 **Automatic Retries** - Exponential backoff for failed jobs
+- 💾 **Transaction Support** - Enqueue jobs within database transactions
+- 🌐 **Cross-Platform** - Compatible with Ruby Que and que-go
+- 📝 **TypeScript Native** - Full type safety and IntelliSense support
 
 ## Quick Start
 
-### Database Setup
-
-#### Option 1: Using Docker (Recommended for Development)
+### Installation
 
 ```bash
-# Start PostgreSQL with Docker
-npm run docker:up
-
-# Run tests
-npm test
-
-# Stop when done
-npm run docker:down
-```
-
-For detailed Docker usage, see [DOCKER.md](DOCKER.md).
-
-#### Option 2: Manual Database Setup
-
-Create the required database table:
-
-```sql
-CREATE TABLE que_jobs (
-    priority    smallint    NOT NULL DEFAULT 100,
-    run_at      timestamptz NOT NULL DEFAULT now(),
-    job_id      bigserial   NOT NULL,
-    job_class   text        NOT NULL,
-    args        json        NOT NULL DEFAULT '[]'::json,
-    error_count integer     NOT NULL DEFAULT 0,
-    last_error  text,
-    queue       text        NOT NULL DEFAULT '',
-    PRIMARY KEY (queue, priority, run_at, job_id)
-);
+npm install worker-que pg express
 ```
 
 ### Basic Usage
 
 ```typescript
-import { Client, Worker } from 'que-ts';
+import { Client, Worker } from 'worker-que';
 
-// Create a client
+// Create client
 const client = new Client({
   host: 'localhost',
   port: 5432,
@@ -113,12 +43,7 @@ const client = new Client({
 // Enqueue jobs
 await client.enqueue('SendEmail', ['user@example.com', 'Welcome!']);
 
-await client.enqueue('ProcessPayment', [{ amount: 100, currency: 'USD' }], {
-  priority: 10,
-  runAt: new Date(Date.now() + 60000) // Run in 1 minute
-});
-
-// Create and start a worker
+// Create worker
 const worker = new Worker({
   host: 'localhost',
   port: 5432,
@@ -127,48 +52,59 @@ const worker = new Worker({
   password: 'password'
 });
 
-// Register job handlers
+// Register handlers
 worker.register('SendEmail', async (job) => {
   const [email, message] = job.args;
   console.log(`Sending email to ${email}: ${message}`);
-  // Email sending logic here
-});
-
-worker.register('ProcessPayment', async (job) => {
-  const paymentData = job.args[0];
-  console.log(`Processing payment of ${paymentData.amount} ${paymentData.currency}`);
-  // Payment processing logic here
+  // Your email logic here
 });
 
-// Start processing jobs
+// Start processing
 await worker.work();
+```
 
-// Graceful shutdown
-process.on('SIGINT', async () => {
-  await worker.shutdown();
-  await client.close();
-});
+### Database Setup
+
+Create the required table:
+
+```sql
+CREATE TABLE que_jobs (
+    priority    smallint    NOT NULL DEFAULT 100,
+    run_at      timestamptz NOT NULL DEFAULT now(),
+    job_id      bigserial   NOT NULL,
+    job_class   text        NOT NULL,
+    args        json        NOT NULL DEFAULT '[]'::json,
+    error_count integer     NOT NULL DEFAULT 0,
+    last_error  text,
+    queue       text        NOT NULL DEFAULT '',
+    PRIMARY KEY (queue, priority, run_at, job_id)
+);
+```
+
+Or use the included migration:
+
+```bash
+psql -U postgres -d myapp -f node_modules/worker-que/migrations/schema.sql
 ```
 
 ## Web Dashboard
 
-que-ts includes a beautiful, real-time web dashboard for monitoring and managing your job queue.
+Monitor and manage your jobs with the built-in web dashboard featuring **Microsoft Fluent Design** and **secure authentication**:
 
 ```typescript
 import express from 'express';
 import { Pool } from 'pg';
-import { createDashboard } from 'que-ts/dashboard';
+import { createDashboard } from 'worker-que/dist/dashboard';
 
 const app = express();
 const pool = new Pool({ /* your config */ });
 
-// Mount dashboard at /admin/queue
+// Built-in email/password authentication
 app.use('/admin/queue', createDashboard(pool, {
-  title: 'My App Queue',
-  refreshInterval: 3000,
-  auth: (req, res, next) => {
-    // Add your authentication logic
-    return req.isAuthenticated();
+  title: 'My Job Queue',
+  auth: {
+    email: 'admin@example.com',
+    password: 'your-secure-password'
   }
 }));
 
@@ -177,238 +113,409 @@ app.listen(3000);
 ```
 
 **Dashboard Features:**
-- 📊 Real-time statistics (total, ready, scheduled, failed jobs)
-- 📈 Visual analytics (charts by queue and job class)
-- 🔍 Advanced filtering (status, queue, job class)
-- 🔄 Job management (retry failed jobs, delete jobs)
-- 🔐 Built-in authentication support
-- 📱 Responsive design
+- 📊 Real-time job statistics with auto-refresh
+- 📈 Visual analytics by queue and class
+- 🔍 Filter and search jobs
+- 🔄 Retry failed jobs
+- 🗑️ Delete jobs
+- 🔐 Built-in secure authentication with sessions
+- 🎨 Modern Microsoft Fluent Design UI
+- 📱 Fully responsive design
 
-For complete dashboard documentation, see [DASHBOARD.md](DASHBOARD.md).
+[**→ Complete Dashboard Documentation**](./DASHBOARD.md)
 
-## SSL Configuration
+## Core Concepts
 
-For detailed SSL configuration including AWS RDS, Google Cloud SQL, Azure, and certificate setup, see [SSL.md](SSL.md).
+### Jobs
 
-## API Reference
+Jobs are units of work stored in PostgreSQL:
 
-### Client
+```typescript
+await client.enqueue('ProcessPayment', [
+  { amount: 100, userId: 123 }
+], {
+  priority: 10,        // Lower = higher priority
+  queue: 'critical',   // Queue name
+  runAt: new Date()    // Scheduled time
+});
+```
 
-#### Constructor
+### Workers
+
+Workers poll for jobs and execute registered handlers:
 
 ```typescript
-new Client(config?: ClientConfig)
-```
+const worker = new Worker({
+  host: 'localhost',
+  port: 5432,
+  database: 'myapp'
+}, {
+  queue: 'critical',   // Process specific queue
+  interval: 1000       // Poll every 1 second
+});
 
-#### Methods
+worker.register('ProcessPayment', async (job) => {
+  const [payment] = job.args;
+  // Process payment...
+  // Job automatically marked as done
+});
 
-- `enqueue(jobClass: string, args?: any[], options?: EnqueueOptions): Promise<Job>`
-- `enqueueInTx(client: PoolClient, jobClass: string, args?: any[], options?: EnqueueOptions): Promise<Job>`
-- `lockJob(queue?: string): Promise<Job | null>`
-- `close(): Promise<void>`
+await worker.work();
+```
 
-### Worker
+### Error Handling
 
-#### Constructor
+Failed jobs are automatically retried with exponential backoff:
+
+- 1st retry: after 1 second
+- 2nd retry: after 16 seconds
+- 3rd retry: after 81 seconds
+- 4th retry: after 256 seconds
 
 ```typescript
-new Worker(clientConfig?: ClientConfig, options?: WorkerOptions)
+worker.register('RiskyJob', async (job) => {
+  try {
+    await riskyOperation();
+  } catch (error) {
+    throw error; // Job will be retried
+  }
+});
 ```
 
-#### Methods
+## Advanced Features
 
-- `register(jobClass: string, workFunc: WorkFunction): void`
-- `work(): Promise<void>`
-- `workOne(): Promise<boolean>`
-- `shutdown(): Promise<void>`
+### Priority Queues
 
-### Job
+Organize jobs by priority and queue:
 
-#### Properties
+```typescript
+// High priority
+await client.enqueue('UrgentTask', [data], { priority: 1 });
+
+// Critical queue
+await client.enqueue('Payment', [data], {
+  priority: 1,
+  queue: 'critical'
+});
+
+// Background queue
+await client.enqueue('Analytics', [data], {
+  priority: 500,
+  queue: 'background'
+});
+```
 
-- `id: number`
-- `queue: string`
-- `priority: number`
-- `runAt: Date`
-- `jobClass: string`
-- `args: any[]`
-- `errorCount: number`
-- `lastError?: string`
+### Scheduled Jobs
 
-#### Methods
+Schedule jobs for future execution:
 
-- `done(): Promise<void>`
-- `delete(): Promise<void>`
-- `error(errorMessage: string): Promise<void>`
+```typescript
+// Run in 1 hour
+const runAt = new Date(Date.now() + 3600000);
+await client.enqueue('SendReminder', [data], { runAt });
+
+// Daily at 9 AM
+const tomorrow = new Date();
+tomorrow.setDate(tomorrow.getDate() + 1);
+tomorrow.setHours(9, 0, 0, 0);
+await client.enqueue('DailyReport', [data], { runAt: tomorrow });
+```
 
-## Configuration
+### Transaction Support
 
-### ClientConfig
+Enqueue jobs within database transactions:
 
 ```typescript
-interface ClientConfig {
-  connectionString?: string;
-  host?: string;
-  port?: number;
-  database?: string;
-  user?: string;
-  password?: string;
-  ssl?: boolean | SSLConfig;  // See SSL Configuration below
-  maxConnections?: number;
+import { Pool } from 'pg';
+
+const pool = new Pool({ /* config */ });
+const client = new Client({ /* config */ });
+
+const pgClient = await pool.connect();
+try {
+  await pgClient.query('BEGIN');
+  
+  // Database operations
+  await pgClient.query('INSERT INTO orders ...');
+  
+  // Enqueue job in same transaction
+  await client.enqueueInTx(pgClient, 'SendOrderEmail', [orderId]);
+  
+  await pgClient.query('COMMIT');
+} catch (error) {
+  await pgClient.query('ROLLBACK');
+  throw error;
+} finally {
+  pgClient.release();
 }
 ```
 
-### SSL Configuration
+### Multiple Workers
 
-que-ts supports SSL/TLS connections with client certificates:
+Run specialized workers for different queues:
 
 ```typescript
-import { Client } from 'que-ts';
-import * as fs from 'fs';
+// Critical queue worker (fast polling)
+const criticalWorker = new Worker(dbConfig, {
+  queue: 'critical',
+  interval: 100
+});
 
-// Basic SSL
-const client = new Client({
-  host: 'your-database.com',
-  port: 5432,
-  database: 'mydb',
-  user: 'myuser',
-  password: 'mypassword',
-  ssl: {
-    rejectUnauthorized: true,
-  }
+// Background queue worker (slow polling)
+const backgroundWorker = new Worker(dbConfig, {
+  queue: 'background',
+  interval: 5000
 });
 
-// SSL with client certificates
-const clientWithCerts = new Client({
-  host: 'your-database.com',
-  port: 5432,
-  database: 'mydb',
-  user: 'myuser',
-  password: 'mypassword',
+criticalWorker.work();
+backgroundWorker.work();
+```
+
+### SSL/TLS Support
+
+Secure database connections with client certificates:
+
+```typescript
+import * as fs from 'fs';
+
+const client = new Client({
+  host: 'db.example.com',
   ssl: {
     rejectUnauthorized: true,
     cert: fs.readFileSync('./certs/client-cert.pem'),
     key: fs.readFileSync('./certs/client-key.pem'),
-    ca: fs.readFileSync('./certs/ca-cert.pem'),
-    passphrase: 'optional-key-passphrase', // For encrypted keys
+    ca: fs.readFileSync('./certs/ca-cert.pem')
   }
 });
 ```
 
-For detailed SSL configuration including AWS RDS, Google Cloud SQL, Azure, and certificate setup, see [SSL.md](SSL.md).
+[**→ Complete SSL Documentation**](./SSL.md)
+
+## API Reference
+
+### Client
+
+#### `new Client(config: ClientConfig)`
 
-### WorkerOptions
+Creates a new client instance.
 
+**Config Options:**
 ```typescript
-interface WorkerOptions {
-  queue?: string;        // Queue name to process (default: '')
-  interval?: number;     // Polling interval in ms (default: 5000)
+{
+  host?: string;
+  port?: number;
+  database?: string;
+  user?: string;
+  password?: string;
+  ssl?: boolean | SSLConfig;
+  maxConnections?: number;
+}
+```
+
+#### `client.enqueue(jobClass, args?, options?): Promise<Job>`
+
+Enqueues a new job.
+
+**Parameters:**
+- `jobClass` - Job type identifier
+- `args` - Job arguments (JSON array)
+- `options` - Priority, queue, runAt
+
+**Returns:** Job instance with ID and metadata
+
+#### `client.enqueueInTx(pgClient, jobClass, args?, options?): Promise<Job>`
+
+Enqueues a job within a transaction.
+
+#### `client.lockJob(queue?): Promise<Job | null>`
+
+Locks and returns the next available job.
+
+#### `client.close(): Promise<void>`
+
+Closes all database connections.
+
+### Worker
+
+#### `new Worker(config: ClientConfig, options?: WorkerOptions)`
+
+Creates a new worker instance.
+
+**Worker Options:**
+```typescript
+{
+  queue?: string;        // Queue to process (default: '')
+  interval?: number;     // Poll interval in ms (default: 60000)
   maxAttempts?: number;  // Max retry attempts (default: 5)
 }
 ```
 
-### EnqueueOptions
+#### `worker.register(jobClass, handler): void`
+
+Registers a job handler function.
+
+#### `worker.work(): Promise<void>`
+
+Starts continuous job processing.
+
+#### `worker.workOne(): Promise<boolean>`
+
+Processes a single job. Returns true if job was processed.
+
+#### `worker.shutdown(): Promise<void>`
+
+Gracefully shuts down the worker.
+
+### Job
+
+Job instances have these properties:
 
 ```typescript
-interface EnqueueOptions {
-  priority?: number;  // Job priority (lower = higher priority, default: 100)
-  runAt?: Date;       // When to run the job (default: now)
-  queue?: string;     // Queue name (default: '')
+{
+  id: number;
+  queue: string;
+  priority: number;
+  runAt: Date;
+  jobClass: string;
+  args: any[];
+  errorCount: number;
+  lastError?: string;
 }
 ```
 
-## Error Handling and Retries
+And these methods:
 
-Failed jobs are automatically retried with exponential backoff:
+- `job.done()` - Mark as completed
+- `job.error(message)` - Mark as failed
+- `job.delete()` - Remove from queue
 
-- 1st retry: after 1 second
-- 2nd retry: after 16 seconds  
-- 3rd retry: after 81 seconds
-- 4th retry: after 256 seconds
-- etc.
+## Performance Tips
 
-Jobs that exceed the maximum number of retries remain in the queue for manual inspection.
+### Database Indexes
 
-## Queues and Priorities
+Add indexes for optimal performance:
 
-Jobs can be organized into named queues and assigned priorities:
+```sql
+CREATE INDEX idx_que_jobs_run_at ON que_jobs(run_at);
+CREATE INDEX idx_que_jobs_error_count ON que_jobs(error_count);
+CREATE INDEX idx_que_jobs_queue ON que_jobs(queue);
+CREATE INDEX idx_que_jobs_job_class ON que_jobs(job_class);
+```
+
+### Connection Pooling
+
+Configure appropriate pool sizes:
 
 ```typescript
-// High priority job in 'critical' queue
-await client.enqueue('ProcessPayment', [paymentData], {
-  queue: 'critical',
-  priority: 1
+const client = new Client({
+  maxConnections: 20  // Adjust based on load
 });
+```
 
-// Low priority job in 'background' queue
-await client.enqueue('SendNewsletter', [newsletterData], {
-  queue: 'background', 
-  priority: 500
-});
+### Worker Tuning
+
+Balance polling frequency with load:
 
-// Worker processing only critical queue
-const criticalWorker = new Worker(config, { queue: 'critical' });
+```typescript
+// High-frequency (for critical jobs)
+new Worker(config, { interval: 100 });
+
+// Low-frequency (for background jobs)
+new Worker(config, { interval: 30000 });
 ```
 
 ## Cross-Language Compatibility
 
-que-ts is designed to be fully compatible with:
-
-- **[Ruby Que](https://github.com/chanks/que)** - The original Ruby implementation
-- **[que-go](https://github.com/bgentry/que-go)** - Golang port (currently unmaintained)
+worker-que is fully compatible with:
+- [**Que (Ruby)**](https://github.com/chanks/que) - Original implementation
+- [**que-go**](https://github.com/bgentry/que-go) - Go implementation
 
-You can enqueue jobs in one language and process them in another, or run workers in multiple languages simultaneously.
+Jobs can be enqueued in one language and processed in another.
 
-## Related Projects
+## Production Deployment
 
-### Official Implementations
-- **[Que (Ruby)](https://github.com/chanks/que)** - The original and most mature implementation
-- **[que-go](https://github.com/bgentry/que-go)** - Go implementation (unmaintained, but stable)
-- **[que-ts](https://github.com/Duke-Engineering/que-ts)** - This TypeScript/Node.js implementation
+### Docker
 
-## Development
+```dockerfile
+FROM node:18-alpine
+WORKDIR /app
+COPY package*.json ./
+RUN npm ci --only=production
+COPY . .
+ENV NODE_ENV=production
+EXPOSE 3000
+CMD ["node", "dist/server.js"]
+```
 
-### Using Docker (Recommended)
+### Environment Variables
 
 ```bash
-# Install dependencies
-npm install
+# Database
+DB_HOST=postgres
+DB_PORT=5432
+DB_NAME=myapp
+DB_USER=postgres
+DB_PASSWORD=secret
+
+# SSL
+DB_SSL_ENABLED=true
+DB_SSL_CERT=/path/to/client-cert.pem
+DB_SSL_KEY=/path/to/client-key.pem
+DB_SSL_CA=/path/to/ca-cert.pem
+
+# Worker
+WORKER_QUEUE=default
+WORKER_INTERVAL=5000
+WORKER_MAX_ATTEMPTS=5
+
+# Dashboard
+DASHBOARD_ENABLED=true
+DASHBOARD_PORT=3000
+DASHBOARD_AUTH_ENABLED=true
+```
 
-# Start PostgreSQL with Docker
-npm run docker:up
+### Graceful Shutdown
 
-# Run tests
-npm test
+```typescript
+process.on('SIGTERM', async () => {
+  console.log('Shutting down...');
+  await worker.shutdown();
+  await client.close();
+  process.exit(0);
+});
+```
 
-# Run tests in watch mode
-npm run test:watch
+## Documentation
 
-# Build
-npm run build
+- [**Dashboard Guide**](./DASHBOARD.md) - Web UI setup and usage
+- [**SSL Configuration**](./SSL.md) - Secure connections
+- [**Docker Setup**](./DOCKER.md) - Container deployment
+- [**Contributing**](./CONTRIBUTING.md) - Development guide
 
-# Lint
-npm run lint
+## Examples
 
-# Stop Docker containers
-npm run docker:down
-```
+Complete examples are available in the [`examples/`](./examples) directory:
 
-### Docker Commands
+- `basic-usage.ts` - Simple job queue
+- `ssl-connection.ts` - SSL configurations
+- `dashboard-server.ts` - Complete server with dashboard
 
-- `npm run docker:up` - Start PostgreSQL and Adminer
-- `npm run docker:down` - Stop containers  
-- `npm run docker:logs` - View PostgreSQL logs
-- `npm run docker:clean` - Remove containers and volumes
-- `npm run test:docker` - Full test cycle with Docker
+## Support
 
-Access database admin at http://localhost:8080 (user: `que_user`, password: `que_password`)
+- 📖 [Documentation](https://github.com/your-username/worker-que#readme)
+- 🐛 [Issue Tracker](https://github.com/your-username/worker-que/issues)
+- 💬 [Discussions](https://github.com/your-username/worker-que/discussions)
 
-See [DOCKER.md](DOCKER.md) for detailed Docker documentation.
+## License
 
-### Manual Setup
+[MIT](./LICENSE) © 2026
 
-If you prefer not to use Docker, ensure PostgreSQL is running and create the database schema manually using `migrations/schema.sql`.
+## Acknowledgments
 
-## License
+worker-que is inspired by and compatible with:
+- [Que](https://github.com/chanks/que) by Chris Hanks
+- [que-go](https://github.com/bgentry/que-go) by Blake Gentry
+
+---
 
-MIT
+**Built with ❤️ using TypeScript and PostgreSQL**