@maestro-js/agent-skills 1.0.0-alpha.0

package/dist/maestro-packages/db.md

@@ -0,0 +1,391 @@
+---
+name: db
+description: "Use when working with @maestro-js/db, the database access package for Maestro. Covers MySQL driver setup, parameterized SQL queries (select, insert, update, delete, scalar, statement), transactions with automatic commit/rollback, streaming results via selectIterable, connection pooling, query formatting, and the provider pattern for multiple database instances. Maestro uses raw parameterized SQL with no ORM. Depends on service-registry and log. Used by queue for message storage. Works with the sql package for query builders."
+---
+
+# @maestro-js/db
+
+## Overview
+
+The `@maestro-js/db` package provides typed database access through the Maestro provider pattern. It wraps `mysql2` connection pools with parameterized query methods, automatic transactions, streaming result sets with backpressure, and structured logging. All queries use raw parameterized SQL -- there is no ORM.
+
+## Quick Setup
+
+```ts
+import { Db } from '@maestro-js/db'
+import { Log } from '@maestro-js/log'
+
+Db.Provider.register('default', Db.Provider.create({
+  driver: Db.drivers.mysql({
+    host: process.env.DB_HOST!,
+    user: process.env.DB_USER!,
+    password: process.env.DB_PASSWORD!,
+    database: process.env.DB_DATABASE!,
+    port: Number(process.env.DB_PORT ?? 3306)
+  })
+}))
+
+// Query immediately
+const users = await Db.select<User>('SELECT * FROM users WHERE active = ?', [true])
+```
+
+## Driver: MySQL (mysql2)
+
+The only built-in driver. Created via `Db.drivers.mysql(config)`.
+
+### Configuration
+
+```ts
+Db.drivers.mysql({
+  // Required
+  host: 'localhost',
+  user: 'root',
+  password: '',          // empty string is allowed, but the field is required
+  database: 'myapp',
+  port: 3306,
+
+  // Optional
+  connectionLimit: 10,   // max pool connections (default: 10)
+  debug: false,
+  multipleStatements: false,
+  ssl: undefined,        // string or tls.SecureContextOptions
+
+  // Optional loggers (default: Log.create())
+  connectionLogger: Log.create(),  // logs connection lifecycle events
+  queryLogger: Log.create()        // logs query start/complete/fail and transaction events
+})
+```
+
+### Type Casting
+
+The MySQL driver automatically casts:
+- `DATE` columns to ISO date strings
+- `TIME` columns to `iso-fns` time objects
+- `DATETIME` columns to ISO 8601 strings (UTC)
+- `TINYINT(1)` columns to booleans (`true`/`false`), with `null` preserved for nullable columns
+- JSON columns returned as strings (`jsonStrings: true`)
+- Decimal columns returned as numbers (`decimalNumbers: true`)
+
+### Connection Behavior
+
+- Pool uses `utf8mb4_0900_ai_ci` charset (MySQL 8.0+ default)
+- Connect timeout: 60 minutes
+- Connections are acquired with retry (up to 10 attempts, 1s delay)
+- Deadlock and lock-timeout errors are retried automatically (up to 3 attempts) for non-transaction queries
+
+## API Reference
+
+### select
+
+Execute a SELECT query and return typed results as an array.
+
+```ts
+select<T = Record<string, unknown>>(query: string, parameters?: any[]): Promise<T[]>
+```
+
+```ts
+interface User { id: number; name: string; active: boolean }
+
+const admins = await Db.select<User>('SELECT * FROM users WHERE role = ?', ['admin'])
+// [{ id: 1, name: 'Alice', active: true }, { id: 2, name: 'Bob', active: true }]
+```
+
+### selectIterable
+
+Execute a SELECT query and return a streaming async iterable with backpressure support. Uses `Helpers.AsyncIterableCollection` which provides `map`, `filter`, and `toArray` chainable methods.
+
+```ts
+selectIterable<T = Record<string, unknown>>(query: string, parameters?: any[]): Helpers.AsyncIterableCollection<T>
+```
+
+```ts
+const orders = Db.selectIterable<Order>('SELECT * FROM orders WHERE status = ?', ['pending'])
+
+for await (const order of orders) {
+  await processOrder(order)
+}
+
+// Or with chaining
+const names = await Db.selectIterable<User>('SELECT * FROM users')
+  .filter(user => user.active)
+  .map(user => user.name)
+  .toArray()
+```
+
+Backpressure: the driver pauses the MySQL connection when the internal row queue exceeds 3000 rows and resumes when it drains below that threshold.
+
+### insert
+
+Execute an INSERT query and return the affected row count and auto-increment ID.
+
+```ts
+insert(query: string, parameters?: any[]): Promise<{ affectedRows: number; insertId: number }>
+```
+
+```ts
+const { insertId } = await Db.insert(
+  'INSERT INTO users (name, email) VALUES (?, ?)',
+  ['Alice', 'alice@example.com']
+)
+// insertId: 42
+```
+
+### update
+
+Execute an UPDATE query and return matched and changed row counts.
+
+```ts
+update(query: string, parameters?: any[]): Promise<{ affectedRows: number; changedRows: number }>
+```
+
+```ts
+const { changedRows } = await Db.update(
+  'UPDATE users SET active = ? WHERE last_login < ?',
+  [false, '2024-01-01']
+)
+// changedRows: 15
+```
+
+### delete
+
+Execute a DELETE query and return the number of removed rows.
+
+```ts
+delete(query: string, parameters?: any[]): Promise<{ affectedRows: number }>
+```
+
+```ts
+const { affectedRows } = await Db.delete(
+  'DELETE FROM sessions WHERE expires_at < ?',
+  [new Date().toISOString()]
+)
+```
+
+### scalar
+
+Return the first column of the first row, or `null` if no results. Returns `null` both when the query returns zero rows and when the value itself is SQL NULL.
+
+```ts
+scalar<T = unknown>(query: string, parameters?: any[]): Promise<T | null>
+```
+
+```ts
+const count = await Db.scalar<number>('SELECT COUNT(*) FROM users WHERE active = ?', [true])
+// 42
+
+const name = await Db.scalar<string>('SELECT name FROM users WHERE id = ?', [999])
+// null (no matching row)
+```
+
+### statement
+
+Execute an arbitrary SQL statement and return the raw result. Use for DDL or other statements that do not fit the typed methods.
+
+```ts
+statement<T = unknown>(query: string, parameters?: any[]): Promise<T>
+```
+
+```ts
+await Db.statement('CREATE INDEX idx_users_email ON users (email)')
+```
+
+### transaction
+
+Wrap a callback in BEGIN/COMMIT/ROLLBACK. All `Db` calls awaited inside the callback share the same connection via `AsyncLocalStorage`. Nested transaction calls reuse the outer transaction (no savepoints).
+
+```ts
+transaction<T>(callback: () => Promise<T>): Promise<T>
+```
+
+```ts
+await Db.transaction(async () => {
+  await Db.update('UPDATE accounts SET balance = balance - ? WHERE id = ?', [100, fromId])
+  await Db.update('UPDATE accounts SET balance = balance + ? WHERE id = ?', [100, toId])
+})
+// Both updates commit together; if either throws, both roll back
+```
+
+### format
+
+Interpolate parameters into a query string for debugging. Does not execute the query.
+
+```ts
+format(query: string, params?: any[]): string
+```
+
+```ts
+const sql = Db.format('SELECT * FROM users WHERE id = ?', [42])
+// "SELECT * FROM users WHERE id = 42"
+```
+
+### destroy
+
+Close all connections in the pool and prevent new ones from being created.
+
+```ts
+destroy(): Promise<void>
+```
+
+## Common Patterns
+
+### Parameterized Queries
+
+Always use `?` placeholders. Never interpolate values into SQL strings.
+
+```ts
+// Correct
+const users = await Db.select<User>('SELECT * FROM users WHERE name = ?', ['Alice'])
+
+// WRONG - SQL injection risk
+const users = await Db.select<User>(`SELECT * FROM users WHERE name = '${name}'`)
+```
+
+### Avoid Table Name Aliases
+
+Write full table names in SQL queries instead of aliases when possible.
+
+```ts
+// Preferred
+await Db.select('SELECT users.name, orders.total FROM users JOIN orders ON orders.user_id = users.id')
+
+// Avoid
+await Db.select('SELECT u.name, o.total FROM users u JOIN orders o ON o.user_id = u.id')
+```
+
+### Transactions with Error Handling
+
+```ts
+try {
+  const orderId = await Db.transaction(async () => {
+    const { insertId } = await Db.insert(
+      'INSERT INTO orders (user_id, total) VALUES (?, ?)',
+      [userId, total]
+    )
+    await Db.insert(
+      'INSERT INTO order_items (order_id, product_id, quantity) VALUES (?, ?, ?)',
+      [insertId, productId, quantity]
+    )
+    await Db.update(
+      'UPDATE products SET stock = stock - ? WHERE id = ?',
+      [quantity, productId]
+    )
+    return insertId
+  })
+} catch (e) {
+  // All three statements rolled back
+}
+```
+
+### Streaming Large Result Sets
+
+Use `selectIterable` for queries returning many rows to avoid loading everything into memory.
+
+```ts
+const rows = Db.selectIterable<AuditLog>('SELECT * FROM audit_logs WHERE created_at > ?', ['2024-01-01'])
+
+let count = 0
+for await (const row of rows) {
+  await archiveToS3(row)
+  count++
+}
+```
+
+### Multiple Providers
+
+Register named providers for separate databases.
+
+```ts
+Db.Provider.register('default', Db.Provider.create({
+  driver: Db.drivers.mysql({ host: 'primary-db', user: 'app', password: '', database: 'myapp', port: 3306 })
+}))
+
+Db.Provider.register('analytics', Db.Provider.create({
+  driver: Db.drivers.mysql({ host: 'analytics-db', user: 'app', password: '', database: 'analytics', port: 3306 })
+}))
+
+// Use the default provider
+const users = await Db.select<User>('SELECT * FROM users')
+
+// Use a named provider
+const events = await Db.provider('analytics').select<Event>('SELECT * FROM events')
+```
+
+## Query Logging
+
+The MySQL driver emits structured log events via the `queryLogger` and `connectionLogger` options.
+
+Query logger events: `queryStarted`, `queryCompleted`, `queryFailed`, `transactionStarted`, `transactionCompleted`, `transactionFailed`. Each event includes the query ID, formatted query, duration (on completion/failure), and transaction ID when applicable.
+
+Connection logger events: `newConnection`, `connectionAcquisitionStarted`, `connectionAcquisitionCompleted`, `connectionAcquisitionFailed`.
+
+The log message type is exported as `Db.drivers.mysql.MysqlDbDriverLogMessageData`.
+
+## Testing
+
+The `@maestro-js/db` package does not currently have a mock driver or built-in tests. To test code that uses `Db`:
+
+1. **Use a real test database** -- spin up a MySQL instance (e.g., via Docker) and register a provider pointing to it.
+2. **Wrap tests in transactions** -- roll back after each test to keep the database clean.
+3. **Abstract database calls** -- write functions that accept a `Db.DbService` so you can substitute a test double.
+
+```ts
+// Test setup with a real database
+import { Db } from '@maestro-js/db'
+
+Db.Provider.register('default', Db.Provider.create({
+  driver: Db.drivers.mysql({
+    host: 'localhost',
+    user: 'test',
+    password: '',
+    database: 'myapp_test',
+    port: 3306
+  })
+}))
+
+// Clean up after tests
+afterAll(async () => {
+  await Db.destroy()
+})
+```
+
+## Cross-Package Integration
+
+### Depends On
+
+- **@maestro-js/service-registry** -- provider pattern (create, register, resolve)
+- **@maestro-js/log** -- structured logging for queries and connections
+- **@maestro-js/context** -- `AsyncLocalStorage`-based request context for scoping log data
+- **@maestro-js/helpers** -- retry logic, `AsyncIterableCollection` for streaming
+
+### Used By
+
+- **@maestro-js/queue** -- stores queue messages and job state in MySQL tables via `Db`
+
+### Works With
+
+- **@maestro-js/sql** -- provides query builder utilities (`Sql.where`, CTEs) that produce parameterized SQL strings compatible with `Db.select`, `Db.update`, etc.
+
+```ts
+import { Db } from '@maestro-js/db'
+import { Sql } from '@maestro-js/sql'
+
+const where = Sql.where({ active: true, role: 'admin' })
+const users = await Db.select<User>(`SELECT * FROM users ${where.sql}`, where.params)
+```
+
+## DbDriver Interface
+
+Implement this interface to create a custom driver.
+
+```ts
+interface DbDriver {
+  selectIterable(query: string, parameters?: any[]): AsyncIterable<Record<string, unknown>>
+  execute(
+    query: string,
+    parameters?: any[]
+  ): Promise<Record<string, unknown>[] | { affectedRows: number; changedRows: number; insertId: number }>
+  format(query: string, parameters: any[]): string
+  transaction<T>(callback: () => Promise<T>): Promise<T>
+  destroy(): Promise<void>
+}
+```

package/dist/maestro-packages/events.md

@@ -0,0 +1,295 @@
+---
+name: events
+description: "Guide for @maestro-js/events — a pub/sub messaging package with pluggable drivers (Redis, WebSocket, Local). Use when working with @maestro-js/events, adding event-driven communication, creating typed event handles, configuring Redis pub/sub, setting up IPC via WebSocket, using in-process local events, or integrating with the Broadcast package. Covers the Provider pattern, driver configuration, typed Event handles, listener lifecycle, and cross-package usage with service-registry and broadcast."
+---
+
+# @maestro-js/events
+
+Pub/sub messaging layer with pluggable drivers for Redis (cross-server), WebSocket (IPC between Node processes), and Local (in-process). Follows the standard Maestro Provider pattern built on `@maestro-js/service-registry`.
+
+## Quick Setup
+
+```ts
+import { Events } from '@maestro-js/events'
+
+// Create and register with a driver
+const service = Events.Provider.create({ driver: Events.drivers.local() })
+Events.Provider.register('default', service)
+
+// Use via the default facade
+Events.emit('order-placed', { orderId: 'order-789', total: 59.99 })
+```
+
+## Drivers
+
+### Local (In-Process)
+
+Use `Events.drivers.local()` for in-process events. Returns a Node.js `EventEmitter`. Best for testing and single-process applications.
+
+```ts
+const service = Events.Provider.create({
+  driver: Events.drivers.local()
+})
+Events.Provider.register('default', service)
+```
+
+### Redis (Cross-Server)
+
+Use `Events.drivers.redis(config)` for cross-server pub/sub via Redis. Creates separate publisher and subscriber connections with lazy initialization. Messages are JSON-serialized over Redis channels prefixed with `maestro.events.`.
+
+```ts
+const service = Events.Provider.create({
+  driver: Events.drivers.redis({
+    url: 'redis-host.example.com',
+    port: 6379,
+    operationTimeout: 5000,   // optional, default 5000ms
+    connectTimeout: 10000,    // optional, default 10000ms
+    keyPrefix: 'myapp:',      // optional, default ''
+    logger: customLogger      // optional, defaults to Log.create()
+  })
+})
+Events.Provider.register('default', service)
+```
+
+Redis driver config interface:
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `url` | `string` | required | Redis host URL |
+| `port` | `number` | required | Redis port |
+| `operationTimeout` | `number` | `5000` | Timeout for publish operations (ms) |
+| `connectTimeout` | `number` | `10000` | Connection timeout (ms) |
+| `keyPrefix` | `string` | `''` | Prefix prepended to all channel names |
+| `logger` | `Log.LogFunctions` | `Log.create()` | Logger instance for connection/publish/subscribe logs |
+
+Redis channel naming: channels are formatted as `${keyPrefix}maestro.events.${eventName}`.
+
+### WebSocket (IPC Between Processes)
+
+Use `Events.drivers.socket(config)` for IPC between Node.js processes via TCP sockets. Automatically creates a server if none exists and connects as a client. Reconnects on socket close.
+
+```ts
+const service = Events.Provider.create({
+  driver: Events.drivers.socket({ port: 8444 })
+})
+Events.Provider.register('default', service)
+```
+
+The socket driver serializes messages as newline-delimited JSON (`{ eventName, data }\n`). One process acts as the server (created automatically if the port is available), and all processes connect as clients.
+
+## API Reference
+
+### Provider Pattern
+
+```ts
+// Create a service instance with a driver
+Events.Provider.create(config: { driver: Events.EventsDriver }): Events.EventsService
+
+// Register under a named key
+Events.Provider.register(name: string, service: Events.EventsService): void
+
+// Resolve a named instance (deferred via Proxy)
+Events.provider(key: string): Events.EventsService
+```
+
+### Direct API (Untyped)
+
+Operate on the default facade without typed handles.
+
+```ts
+// Emit an event with a payload
+Events.emit(name: string, message: Record<string, any>): void
+
+// Subscribe to an event, returns a handle with remove()
+Events.addListener<T>(event: string, listener: (message: T) => unknown): { remove(): void }
+
+// Remove a specific listener by reference
+Events.removeListener(event: string, listener: (message: Record<string, any>) => unknown): void
+
+// Remove all listeners and tear down driver connections
+Events.removeAllListeners(): void
+```
+
+### Typed Event Handles
+
+Create a typed event scoped to a single name. All methods enforce the payload type `T` at compile time.
+
+```ts
+// Create a typed event handle
+Events.create<T extends Record<string, any>>(eventName: string): Events.Event<T>
+```
+
+The returned `Event<T>` interface:
+
+```ts
+interface Event<T extends Record<string, any>> {
+  emit(message: T): void
+  addListener(listener: (message: T) => unknown): { remove(): void }
+  removeListener(listener: (message: T) => unknown): void
+}
+```
+
+### EventsDriver Interface
+
+All drivers implement this interface:
+
+```ts
+type EventsDriver = {
+  addListener(event: string, listener: (message: Record<string, any>) => void): unknown
+  removeListener(event: string, listener: (message: Record<string, any>) => void): unknown
+  emit(name: string, message: Record<string, any>): unknown
+  removeAllListeners(): unknown
+}
+```
+
+### Type Augmentation for Provider Keys
+
+Extend provider keys for type-safe multi-instance resolution:
+
+```ts
+declare namespace Events {
+  namespace Provider {
+    interface Keys {
+      default: unknown
+      analytics: unknown
+    }
+  }
+}
+```
+
+## Common Patterns
+
+### Typed Event Handles
+
+Define typed events for compile-time payload safety:
+
+```ts
+const orderPlaced = Events.create<{ orderId: string; total: number }>('order-placed')
+
+const sub = orderPlaced.addListener((msg) => {
+  console.log(`Order ${msg.orderId} placed for $${msg.total}`)
+})
+
+orderPlaced.emit({ orderId: 'order-1', total: 42.5 })
+
+// Unsubscribe
+sub.remove()
+```
+
+### Listener Removal by Reference
+
+```ts
+const listener = (msg: Record<string, any>) => console.log(msg)
+Events.addListener('my-event', listener)
+
+// Later, remove by reference
+Events.removeListener('my-event', listener)
+```
+
+### Multiple Listeners on One Event
+
+```ts
+Events.addListener('user-signup', (msg) => sendWelcomeEmail(msg))
+Events.addListener('user-signup', (msg) => trackAnalytics(msg))
+
+Events.emit('user-signup', { userId: 'u-123', email: 'alice@example.com' })
+// Both listeners fire
+```
+
+### Multiple Named Instances
+
+```ts
+const redisEvents = Events.Provider.create({
+  driver: Events.drivers.redis({ url: 'redis-host', port: 6379 })
+})
+Events.Provider.register('default', redisEvents)
+
+const localEvents = Events.Provider.create({
+  driver: Events.drivers.local()
+})
+Events.Provider.register('internal', localEvents)
+
+// Use each instance
+Events.provider('default').emit('cross-server-event', { data: 1 })
+Events.provider('internal').emit('local-only-event', { data: 2 })
+```
+
+### Cleanup on Shutdown
+
+```ts
+// Tears down driver connections (Redis disconnect, socket close) and removes all listeners
+Events.removeAllListeners()
+```
+
+## Testing
+
+Use the Local driver for tests. It is synchronous and requires no external services.
+
+```ts
+import { Events } from '@maestro-js/events'
+import { test } from 'beartest-js'
+import { expect } from 'expect'
+
+const service = Events.Provider.create({ driver: Events.drivers.local() })
+
+test('emit + addListener fires listener with correct payload', () => {
+  const received: any[] = []
+  service.addListener('test-event', (msg) => received.push(msg))
+  service.emit('test-event', { hello: 'world' })
+  expect(received).toEqual([{ hello: 'world' }])
+  service.removeAllListeners()
+})
+
+test('typed events via create()', () => {
+  const orderPlaced = service.create<{ orderId: string; total: number }>('order-placed')
+  const received: any[] = []
+  orderPlaced.addListener((msg) => received.push(msg))
+  orderPlaced.emit({ orderId: 'order-1', total: 42.5 })
+  expect(received).toEqual([{ orderId: 'order-1', total: 42.5 }])
+  service.removeAllListeners()
+})
+```
+
+Always call `service.removeAllListeners()` after each test to prevent listener leaks.
+
+Run tests:
+
+```bash
+pnpm --filter @maestro-js/events test
+cd packages/events && npx beartest ./tests/events.test.ts
+```
+
+## Cross-Package Integration
+
+### Dependencies
+
+- **@maestro-js/service-registry** -- foundation for the Provider pattern (create, register, resolve)
+- **@maestro-js/log** -- used by the Redis driver for structured logging of connections, publishes, and subscribes
+
+### Used By: Broadcast
+
+The `@maestro-js/broadcast` package accepts an Events service instance as its driver. The Events service satisfies the `BroadcastingDriver` interface (`emit`, `addListener`, `removeListener`, `removeAllListeners`).
+
+```ts
+import { Events } from '@maestro-js/events'
+import { Broadcast } from '@maestro-js/broadcast'
+
+const users = Broadcast.channel('/user/:id').create<{ name: string }>()
+
+const broadcast = Broadcast.Provider.create({
+  channels: [users],
+  driver: Events   // Events facade satisfies BroadcastingDriver
+})
+Broadcast.Provider.register('default', broadcast)
+```
+
+## Source Files
+
+| File | Purpose |
+|------|---------|
+| `packages/events/src/index.ts` | Main export, Provider pattern, facade, driver registry |
+| `packages/events/src/events-types.ts` | `EventsDriver` interface definition |
+| `packages/events/src/redis-events-driver.ts` | Redis pub/sub driver using ioredis |
+| `packages/events/src/sockets-events-driver.ts` | TCP socket IPC driver for inter-process events |
+| `packages/events/tests/events.test.ts` | Unit tests using Local driver |
+| `packages/events/tests/socket-events-driver.test.ts` | Socket driver integration test (skipped by default) |