qhttpx 1.8.0

package/docs/ROADMAP.md

@@ -0,0 +1,366 @@
+# QHTTPX Roadmap
+
+## Overview
+
+QHTTPX is evolving toward a **lightweight, composable runtime** that prioritizes structural efficiency over microbenchmarks. Each phase targets a specific bottleneck in the HTTP request path, guided by the principle: *lighter core, more powerful abstractions*.
+
+---
+
+## Current State: v0.1
+
+### Phase 1 – Hot-path Allocations ✅
+
+**Status: Implemented and Preallocated**
+
+#### Context Pooling
+- `contextPool: QHTTPXContext[]` preallocates up to `maxConcurrency` contexts at startup
+- `acquireContext()` pulls from pool and resets fields
+- `releaseContext()` cleans and returns to pool
+- **Effect**: No new context allocations under normal load; stable object shapes for hidden-class optimization
+
+#### Buffer Reuse
+- Engine supports pluggable `jsonSerializer?: (value: unknown) => string | Buffer`
+- Apps can precompile response buffers (e.g., health checks, metrics)
+- Benchmarks demonstrate zero-allocation JSON on hot paths
+- **Effect**: Users can achieve static buffer responses; mechanism generalizable
+
+#### Outcome
+- GC pauses reduced; p99 latencies stabilized
+- Foundation for buffer pools and serialization optimization
+
+---
+
+### Phase 2 – Routing (Partial) ⚠️
+
+**Status: Functional but Not Optimized**
+
+#### Current Implementation
+- Simple `RouteDefinition[]` list
+- Linear scan per HTTP method
+- Per-request normalization: `path.split('/')`, then segment matching
+
+#### What's Missing
+- No per-method route buckets (still scans all routes regardless of method)
+- No freeze-on-boot design (routes can be added/modified at any time)
+- No compiled/flattened structure (no radix tree, no array indices)
+- Per-request overhead: string splits, object allocations in match logic
+
+---
+
+### Phase 3 – Zero-cost Middleware ✅
+
+**Status: Implemented (Lighter Pipeline)**
+
+#### Pipeline Flattening
+- Removed per-request stack copy combining middlewares + route handler
+- Route handler no longer wrapped as extra middleware
+- Simple Koa-compatible dispatch loop: `middlewares[i](ctx, next)`
+- **Effect**: Fewer allocations; maintains familiar middleware API
+
+#### What's Not Yet Done
+- No precompiled runner (still dynamically dispatches through middleware chain)
+- Could evolve to single-loop execution with inlined middleware logic
+
+---
+
+### Phase 4 – Serialization (Partial) ✅
+
+**Status: Pluggable Hook Available**
+
+- Engine supports custom `jsonSerializer` at options level
+- Hot paths can use prebuilt Buffers
+- Mechanism ready for future schema compilers (e.g., fast-json-stringify style)
+
+#### What's Not Yet Done
+- No built-in schema compiler integration
+- No automatic path detection (e.g., `/health` → static buffer)
+- General buffer pools (small/medium/large) not yet part of runtime
+
+---
+
+### Phase 5+ – Scheduler & Runtime
+
+**Status: Skeleton Present**
+
+- `Scheduler` class with concurrency model
+- `TaskEngine` for background tasks
+- **Not yet**: Per-worker queues, work-stealing, lock-free primitives
+
+---
+
+## v0.2: Focus on Router & Consolidation
+
+### Goals
+1. **Make routing lazy-safe and indexable**: Support per-method buckets and freeze-on-boot design
+2. **Improve measurability**: Track what v0.1 achieved; establish baselines for v0.2 improvements
+3. **Maintain backward compatibility**: Existing API surface unchanged
+
+### Scope
+
+#### 1. Router Optimization – Frozen, Per-Method Buckets
+**Deliverable**: `router.ts` refactor
+
+- **Step 1**: Reorganize routes internally into per-method buckets
+  - `routes: { GET: RouteDefinition[], POST: RouteDefinition[], ... }`
+  - Reduce scan loop overhead by method-specific lookups
+  
+- **Step 2**: Introduce freeze-on-boot pattern
+  - After `listen()` is called, warn (or optionally error) on new route registrations
+  - Document that routes must be registered before server start
+  
+- **Step 3**: Prebuild derived structures
+  - Static routes table (path → handler, no params)
+  - Parameterized routes list (faster fallback for `/foo/:id` patterns)
+  - Prepare for radix compilation in v0.3
+
+**Why v0.2**: Routing is a bottleneck on every request. Fixing it early unlocks measurable gains and clearer architecture for v0.3's radix tree.
+
+**Risk**: Low. Public API unchanged; internals reorganized.
+
+---
+
+#### 2. Buffer Pools (Optional, If Time)
+**Deliverable**: Internal buffer pool API
+
+- Small/medium/large pools for response bodies
+- Available to engine and future plugins
+- Keep mechanism simple: leak-proof, no external memory pressure
+
+**Why optional**: Nice-to-have for future buffer reuse. Can ship in v0.2 if implementation is lightweight; defer to v0.3 if it adds complexity.
+
+---
+
+#### 3. Documentation & Baselines
+**Deliverable**: Updated `docs/`
+
+- Clarify v0.1 achievements (context pooling, pipeline flattening, pluggable JSON)
+- Benchmark baseline for v0.2 router changes
+- Explain freezing pattern and its implications for startup/shutdown
+
+---
+
+### Out of Scope for v0.2
+- Radix tree implementation (defer to v0.3)
+- Precompiled middleware runner (v0.3+)
+- Scheduler work-stealing (v0.5+)
+- Built-in schema compiler (v0.3+)
+
+---
+
+## v0.3: Radix Router & Middleware Compilation
+
+### Goals
+1. **Flatten routing further**: Build radix tree at boot, match in tight loop
+2. **Precompile middleware chains**: Remove per-request dispatch overhead
+3. **Introduce buffer pools formally**: General-purpose pooling for response bodies
+
+### Scope
+
+#### 1. Radix Router
+- Build tree structure at `listen()` time
+- Flatten into arrays: `{ charCode, childIndex, handlerIndex, paramKeyIndex }`
+- Match becomes a tight loop with zero allocations
+- Support param extraction and wildcard routes
+
+#### 2. Middleware Precompilation
+- When `use()` or routes are registered, compile a single runner function
+- Runner is array of middleware + final handler, executed in simple loop
+- Maintains Koa semantics, removes dispatch overhead
+
+#### 3. Buffer Pools Formalized
+- Integrate into `server.ts` as standard pools (S/M/L)
+- Make available to route handlers via context
+- Document pool semantics (request-scoped resets)
+
+---
+
+## v0.5: Scheduler & Lock-Free Queues
+
+### Goals
+1. **Per-worker concurrency**: Ring buffers, per-core task queues
+2. **Work-stealing**: Tasks move to idle cores
+3. **Unified HTTP + Task scheduling**: Same primitives, different priorities
+
+### Scope
+- Refactor `scheduler.ts` and `tasks.ts`
+- Introduce lock-free work queues
+- Integrate with existing concurrency model
+
+---
+
+## Key Principles (All Phases)
+
+1. **Lighter, not faster**: Optimize structure, not just throughput
+2. **Backward compatible**: Public API stays stable across minor versions
+3. **Measurable**: Baseline before, measure after, document gains
+4. **Composable**: Each phase builds on previous; no regressions
+5. **No benchmark chasing**: Focus on real bottlenecks (allocation, dispatch, lookup)
+
+---
+
+## Success Criteria
+
+### v0.2
+- [ ] Router per-method buckets implemented
+- [ ] Freeze-on-boot pattern enforced
+- [ ] Derived structures (static/param routes) prebuild correctly
+- [ ] Tests pass; backward compatibility verified
+- [ ] Baseline benchmarks established (context pooling, pipeline, router latency)
+
+### v0.3
+- [ ] Radix tree built at boot; match is tight loop
+- [ ] Middleware chain precompiled; no per-request dispatch
+- [ ] Buffer pools integrated into `server.ts`
+- [ ] P50/P99 latencies improve measurably vs v0.2
+- [ ] Public API unchanged
+
+### v0.5
+- [ ] Scheduler supports per-worker queues
+- [ ] Work-stealing implemented and tested
+- [ ] HTTP and task scheduling unified under same primitives
+- [ ] System handles burst loads better (work moves to idle cores)
+
+---
+
+## Next Steps
+
+1. **Immediate**: Start v0.2 router refactor (per-method buckets, freeze pattern)
+2. **Parallel**: Update benchmarks to track v0.2 improvements
+3. **Document**: Clarify startup/shutdown semantics with frozen routes
+
+---
+
+## Framework & Production Roadmap
+
+This section focuses on framework-level features: production readiness, observability, safety, and capabilities that differentiate QHTTPX from frameworks like Express and Fastify.
+
+### v0.4 – Production Framework Essentials ✅
+
+**Status: Implemented**
+
+**Goals**
+
+- Make behavior predictable in real services (errors, shutdown, config).
+- Provide first-class observability and safety, not just hot-path speed.
+- Keep the core small and composable.
+
+**Scope**
+
+1. **Error Handling & HTTP Error Model**
+   - Introduce an `HttpError` type with `status`, `code`, `details`.
+   - Add an opt-in global error middleware:
+     - Catches thrown errors from handlers/middlewares.
+     - Maps `HttpError` to consistent JSON or text responses.
+     - Hides internal errors behind a generic 500 except in development.
+   - Provide helpers to register custom 404 and 405 handlers instead of hardcoded text responses.
+
+2. **Configuration & Environment Handling**
+   - Clarify how `QHTTPXOptions` is used for runtime configuration.
+   - Provide a recommended pattern for constructing `QHTTPX` instances from environment variables.
+   - Optionally expose a small `loadConfig({ env, defaults })` helper to centralize configuration parsing and validation.
+
+3. **Graceful Shutdown & Lifecycle Hooks**
+   - Add lifecycle hooks on the `QHTTPX` instance, such as `onStart`, `onShutdown`, `onBeforeShutdown`.
+   - Implement graceful shutdown:
+     - Stop accepting new connections.
+     - Let in-flight requests complete up to a configurable timeout.
+     - Flush metrics and background tasks before process exit.
+   - Provide a helper to wire process signals, e.g. `attachSignalHandlers(app, { signals: ['SIGINT', 'SIGTERM'] })`.
+
+4. **Observability & Request Identity**
+   - Extend metrics and logger support with optional request ID correlation:
+     - Generate a `x-request-id` when missing.
+     - Expose the request ID on the context and in log entries/metrics.
+   - Add a minimal interface for integrating external tracers (e.g., OpenTelemetry) without coupling QHTTPX to specific libraries.
+   - Promote a structured logging example (JSON sink) as the recommended production default.
+
+5. **Security Baseline Pack**
+   - Combine CORS and security headers middlewares into a recommended preset:
+     - `createSecureDefaults({ cors, csp, hsts, ... })` returning a set of middlewares.
+   - Add a simple rate-limiting middleware (token bucket per IP or identifier) with pluggable storage, starting with in-memory.
+   - Extend `readBody` handling to enforce a `maxBodyBytes` option and return 413 on oversized requests.
+
+6. **Testing Utilities**
+   - Provide a lightweight testing helper:
+     - `createTestClient(app: QHTTPX)` to start the server on a random port.
+     - `client.request({ method, path, body, headers })` for integration tests.
+   - Ensure it works smoothly with Vitest and document recommended test patterns.
+
+---
+
+### v0.6 – Differentiating Features vs Fastify/Express
+
+**Goals**
+
+- Turn QHTTPX into a resource-aware HTTP runtime, not just a router.
+- Offer built-in quality-of-service and safety mechanisms.
+- Leverage existing scheduler, task engine, and metrics as first-class features.
+
+**Scope**
+
+1. **Unified HTTP + Background Task Runtime**
+   - Elevate the existing `TaskEngine` and `Scheduler` into a core story:
+     - Clear API to register tasks tied to the application lifecycle.
+     - Expose task metrics (queue depths, retries, failures) via `/__qhttpx/metrics`.
+     - Introduce simple task priority classes (e.g., high vs low priority queues).
+
+2. **Resource-Aware Admission Control**
+   - Extend resource checks beyond RSS to include:
+     - In-flight requests.
+     - Scheduler queue sizes.
+     - Latency metrics (e.g., p99).
+   - Introduce a load shedding mechanism:
+     - Optionally classify routes as "critical" or "best-effort".
+     - Shed lower-priority requests under high load (returning 503).
+
+3. **Latency Budgets & SLO-Aware Routing**
+   - Allow per-route or per-group latency budgets (e.g., `100ms`).
+   - Track budget violations and optionally:
+     - Reject or degrade noncritical routes.
+   - Provide simple SLO classes such as `critical`, `standard`, `background`.
+
+4. **Concurrency & Priority Classes for Requests**
+   - Build on `SchedulerOptions` and per-worker queues:
+     - Annotate routes with priority and map them to worker queues.
+   - Route latency-sensitive paths to high-priority queues and batch/admin endpoints to lower-priority queues.
+
+5. **First-Class Streaming & SSE Patterns**
+   - Promote existing SSE and streaming helpers into a "live services" story:
+     - Provide patterns for broadcasting events to many SSE clients.
+     - Ensure backpressure-aware file and data streaming.
+   - Integrate connection counts and streaming durations into metrics.
+
+6. **Introspectable Runtime**
+   - Extend `/__qhttpx/metrics` and related endpoints to expose:
+     - Per-worker queue statistics.
+     - Task engine statistics (registered tasks, pending retries).
+     - Router statistics (route counts, frozen state).
+   - Optionally add `/__qhttpx/runtime` for a JSON summary of runtime state (excluding secrets).
+
+---
+
+### v0.8 – Developer Experience & Ergonomics
+
+**Goals**
+
+- Improve ergonomics without bloating the core engine.
+- Provide opinionated presets that are still decomposable into primitives.
+
+**Scope**
+
+1. **Routing Ergonomics**
+   - Add route builders such as `app.route('/users/:id')` to group methods.
+   - Provide a typed helper layer for TypeScript users:
+     - Wrap `QHTTPXHandler` with generics for params, query, body, and response types.
+
+2. **Preset Stacks**
+   - "API preset":
+     - JSON responses, CORS, logger, metrics, error handler, security headers, and size limits.
+   - "Static site + API preset":
+     - Adds static middleware with recommended settings on top of the API preset.
+   - Implement presets as thin helpers that call `app.use(...)` with existing middlewares.
+
+3. **CLI (Optional)**
+   - Introduce a minimal CLI to scaffold starter projects:
+     - Example app with `/health`, `/metrics`, SSE example, a background task, and basic tests.
+   - Keep the CLI small and optional so QHTTPX remains primarily a runtime and library.

package/docs/VALIDATION.md

@@ -0,0 +1,136 @@
+# Input Validation & Coercion
+
+QHTTPX v1.2.0 introduces a powerful, zero-dependency validation system. It ensures that incoming data (Body, Query Parameters, and URL Params) matches your expectations before your handler ever runs.
+
+## Features
+
+- **Zero Dependency**: Built-in `SimpleValidator` handles common use cases without bloating your `node_modules`.
+- **Automatic Coercion**: Automatically converts query parameters (which are always strings) into numbers or booleans based on your schema.
+- **Type Safety**: Ensures `ctx.body`, `ctx.query`, and `ctx.params` contain valid data.
+- **Extensible**: You can plug in your own validator (like Zod, Joi, or TypeBox) if you need more power.
+
+## Basic Usage
+
+Define a `schema` object in your route options. The schema can have `body`, `query`, `params`, and `response` sections.
+
+```typescript
+import { createHttpApp } from 'qhttpx';
+
+const app = createHttpApp();
+
+app.post('/users', {
+  schema: {
+    // Validate Request Body
+    body: {
+      type: 'object',
+      properties: {
+        username: { type: 'string', min: 3, max: 20 },
+        age: { type: 'number', min: 18 },
+        email: { type: 'string', pattern: '^\\S+@\\S+\\.\\S+$' },
+        tags: { type: 'array', items: { type: 'string' } }
+      },
+      required: true // default is true
+    },
+    // Validate Query Parameters
+    query: {
+      type: 'object',
+      properties: {
+        ref: { type: 'string' }
+      }
+    }
+  },
+  handler: (ctx) => {
+    // TypeScript knows these are valid if you cast them, 
+    // or trust the runtime validation.
+    const { username, age } = ctx.body as any;
+    
+    ctx.json({ 
+      status: 'created', 
+      user: { username, age } 
+    });
+  }
+});
+```
+
+## Schema Reference
+
+The built-in `SimpleValidator` supports the following types:
+
+### `string`
+- `min`: Minimum length
+- `max`: Maximum length
+- `pattern`: Regex pattern (string)
+- `enum`: Array of allowed values
+
+### `number`
+- `min`: Minimum value
+- `max`: Maximum value
+- `enum`: Array of allowed values
+
+### `boolean`
+- Coerces `"true"`, `"false"` strings to booleans automatically.
+
+### `array`
+- `min`: Minimum items
+- `max`: Maximum items
+- `items`: Schema for array items
+
+### `object`
+- `properties`: Map of field names to schemas.
+
+## Automatic Coercion
+
+Query parameters are inherently strings. The validator automatically converts them for you.
+
+```typescript
+// URL: /items?page=5&active=true
+
+app.get('/items', {
+  schema: {
+    query: {
+      type: 'object',
+      properties: {
+        page: { type: 'number' },    // "5" -> 5
+        active: { type: 'boolean' } // "true" -> true
+      }
+    }
+  },
+  handler: (ctx) => {
+    const page = ctx.query.page; // This is now a number!
+    // ...
+  }
+});
+```
+
+## Custom Validators (Zod, etc.)
+
+If you prefer a library like Zod, you can implement the `Validator` interface and pass it to the app options.
+
+```typescript
+import { z } from 'zod';
+import { Validator, ValidationResult } from 'qhttpx';
+
+class ZodValidator implements Validator {
+  validate(schema: any, data: any): ValidationResult {
+    const result = schema.safeParse(data);
+    if (result.success) {
+      return { success: true, data: result.data };
+    }
+    return { success: false, error: result.error };
+  }
+}
+
+const app = createHttpApp({
+  validator: new ZodValidator()
+});
+
+// Now use Zod schemas in your routes!
+app.post('/zod', {
+  schema: {
+    body: z.object({
+      name: z.string().min(3)
+    })
+  },
+  handler: (ctx) => { ... }
+});
+```

package/eslint.config.cjs

@@ -0,0 +1,26 @@
+const tsParser = require("@typescript-eslint/parser");
+const tsPlugin = require("@typescript-eslint/eslint-plugin");
+const prettierPlugin = require("eslint-plugin-prettier");
+
+module.exports = [
+  {
+    files: ["**/*.ts"],
+    ignores: ["dist/**", "node_modules/**"],
+    languageOptions: {
+      parser: tsParser,
+      parserOptions: {
+        project: "./tsconfig.json",
+        tsconfigRootDir: __dirname,
+        sourceType: "module",
+      },
+    },
+    plugins: {
+      "@typescript-eslint": tsPlugin,
+      prettier: prettierPlugin,
+    },
+    rules: {
+      ...tsPlugin.configs.recommended.rules,
+      "prettier/prettier": "off",
+    },
+  },
+];