jsgui3-server 0.0.150 → 0.0.152

package/docs/middleware-guide.md

@@ -0,0 +1,236 @@
+# Middleware Guide
+
+jsgui3-server includes a lightweight middleware pipeline inspired by Express's `app.use()`. Middleware functions run **before** every request reaches the router, letting you add cross-cutting concerns — compression, CORS, logging, auth — without modifying route handlers.
+
+## Quick Start
+
+```javascript
+const Server = require('jsgui3-server');
+const { compression } = require('jsgui3-server/middleware');
+
+const server = new Server({ Ctrl: My_Control, src_path_client_js: __dirname + '/client.js' });
+
+// Enable gzip/deflate/brotli compression for JSON, HTML, CSS, JS responses
+server.use(compression());
+
+server.on('ready', () => server.start(8080));
+```
+
+Or via `Server.serve()`:
+
+```javascript
+Server.serve({
+    Ctrl: My_Control,
+    src_path_client_js: __dirname + '/client.js',
+    compression: true,          // shorthand: enables compression with defaults
+    port: 8080
+});
+```
+
+---
+
+## API
+
+### `server.use(fn)`
+
+Register a middleware function. Middleware is executed in registration order before the request reaches the router.
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `fn` | `function(req, res, next)` | Middleware function |
+
+**Returns:** `this` (chainable)
+
+**Middleware signature:**
+```javascript
+function my_middleware(req, res, next) {
+    // Do work before routing
+    // ...
+    next();      // Continue to next middleware / router
+    // -- or --
+    next(err);   // Skip remaining middleware, trigger error handler
+}
+```
+
+**Example — request logger:**
+```javascript
+server.use((req, res, next) => {
+    console.log(`${req.method} ${req.url}`);
+    next();
+});
+```
+
+**Example — CORS headers:**
+```javascript
+server.use((req, res, next) => {
+    res.setHeader('Access-Control-Allow-Origin', '*');
+    res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
+    res.setHeader('Access-Control-Allow-Headers', 'Content-Type');
+    if (req.method === 'OPTIONS') {
+        res.writeHead(204);
+        res.end();
+        return; // Don't call next() — request is handled
+    }
+    next();
+});
+```
+
+### `Server.serve()` Options
+
+| Option | Type | Description |
+|--------|------|-------------|
+| `middleware` | `function \| function[]` | One or more middleware functions to register |
+| `compression` | `boolean \| object` | Enable built-in compression middleware. Pass `true` for defaults or an options object |
+
+```javascript
+Server.serve({
+    Ctrl: My_Control,
+    src_path_client_js: __dirname + '/client.js',
+    middleware: [
+        (req, res, next) => { console.log(req.url); next(); }
+    ],
+    compression: { threshold: 512 },
+    port: 8080
+});
+```
+
+---
+
+## Built-in Middleware
+
+### `compression([options])`
+
+Response compression middleware. Transparently compresses response bodies when:
+
+1. The client sends an `Accept-Encoding` header matching a supported algorithm
+2. The response `Content-Type` is compressible (JSON, HTML, CSS, JS, XML, SVG, plain text)
+3. The body size meets or exceeds the threshold
+
+Streaming responses (SSE, chunked writes via `res.write()`) are passed through uncompressed.
+
+```javascript
+const { compression } = require('jsgui3-server/middleware');
+server.use(compression());
+```
+
+**Options:**
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `threshold` | `number` | `1024` | Minimum body size in bytes to compress |
+| `level` | `number` | `Z_DEFAULT_COMPRESSION` | zlib compression level (1–9, or -1 for default) |
+
+**Encoding negotiation priority:** gzip → deflate → brotli
+
+gzip is preferred for dynamic content because it offers the best speed/ratio trade-off. Brotli provides better compression ratios but is significantly slower for on-the-fly compression; it's more appropriate for pre-compressed static assets.
+
+**Compressible content types:**
+- `application/json`
+- `text/html`, `text/plain`, `text/css`, `text/xml`, `text/csv`
+- `application/javascript`, `text/javascript`
+- `application/xml`, `application/xhtml+xml`, `application/manifest+json`
+- `image/svg+xml`
+
+**What is NOT compressed:**
+- Bodies smaller than `threshold`
+- Non-text content types (images, video, binary)
+- Responses with an existing `Content-Encoding` header
+- Streaming responses (where `res.write()` is called before `res.end()`)
+
+---
+
+## How It Works
+
+### Execution Order
+
+```
+HTTP Request
+  │
+  ▼
+middleware[0](req, res, next)  ──next()──►
+  middleware[1](req, res, next)  ──next()──►
+    ...
+      middleware[n](req, res, next)  ──next()──►
+        server_router.process(req, res)  ◄── routing + publishers
+```
+
+### Response Wrapping (Compression Pattern)
+
+The compression middleware intercepts `res.writeHead()` and `res.end()`:
+
+1. **`res.writeHead()`** — Buffers the status code and headers instead of sending them immediately
+2. **`res.end(body)`** — Checks the content type and body size; if compressible, compresses the body, updates `Content-Encoding` and `Content-Length` headers, then sends everything
+3. **`res.write(chunk)`** — If called (streaming), flushes buffered headers and passes writes through unmodified
+
+This means compression is transparent to publishers and route handlers — they continue to write responses normally.
+
+---
+
+## Writing Custom Middleware
+
+### Basic Pattern
+
+```javascript
+function my_middleware(req, res, next) {
+    // 1. Pre-processing (before routing)
+    //    Modify req, set headers, check auth, etc.
+
+    // 2. Continue the chain
+    next();
+
+    // 3. Post-processing is NOT supported in this simple model.
+    //    For response wrapping, intercept res.end() like compression does.
+}
+```
+
+### Response Wrapping Pattern
+
+To modify the response body (like compression does), intercept `res.end()`:
+
+```javascript
+function response_timer(req, res, next) {
+    const start = Date.now();
+    const _end = res.end;
+
+    res.end = function () {
+        res.setHeader('X-Response-Time', `${Date.now() - start}ms`);
+        _end.apply(res, arguments);
+    };
+
+    next();
+}
+
+server.use(response_timer);
+```
+
+### Error Handling
+
+If a middleware throws or calls `next(err)`, the error handler sends a 500 response and logs the error. Remaining middleware and routing are skipped.
+
+```javascript
+server.use((req, res, next) => {
+    try {
+        validate_request(req);
+        next();
+    } catch (err) {
+        next(err); // → 500 Internal Server Error
+    }
+});
+```
+
+---
+
+## Access via Module Exports
+
+```javascript
+// Direct require
+const { compression } = require('jsgui3-server/middleware');
+
+// Via Server class
+const Server = require('jsgui3-server');
+const { compression } = Server.middleware;
+
+// Via jsgui module
+const jsgui = require('jsgui3-server');
+const { compression } = jsgui.middleware;
+```

package/docs/proposals/jsgui3-website-and-webpage-design-jsgui3-server-support.md

@@ -0,0 +1,257 @@
+# Proposal: jsgui3-server Support for New Website/Webpage Primitives
+
+> **Date**: 2026-02-16  
+> **Status**: Proposal for review (no implementation)  
+> **Companion docs**:  
+> `docs/proposals/jsgui3-website-and-webpage-design.md`  
+> `docs/proposals/jsgui3-website-and-webpage-design-review.md`
+
+## 1. Purpose
+
+Define the server-side changes required so `jsgui3-server` can fully support richer `jsgui3-website` / `jsgui3-webpage` primitives and the extra information they encapsulate, while preserving backward compatibility with existing `Server.serve(...)` usage.
+
+## 2. Core Integration Principles
+
+1. `jsgui3-server` should accept both classic inputs (Ctrl/functions/options) and new primitive objects.
+2. Integration should use capability/shape normalization, not fragile runtime class identity checks.
+3. Primitive metadata should flow into routing, bundling, HTTP headers, and admin introspection.
+4. Static and dynamic page rendering must be first-class, explicit modes.
+5. Existing behavior must remain intact by default.
+
+## 3. Current Gaps in jsgui3-server
+
+1. `publishers/http-website-publisher.js` depends on `instanceof Website` and `website.pages._arr` internals.
+2. `publishers/http-website-publisher.js` has incomplete multi-page publication code paths (`NYI`).
+3. `serve-factory.js` mainly normalizes function/options inputs, not richer website primitives.
+4. API publishing currently assumes `api` as a plain name→handler map; endpoint metadata is not preserved.
+5. Per-page metadata (render mode, page-level assets, cache hints, tags) is not represented in publication decisions.
+6. Admin/tooling introspection is not using a stable website manifest contract.
+
+## 4. Proposed Server Data Contract (Internal)
+
+Add an internal, normalized manifest that all publishers consume.
+
+### 4.1 `normalized_website_manifest`
+
+```js
+{
+  name,
+  base_path,
+  meta,
+  assets,
+  pages: [normalized_page_manifest],
+  api_endpoints: [normalized_api_manifest],
+  policies
+}
+```
+
+### 4.2 `normalized_page_manifest`
+
+```js
+{
+  id,
+  name,
+  path,
+  title,
+  content,
+  render_mode,      // 'static' | 'dynamic'
+  head,
+  meta,
+  assets,
+  scripts,
+  stylesheets,
+  cache_policy,
+  route_priority
+}
+```
+
+### 4.3 `normalized_api_manifest`
+
+```js
+{
+  name,
+  method,
+  path,
+  handler,
+  description,
+  auth,
+  rate_limit,
+  tags
+}
+```
+
+The key idea: all incoming forms normalize into this one server contract before routing/bundling begins.
+
+## 5. File-by-File Changes in jsgui3-server
+
+### 5.1 `serve-factory.js`
+
+Add a normalization phase before server startup.
+
+1. Add helpers such as `normalize_serve_input`, `normalize_website_input`, `normalize_webpage_input`, `normalize_api_input`.
+2. Accept additional input forms: `new Website(...)`, website-like plain objects with `pages`, `new Webpage(...)`, and webpage-like plain objects with `path` + `content`.
+3. Build `normalized_website_manifest` and attach it to `server_instance`.
+4. Preserve old `ctrl/page/pages/api` options by translating them into the same normalized manifest.
+5. Resolve route conflicts early with deterministic precedence rules.
+
+Impact: `serve-factory.js` becomes the single place where compatibility and input variability are handled.
+
+### 5.2 `publishers/http-webpageorsite-publisher.js`
+
+Refactor this shared base to consume a normalized manifest instead of raw ad hoc shape assumptions.
+
+1. Add `prepare_manifest_bundle(manifest, options)` entrypoint.
+2. Split shared logic into clear units such as `prepare_shared_client_bundle`, `prepare_page_static_assets`, and `build_static_route_response_items`.
+3. Keep SSR-only mode supported when no client bundle path exists.
+
+Impact: all page/site publishers share one bundling and route-item preparation pipeline.
+
+### 5.3 `publishers/http-webpage-publisher.js`
+
+Extend single-page publisher to honor encapsulated page metadata.
+
+1. Respect `render_mode`.
+2. `static`: pre-render HTML and serve static response items.
+3. `dynamic`: register request-time render handler for that route.
+4. Inject page-level assets/scripts/stylesheets in addition to shared site assets.
+5. Apply page-level response policies (`cache_policy`, optional headers).
+6. Include page manifest details in emitted `ready` payload for introspection.
+
+Impact: one-page apps gain parity with multi-page site semantics.
+
+### 5.4 `publishers/http-website-publisher.js`
+
+This file needs the largest rewrite.
+
+1. Remove hard dependency on `instanceof Website` and `website.pages._arr`.
+2. Accept normalized website manifest from `serve-factory.js`.
+3. For each page in `manifest.pages`, route by render mode (static page: pre-render + static route registration, dynamic page: register dynamic render handler).
+4. Register API endpoints from `manifest.api_endpoints` with method/path semantics.
+5. Add support for site-level `base_path` prefixing.
+6. Emit a structured publication summary object (`routes`, `assets`, `api`, `warnings`).
+
+Impact: `HTTP_Website_Publisher` becomes complete and deterministic for multi-page websites.
+
+### 5.5 `server.js`
+
+Add lightweight hooks so publication metadata can be surfaced consistently.
+
+1. Add structured route registration helper for static and dynamic routes.
+2. Add endpoint publication helper that accepts method/path metadata.
+3. Preserve existing `publish(name, fn)` API but map it to default endpoint metadata.
+4. Expose `server_instance.website_manifest` and `server_instance.publication_summary` for tooling/admin.
+
+Impact: no breaking API change, but much better visibility.
+
+### 5.6 `website/website.js` and `website/webpage.js`
+
+Keep these wrappers, but add comments and type-guards that describe the normalized contract expected by server internals.
+
+Impact: clearer boundaries between primitive packages and server adapters.
+
+### 5.7 New helper modules (recommended)
+
+Create a dedicated server-side normalization area.
+
+1. `website/normalize_website_manifest.js`
+2. `website/normalize_page_manifest.js`
+3. `website/normalize_api_manifest.js`
+4. `website/resolve_route_conflicts.js`
+5. `website/build_publication_summary.js`
+
+Impact: removes format-specific logic from publishers and centralizes compatibility policy.
+
+## 6. Primitive Field → Server Behavior Mapping
+
+| Primitive field | Server subsystem | Required behavior |
+|---|---|---|
+| `page.path` | Router | Normalize leading slash, register route, detect conflicts |
+| `page.render_mode` | Publisher/rendering | Choose static pre-render vs dynamic request-time render |
+| `page.content` | Renderer | Validate constructor/function and render consistently |
+| `page.assets/scripts/stylesheets` | Bundling/publish | Include per-page resources in route item list |
+| `page.cache_policy` | HTTP response | Set cache headers for HTML/assets |
+| `page.meta/head` | HTML generation | Emit tags into rendered document |
+| `website.base_path` | Routing | Prefix all page and API routes deterministically |
+| `website.assets` | Bundling | Register site-wide static assets |
+| `website.api` / endpoint metadata | API publisher | Publish endpoints with method/path/description/auth metadata |
+| `website.meta` | Admin/introspection | Expose in summary and diagnostics APIs |
+
+## 7. Backward Compatibility Strategy
+
+1. Maintain current `Server.serve(Ctrl)` behavior unchanged.
+2. Maintain current options object behavior (`ctrl`, `page`, `pages`, `api`).
+3. Introduce a compatibility normalization layer that converts old options to manifest format.
+4. Keep `server.publish(name, fn)` valid; internally convert to `{ method: 'GET', path: '/api/:name' }` (or current default route convention).
+5. Log non-breaking warnings when deprecated shapes are detected.
+
+## 8. Error Handling and Diagnostics
+
+Standardize errors from normalization and publication stages.
+
+1. `invalid_page_definition`
+2. `invalid_api_endpoint`
+3. `duplicate_route`
+4. `unsupported_render_mode`
+5. `asset_resolution_failure`
+
+Each error should include:
+
+1. `code`
+2. `message`
+3. `context` (`page_id`, `route`, `endpoint_name`, `source`)
+
+This is necessary for actionable admin UX and debugging.
+
+## 9. Testing Changes Required
+
+### 9.1 Unit tests
+
+1. Normalization from legacy inputs to manifest.
+2. Route conflict detection and precedence.
+3. API endpoint normalization (method/path defaults).
+4. Render mode routing decisions.
+
+### 9.2 Integration tests
+
+1. Single-page static primitive.
+2. Single-page dynamic primitive.
+3. Multi-page mixed static/dynamic website.
+4. Website with API endpoint metadata.
+5. Base path + asset publication behavior.
+6. Legacy `Server.serve` calls still passing unchanged.
+
+### 9.3 Regression tests
+
+1. No client JS path still starts server in SSR-only mode.
+2. Admin route still comes up with website manifest present.
+3. Existing compression and middleware pipeline remains intact.
+
+## 10. Suggested Delivery Phases
+
+1. **Phase 1: normalization + compatibility**  
+Implement manifest normalization in `serve-factory.js`, keep existing publishers mostly intact.
+
+2. **Phase 2: publisher modernization**  
+Refactor `http-webpageorsite-publisher.js`, `http-webpage-publisher.js`, `http-website-publisher.js` to consume normalized manifest and remove `_arr`/`instanceof` coupling.
+
+3. **Phase 3: metadata completeness**  
+Implement full field mapping (assets, cache policy, endpoint metadata, introspection summary).
+
+4. **Phase 4: admin/tooling surface**  
+Expose publication summaries and diagnostics for admin views and debugging.
+
+## 11. Recommended Initial Scope
+
+For a practical first milestone, support this subset end-to-end.
+
+1. `website.pages` with explicit `path` + `content`
+2. `page.render_mode` (`static`/`dynamic`)
+3. `website.api` endpoint metadata (`method`, `path`, `handler`)
+4. normalized publication summary
+5. full backward compatibility with current `Server.serve` signatures
+
+This gives immediate value without requiring every advanced field on day one.
+
+## 12. Outcome
+
+With these changes, `jsgui3-server` will be able to serve richer website/webpage primitives as first-class inputs, preserve all encapsulated structure/metadata, and remain backward compatible with existing users.

package/docs/proposals/jsgui3-website-and-webpage-design-review.md

@@ -0,0 +1,73 @@
+# Review: jsgui3-website-and-webpage-design
+
+> **Date**: 2026-02-16  
+> **Scope**: Review + design contributions only (no implementation)
+
+## Best ideas
+
+1. **Keeping `jsgui3-website` / `jsgui3-webpage` optional to `jsgui3-server` is the strongest architectural choice.**
+Reason: this preserves low-friction serving for simple cases while still allowing richer abstractions for apps that want them.
+
+2. **Making inspectability a first-class goal (`toJSON()`, metadata exposure) is excellent.**
+Reason: admin tooling, diagnostics, docs generation, and tests all benefit from a stable introspection surface.
+
+3. **Supporting multiple page input shapes (`Array` and object-map) is pragmatic.**
+Reason: object-map is concise for route-centric definitions, while arrays are better for ordered or generated page sets.
+
+4. **Calling out the `API.js` export bug explicitly is very good proposal hygiene.**
+Reason: the proposal is grounded in reality and identifies a hard runtime blocker.
+
+5. **The open-question section is high quality and correctly focused.**
+Reason: it addresses exactly the unknowns that matter for primitives (dynamic rendering, ordering, nesting, server vs page concerns).
+
+## Worst ideas
+
+1. **Leaning on `Collection` internals (`pages._arr`) is the weakest technical direction.**
+Reason: this bakes internal details into public behavior, increases fragility, and makes future refactors painful.
+
+2. **Using `Evented_Class` as a default `Webpage` model is over-engineered for current needs.**
+Reason: it adds boilerplate, coupling, and cognitive load before there is a concrete consumer that needs per-field change events.
+
+3. **Hard-coupling primitives to `jsgui3-html` creates unnecessary dependency weight.**
+Reason: website/webpage definitions are mostly domain objects; making them require the full UI/control stack limits reuse (including non-server tooling).
+
+4. **`instanceof` as a core integration strategy is brittle across duplicated installs/workspaces.**
+Reason: two copies of a package can fail `instanceof` even when shape-compatible; server boundaries should prefer capability checks and normalization.
+
+5. **Defaulting page `path` to `'/'` can silently mask configuration mistakes.**
+Reason: accidental missing routes become root collisions; this should be explicit in multi-page scenarios.
+
+## Contributions to the design discussion
+
+1. **Use a layered model: core primitives + server adapter layer.**
+Core primitives should be plain and lightweight (`Webpage`, `Website`, `Website_Api`) with minimal runtime dependencies. Server-specific translation can stay in `jsgui3-server`.
+
+2. **Define a strict minimal contract for primitives.**
+For `Webpage`: `path`, `title`, `content`, `meta`, `assets`, `render_mode` (`static` / `dynamic`).  
+For `Website`: page registry, endpoint registry, metadata, and deterministic serialization.
+
+3. **Prefer `Map` semantics for page identity, but expose stable methods, not internals.**
+Use `add_page`, `get_page`, `has_page`, `remove_page`, `list_pages`; do not expose storage internals like `_arr`.
+
+4. **Separate endpoint declaration from server publishing.**
+`Website_Api` should store endpoint definitions (`name`, `method`, `path`, `handler`, `description`, `auth`) while `jsgui3-server` decides how to publish them.
+
+5. **Normalize and validate early.**
+Route normalization (`/` prefix), duplicate policy, and handler/type checks should happen when constructing primitives, not at publish time.
+
+6. **Add lifecycle semantics instead of full reactivity.**
+A simple `finalize()` (or equivalent) model is likely enough: mutable during composition, read-mostly after publish. This gives safety without event-system complexity.
+
+7. **Make introspection a formal API contract.**
+Keep `toJSON()` deterministic and tool-friendly so admin UIs and docs generators can rely on it without peeking into implementation details.
+
+## Suggested direction
+
+A strong near-term path is effectively **Webpage B-lite + Website C-lite**:
+
+- Keep property helpers and introspection.
+- Use `Map`-style semantics for routes and endpoint registration.
+- Avoid `Evented_Class` and `Collection._arr` coupling.
+- Keep primitives lightweight and let `jsgui3-server` own serving/bundling mechanics.
+
+This keeps the primitives expressive enough for creative website/page design while staying robust for server integration.