jsgui3-server 0.0.147 → 0.0.149

package/docs/core/jsgui3-server-core-book/01-startup-readiness-state-machine.md

@@ -0,0 +1,41 @@
+# 1. Startup Readiness State Machine
+
+## Core Problem
+
+A server that binds a socket before its route graph is installed can return transient `404` responses during cold start. This is a correctness defect, not a cosmetic delay.
+
+## Current Model
+
+`Server.serve(...)` now enforces a strict ordering:
+
+1. construct `Server` instance
+2. install API publishers and optional SSE publisher
+3. wait for server `ready` event (emitted after webpage/website publisher route installation)
+4. resolve additional page route preparations
+5. bind sockets with `server.start(...)`
+6. start configured resources
+7. resolve `serve(...)` promise
+
+## Removed Failure Pattern
+
+Previous behavior had a fallback timer that could start listening before `ready`. This allowed requests to arrive before `/` and bundle routes were present.
+
+That path is removed. Startup now rejects on readiness timeout instead of entering a half-ready listening state.
+
+## Startup Timeout
+
+`readyTimeoutMs` (default `120000`) bounds waiting for readiness. Timeout behavior:
+
+- if readiness is not reached in time, `serve(...)` rejects
+- no forced early listen occurs
+
+This makes startup failure explicit and testable.
+
+## Verification
+
+`tests/serve.test.js` includes a delayed fake webpage publisher and asserts that:
+
+- `serve(...)` does not resolve until delayed readiness arrives
+- post-resolution `/` is immediately routable
+
+This test prevents regression to pre-ready listening.

package/docs/core/jsgui3-server-core-book/02-resource-abstraction-and-lifecycle.md

@@ -0,0 +1,92 @@
+# 2. Resource Abstraction and Lifecycle
+
+## Uniform Contract
+
+`Process_Resource` and `Remote_Process_Resource` expose the same operational surface:
+
+- `start()`
+- `stop()`
+- `restart()`
+- `status` getter
+- `get_abstract()`
+
+The contract allows client/API code to remain management-backend agnostic.
+
+## Process_Resource Modes
+
+`Process_Resource` supports two process managers:
+
+- `direct` (default): local child process (`spawn`)
+- `pm2`: external manager integration (`pm2 start/stop/restart/jlist`)
+
+PM2 path resolution is optional and automatic:
+
+1. `processManager.pm2Path`
+2. `PM2_PATH` env
+3. local `node_modules/.bin/pm2(.cmd)`
+4. `pm2` from `PATH`
+
+Therefore explicit `pm2Path` is not required for default operation.
+
+## State Machine
+
+Nominal states:
+
+- `stopped`
+- `starting`
+- `running`
+- `stopping`
+- `restarting`
+- `crashed`
+
+`Remote_Process_Resource` additionally uses `unreachable` for transport failure.
+
+Each transition emits `state_change` with `{ from, to, timestamp }`.
+
+## Operational Signals
+
+`Process_Resource` emits:
+
+- `stdout` / `stderr` (line-wise)
+- `exit`
+- `health_check`
+- `unhealthy`
+- `crashed`
+
+`Remote_Process_Resource` emits:
+
+- `state_change`
+- `unreachable`
+- `recovered`
+
+## Failure Handling
+
+### Local process
+
+Unexpected non-zero exit with `autoRestart: true` enters exponential backoff restart.
+
+Backoff progression:
+
+- base `restartBackoffBaseMs` (default 1000)
+- doubles per attempt
+- capped at 60000ms
+
+Exceeding `maxRestarts` transitions to `crashed` and halts auto-restart.
+
+### Remote process
+
+Transport failures do not map to `crashed`; they map to `unreachable`. Recovery is explicit via `recovered` event when polling succeeds again.
+
+## Observability Payload
+
+`status` normalizes operational metrics:
+
+- state
+- pid (if meaningful)
+- uptime
+- restartCount
+- lastHealthCheck
+- memoryUsage
+- processManager metadata
+
+The normalized shape is essential for stable API and UI projections.

package/docs/core/jsgui3-server-core-book/03-resource-pool-and-event-topology.md

@@ -0,0 +1,47 @@
+# 3. Resource Pool and Event Topology
+
+## Role
+
+`Server_Resource_Pool` is the system-wide resource coordinator. It extends `Resource_Pool` and adds lifecycle orchestration plus event fan-in.
+
+## Lifecycle Surface
+
+Implemented operations:
+
+- `add(resource)` / `push(resource)`
+- `remove(name)` with stop-on-remove semantics
+- `start(callback)` start-all
+- `stop(callback)` stop-all
+- `get_resources_by_type(type)`
+- `summary` getter
+
+## Event Fan-In
+
+Pool-level forwarded events:
+
+- `resource_state_change`
+- `crashed`
+- `unhealthy`
+- `unreachable`
+- `recovered`
+
+Forwarding payload includes `resourceName` plus original event fields.
+
+This gives a single subscription point for system telemetry.
+
+## Summary as Fleet Snapshot
+
+`summary` aggregates:
+
+- totals by state (`running`, `stopped`, `crashed`, `unreachable`, ...)
+- `byType` entries with `get_abstract()` output per resource
+
+`summary` is designed for status APIs and dashboard views.
+
+## Correctness Invariants
+
+1. removing a resource must detach forwarded listeners
+2. stop-all must tolerate mixed callback/promise resource implementations
+3. no event should lose resource identity when forwarded
+
+`tests/server-resource-pool.test.js` and `tests/serve-resources.test.js` exercise these invariants.

package/docs/core/jsgui3-server-core-book/04-sse-publisher-semantics.md

@@ -0,0 +1,41 @@
+# 4. SSE Publisher Semantics
+
+## Component
+
+`HTTP_SSE_Publisher` is a general-purpose server-sent events publisher, separate from observable-specific transport.
+
+## Responsibilities
+
+- maintain connection registry (`client_id -> response stream`)
+- format protocol-correct SSE frames (`id`, `event`, `data`)
+- provide `broadcast(event, data)`
+- provide `send(client_id, event, data)`
+- issue keepalive comments (`:keepalive`) to prevent idle timeout
+- maintain bounded event history for replay
+- support `Last-Event-ID` replay on reconnect
+
+## Delivery Semantics
+
+- event IDs are monotonic and server-local
+- replay sends events with `id > last_event_id`
+- delivery is best-effort per connected client
+- failed writes trigger client disconnect cleanup
+
+## Operational Constraints
+
+- max client guard (`maxClients`)
+- explicit `stop()` for deterministic shutdown
+- keepalive timer lifecycle tied to active client count
+
+## Integration Contract
+
+When `Server.serve({ events: true })` is used, pool events are bridged into SSE broadcasts. Resource state transitions become browser-consumable without client polling loops.
+
+## Verification
+
+`tests/http-sse-publisher.test.js` validates:
+
+- multi-client broadcast
+- targeted delivery
+- keepalive emission
+- replay via `Last-Event-ID`

package/docs/core/jsgui3-server-core-book/05-serve-factory-resource-wiring.md

@@ -0,0 +1,46 @@
+# 5. Serve Factory Resource Wiring
+
+## Objective
+
+`Server.serve(...)` must be a single-entry composition API that can instantiate and wire:
+
+- pages / controls
+- API handlers
+- resources (process or remote)
+- SSE endpoint for resource events
+
+without requiring the caller to manually compose internals.
+
+## Resource Configuration Forms
+
+Accepted `resources` forms include:
+
+- direct resource instance
+- constructor function / class
+- `{ class | Ctor | constructor_fn, ...spec }`
+- `{ type: 'process' | 'remote', ...spec }`
+
+Type inference fallbacks:
+
+- `command` or `processManager` -> `Process_Resource`
+- `host` or `endpoints` -> `Remote_Process_Resource`
+
+## Event Bridge
+
+With `events: true`:
+
+- `HTTP_SSE_Publisher` mounted on `/events` (or configured route)
+- pool events broadcast onto SSE stream
+
+This creates a default telemetry channel with zero extra server code.
+
+## Startup Ordering
+
+1. configure APIs, events, and resource specs
+2. wait for ready
+3. start listening
+4. start configured resources
+
+## Shutdown Ordering
+
+`server.close()` triggers resource pool stop and SSE stop before closing HTTP servers. This prevents orphan resources and dangling SSE clients.

package/docs/core/jsgui3-server-core-book/06-e2e-testing-methodology.md

@@ -0,0 +1,48 @@
+# 6. E2E Testing Methodology
+
+## Principle
+
+A browser E2E test is valid only if it asserts both:
+
+- user-visible state (DOM/control behavior)
+- server truth (resource/process state, event emission)
+
+UI-only assertions are necessary but insufficient for fullstack correctness.
+
+## Harness
+
+`tests/helpers/puppeteer-e2e-harness.js` provides:
+
+- deterministic step runner (`run_interaction_story`)
+- browser probe capture (`console`, `pageerror`, request failures)
+- shared page/server boot helpers
+- stable interaction primitives (input event dispatch, drag)
+
+This converts long imperative test scripts into explicit state-transition stories.
+
+## Story Pattern
+
+Each step has:
+
+- `name`
+- `run(page)`
+- optional `assert(page)`
+
+Failure is annotated with `[story :: step]` to localize defects quickly.
+
+## Current High-Value Scenarios
+
+1. control interaction precision (`Date_Picker` + `Datetime_Picker`)
+2. client-driven resource lifecycle control (`start/stop/restart`)
+3. SSE propagation to browser and UI projection
+4. parity check between browser-observed state and server-side resource state
+
+Implemented in:
+
+- `tests/window-resource-integration.puppeteer.test.js`
+
+## Stability Guards
+
+- route readiness gate before browser navigation in resource integration tests
+- allowlist-based probe assertions for expected disconnect noise (`/events` abort on close)
+- strict no-unexpected-console-error policy

package/docs/core/jsgui3-server-core-book/07-defect-detection-and-hardening-loop.md

@@ -0,0 +1,47 @@
+# 7. Defect Detection and Hardening Loop
+
+## Goal
+
+Maximize bug discovery rate while minimizing false confidence.
+
+## Loop
+
+1. define invariant
+2. encode invariant as automated test
+3. reproduce failure deterministically
+4. implement smallest correctness-preserving fix
+5. re-run focused suite
+6. run regression suites touching adjacent surfaces
+7. document invariant and fix rationale
+
+## Practical Invariants for Current Surface
+
+### Startup / Routing
+
+- `serve()` must not resolve before root route availability
+- startup timeout must fail explicitly, not partially start
+
+### Resource Management
+
+- `stop()` semantics are idempotent for already-stopped resources
+- crash vs unreachable classification must remain distinct
+- process manager backend must not leak into consumer API shape
+
+### Eventing
+
+- every forwarded resource event must include `resourceName`
+- SSE replay must preserve event ordering by event ID
+
+### Browser Integration
+
+- client action routes must be observable in captured API call log
+- SSE state transitions must drive UI state within bounded time
+
+## Recommended Additional Test Work
+
+1. Add a chaos-style resource test with rapid start/stop/restart bursts and assert no deadlock.
+2. Add remote resource failure flapping test (unreachable/recovered oscillation) with threshold validation.
+3. Add SSE reconnection E2E test in browser with explicit `Last-Event-ID` continuity assertion.
+4. Add matrix tests across process manager modes (`direct`, `pm2` if available) under the same API contract assertions.
+
+These increase confidence in concurrency and degraded-network behavior, where latent bugs usually appear.

package/docs/publishers-guide.md

@@ -100,7 +100,7 @@ const publisher = new HTTP_Function_Publisher({
 - Support for common image formats (JPEG, PNG, SVG, etc.)
 - Efficient streaming for large files
 
-### HTTP_Observable_Publisher
+### HTTP_Observable_Publisher
 
 **Purpose:** Streams observable data to clients using Server-Sent Events (SSE).
 
@@ -190,9 +190,64 @@ data: {"tick":1,"timestamp":1234567890,"message":"Server tick #1"}
 data: {"tick":2,"timestamp":1234567891,"message":"Server tick #2"}
 ```
 
-**See Also:** [Observable SSE Demo](../examples/controls/15)%20window,%20observable%20SSE/) for a complete working example.
-
-## Publisher Architecture
+**See Also:** [Observable SSE Demo](../examples/controls/15)%20window,%20observable%20SSE/) for a complete working example.
+
+### HTTP_SSE_Publisher
+
+**Purpose:** General-purpose Server-Sent Events publisher for manual event fan-out (not tied to an observable source).
+
+**Key Features:**
+- SSE client connection registry with `client_count`
+- `broadcast(event_name, data)` for fan-out to all clients
+- `send(client_id, event_name, data)` for targeted delivery
+- Keepalive comments (`:keepalive`) to prevent idle proxy timeouts
+- Last-Event-ID replay support for reconnecting clients
+- Optional max-client limit
+- Emits:
+  - `client_connected`
+  - `client_disconnected`
+
+**Usage:**
+```javascript
+const HTTP_SSE_Publisher = require('jsgui3-server/publishers/http-sse-publisher');
+
+const sse_publisher = new HTTP_SSE_Publisher({
+    name: 'events',
+    keepaliveIntervalMs: 15000,
+    maxClients: 100
+});
+
+server.server_router.set_route('/events', sse_publisher, sse_publisher.handle_http);
+
+sse_publisher.broadcast('resource_state_change', {
+    resourceName: 'worker_1',
+    from: 'running',
+    to: 'crashed'
+});
+```
+
+**With `Server.serve()` Auto Wiring:**
+```javascript
+const server = await Server.serve({
+    events: true,
+    resources: {
+        worker_1: {
+            type: 'process',
+            command: process.execPath,
+            args: ['worker.js']
+        }
+    }
+});
+
+// SSE endpoint is available at /events by default.
+```
+
+### Choosing an SSE Publisher
+
+- Use `HTTP_Observable_Publisher` when your source is an observable stream.
+- Use `HTTP_SSE_Publisher` when you need explicit event push control (`broadcast`/`send`) from arbitrary server logic.
+
+## Publisher Architecture
 
 ### Base Publisher Class
 

package/docs/resources-guide.md

@@ -17,19 +17,26 @@ Resources are JSGUI3 Server's abstraction layer for accessing data, functionalit
 
 ## Resource Types
 
-### Server_Resource_Pool
-
-**Purpose:** Manages collections of resources with access control and lifecycle management.
-
-**Key Features:**
-- Resource registration and discovery
-- Access control through permission systems
-- Lifecycle management (start/stop/cleanup)
-- Resource dependency resolution
-
-**Usage:**
-```javascript
-const pool = new Server_Resource_Pool({
+### Server_Resource_Pool
+
+**Purpose:** Manages collections of resources with access control and lifecycle management.
+
+**Key Features:**
+- Resource registration and discovery
+- Access control through permission systems
+- Lifecycle management (`start()`, `stop()`, `remove(name)`)
+- Type filtering with `get_resources_by_type(type)`
+- Aggregated `summary` view of resource states
+- Resource event forwarding:
+  - `resource_state_change`
+  - `crashed`
+  - `unhealthy`
+  - `unreachable`
+  - `recovered`
+
+**Usage:**
+```javascript
+const pool = new Server_Resource_Pool({
     access: {
         full: ['admin'],
         read: ['user']
@@ -37,14 +44,117 @@ const pool = new Server_Resource_Pool({
 });
 
 pool.add(myResource);
-pool.start((err) => {
-    // Pool is ready
-});
-```
-
-### Website_Resource
-
-**Purpose:** Wraps website objects for integration with the server's resource system.
+pool.start((err) => {
+    // Pool is ready
+});
+
+const summary = pool.summary;
+const process_resources = pool.get_resources_by_type('Process_Resource');
+await pool.remove('legacy_worker');
+```
+
+### Process_Resource
+
+**Purpose:** Represents a local long-running process as a resource.
+
+**Key Features:**
+- State machine lifecycle:
+  - `stopped`
+  - `starting`
+  - `running`
+  - `stopping`
+  - `restarting`
+  - `crashed`
+- Emits:
+  - `state_change`
+  - `stdout`
+  - `stderr`
+  - `exit`
+  - `health_check`
+  - `unhealthy`
+  - `crashed`
+- Auto-restart with exponential backoff (when `autoRestart: true`)
+- Optional health checks (`http`, `tcp`, `custom`)
+- Consistent status shape for direct and PM2-backed processes
+
+**Process Manager Modes:**
+- `direct` (default): Uses `child_process.spawn()`
+- `pm2`: Uses PM2 CLI commands
+
+**PM2 Path Resolution:**
+- If `processManager.pm2Path` is set, it is used
+- Else `PM2_PATH` env var is used (if set)
+- Else local `node_modules/.bin/pm2` is used if present
+- Else `pm2` is resolved from system `PATH`
+
+**Usage:**
+```javascript
+const Process_Resource = require('jsgui3-server/resources/process-resource');
+
+// Direct mode (default)
+const worker_direct = new Process_Resource({
+    name: 'worker_direct',
+    command: process.execPath,
+    args: ['worker.js'],
+    autoRestart: true
+});
+
+// PM2 mode (pm2Path optional)
+const worker_pm2 = new Process_Resource({
+    name: 'worker_pm2',
+    processManager: { type: 'pm2' },
+    command: process.execPath,
+    args: ['worker.js']
+});
+
+await worker_direct.start();
+console.log(worker_direct.status);
+await worker_direct.stop();
+```
+
+### Remote_Process_Resource
+
+**Purpose:** Represents a remote process controlled via HTTP API while keeping a resource-compatible interface.
+
+**Key Features:**
+- Same high-level lifecycle API as `Process_Resource`:
+  - `start()`
+  - `stop()`
+  - `restart()`
+  - `status`
+  - `get_abstract()`
+- Periodic polling of remote status endpoint
+- Emits:
+  - `state_change`
+  - `unreachable`
+  - `recovered`
+- Maintains bounded history snapshots for diagnostics
+
+**Usage:**
+```javascript
+const Remote_Process_Resource = require('jsgui3-server/resources/remote-process-resource');
+
+const remote_worker = new Remote_Process_Resource({
+    name: 'remote_worker',
+    host: '192.168.1.100',
+    port: 3400,
+    pollIntervalMs: 30000,
+    httpTimeoutMs: 6000,
+    endpoints: {
+        status: '/',
+        start: '/api/start',
+        stop: '/api/stop',
+        health: '/api/health'
+    }
+});
+
+await remote_worker.start();
+console.log(remote_worker.status);
+```
+
+### Website_Resource
+
+**Purpose:** Wraps website objects for integration with the server's resource system.
 
 **Key Features:**
 - Website object encapsulation
@@ -110,16 +220,34 @@ class Custom_Resource extends Resource {
 }
 ```
 
-### Lifecycle Management
-
-Resources follow a consistent lifecycle:
+### Lifecycle Management
+
+Resources follow a consistent lifecycle:
 
 1. **Construction**: Resource is instantiated with configuration
 2. **Registration**: Added to resource pool
 3. **Start**: Resource becomes active and available
 4. **Usage**: Resource handles requests and operations
 5. **Stop**: Resource is deactivated
-6. **Cleanup**: Resources are released
+6. **Cleanup**: Resources are released
+
+For process resources, consumers can rely on the same lifecycle API regardless of execution mode (`direct`, `pm2`, or `remote`).
+
+### Unified Process Status
+
+Both `Process_Resource` and `Remote_Process_Resource` expose a compatible `status` shape:
+
+```javascript
+{
+    state: 'running',
+    pid: 12345,
+    uptime: 61520,
+    restartCount: 1,
+    lastHealthCheck: null,
+    memoryUsage: { rssBytes: 104857600 },
+    processManager: { type: 'direct' } // or 'pm2' / 'remote'
+}
+```
 
 ### Configuration Patterns
 
@@ -171,18 +299,39 @@ class Database_Resource extends Resource {
 }
 ```
 
-### Integration with Server
-
-```javascript
-// Add to resource pool
-server.resource_pool.add(databaseResource);
+### Integration with Server
+
+```javascript
+// Add to resource pool
+server.resource_pool.add(databaseResource);
 
 // Access from other components
 const db = server.resource_pool.get_resource('Database_Resource');
-db.query('SELECT * FROM users', [], (err, results) => {
-    // Handle results
-});
-```
+db.query('SELECT * FROM users', [], (err, results) => {
+    // Handle results
+});
+```
+
+With `Server.serve()` you can register resources declaratively:
+
+```javascript
+const server = await Server.serve({
+    resources: {
+        worker_direct: {
+            type: 'process',
+            command: process.execPath,
+            args: ['worker.js']
+        },
+        remote_worker: {
+            type: 'remote',
+            host: '127.0.0.1',
+            port: 3400
+        },
+        local_cache: new In_Process_Cache_Resource({ name: 'local_cache' })
+    },
+    events: true
+});
+```
 
 ## Resource Communication Patterns
 
@@ -612,4 +761,4 @@ class Logged_Resource extends Resource {
 
 ---
 
-This guide provides the foundation for understanding and extending the resource system. For specific resource implementations, refer to their individual source files in the `resources/` directory.
+This guide provides the foundation for understanding and extending the resource system. For specific resource implementations, refer to their individual source files in the `resources/` directory.