jsgui3-server 0.0.151 → 0.0.152

package/docs/books/website-design/09-cross-agent-review.md

@@ -0,0 +1,190 @@
+# Chapter 9: Cross-Agent Review
+
+An OpenAI o3 agent was given the original proposals document and asked to review it and contribute to the design discussion. This chapter presents their findings and discusses the responses.
+
+---
+
+## The Review Process
+
+The agent was asked to do two things:
+
+1. **Review the proposals** — identify Best and Worst ideas (5 Best, 5 Worst)
+2. **Contribute to the design** — propose new ideas or refinements, then write a separate server-support document
+
+The results were two documents:
+- `jsgui3-website-and-webpage-design-review.md` — the review
+- `jsgui3-website-and-webpage-design-jsgui3-server-support.md` — a detailed server integration proposal
+
+---
+
+## Best Ideas (Their Assessment)
+
+### 1. "Keeping Website/Webpage optional to jsgui3-server"
+
+They identified this as the strongest architectural choice. The reasoning: it preserves low-friction serving for simple cases while allowing richer abstractions for apps that want them.
+
+**Response**: Agreed completely. This is a hard constraint, not just a design preference. The server's `Server.serve(MyCtrl)` one-liner is one of its best features and must not be complicated.
+
+### 2. "Making inspectability a first-class goal"
+
+They praised `toJSON()` and metadata exposure for admin tooling, diagnostics, docs generation, and tests.
+
+**Response**: Agreed. The admin UI already introspects the resource pool and router. Having structured `toJSON()` on Website/Webpage gives it a stable contract to display.
+
+### 3. "Supporting multiple page input shapes"
+
+Both array and object-map formats were called "pragmatic."
+
+**Response**: Agreed. This is about meeting users where they are, not forcing one format.
+
+### 4. "Calling out the API.js export bug"
+
+Good proposal hygiene — the design document is grounded in reality and identifies hard runtime blockers.
+
+**Response**: The bug (`MediaSourceHandle.exports` instead of `module.exports`) would crash on any `require('./API')` call. Including it prevents someone from trying to use the existing code and being confused.
+
+### 5. "The open-question section is high quality"
+
+They noted it addresses exactly the unknowns that matter for primitives.
+
+**Response**: Agreed. Design documents should be honest about what they don't know.
+
+---
+
+## Worst Ideas (Their Assessment)
+
+### 1. "Leaning on `Collection` internals (`pages._arr`)"
+
+Called the weakest technical direction. Bakes internal details into public behavior, increases fragility.
+
+**Response**: **Strongly agree.** The current `http-website-publisher.js` accesses `website.pages._arr` directly. This is an implementation detail of Collection that shouldn't appear in any public interface. Chapter 6 discusses alternatives; Map with method-based access is the cleaner direction.
+
+### 2. "Using `Evented_Class` as a default Webpage model"
+
+Called "over-engineered for current needs." Adds boilerplate, coupling, and cognitive load before there's a concrete consumer.
+
+**Response**: **Partially disagree.** The reviewer is right that per-field change events are premature. But extending `Evented_Class` as a base class is essentially free — it adds `.on()` and `.raise()` without requiring their use. In an ecosystem where nearly everything extends `Evented_Class`, *not* using it is the choice that needs justification. The counter-argument is detailed in Chapter 3.
+
+The key insight the reviewer missed: extending `Evented_Class` does not mean firing events on every property change. You can extend it and only raise events for meaningful lifecycle moments like `'page-added'` or `'ready'`.
+
+### 3. "Hard-coupling primitives to jsgui3-html"
+
+Called unnecessary dependency weight that limits reuse.
+
+**Response**: **Partially disagree.** Both modules will depend on `jsgui3-html` — this is a stated requirement, not a design choice. The page `content` *is* a jsgui3-html control constructor. The question isn't whether to depend on jsgui3-html but how deeply: just importing `Evented_Class` (light coupling) vs. using Collection, tof, obext, etc. (deep coupling).
+
+That said, the reviewer's principle is sound: keep the dependency surface as small as possible even within a required dependency.
+
+### 4. "`instanceof` as a core integration strategy"
+
+Called brittle across duplicated installs/workspaces.
+
+**Response**: **Agree.** This is a real issue in the jsgui3 ecosystem, where packages are often `npm link`ed during development. Two copies of `jsgui3-website` would break `instanceof`. Chapter 8 discusses alternatives including duck typing and Symbol-based type markers.
+
+### 5. "Defaulting page `path` to `'/'`"
+
+Can silently mask configuration mistakes in multi-page scenarios.
+
+**Response**: **Agree.** Two pages both defaulting to `/` would silently collide. The proposals in Chapter 4 do not default `path`, making it explicitly required.
+
+---
+
+## Their Design Contributions
+
+### 1. Layered model: core primitives + server adapter
+
+**Idea**: Keep primitives lightweight. Put all server-specific logic in jsgui3-server.
+
+**Response**: Correct architectural separation. The primitives define *what*, the server decides *how*. The Webpage shouldn't know about Express, HTTP, or bundling.
+
+### 2. Strict minimal contract for primitives
+
+**Idea**: Explicitly define what properties each primitive has and stick to it.
+
+**Response**: Agreed, with nuance. The contract should be clear, but extensibility should be preserved. Unknown spec properties shouldn't crash the constructor.
+
+### 3. Map semantics with stable methods
+
+**Idea**: Use `add_page`, `get_page`, `has_page`, `remove_page`, `list_pages` — never expose storage internals.
+
+**Response**: Agreed. This is the recommendation in Chapter 6 (Map option) and is implemented in Chapter 5, approaches B and C.
+
+### 4. Separate endpoint declaration from server publishing
+
+**Idea**: `Website_API` stores definitions; the server publishes them.
+
+**Response**: Agreed. This is the central principle in Chapter 7. The naming discussion (declare/define/register vs. publish) addresses exactly this.
+
+### 5. Normalize and validate early
+
+**Idea**: Route normalization, duplicate detection, and handler checks at construction time.
+
+**Response**: Agreed but with limits. Validation in the constructor catches errors early but can be annoying during development when building objects incrementally. The `finalize()` lifecycle model (discussed in Chapter 4, Approach C) offers a compromise: be permissive during construction, validate at finalization.
+
+### 6. Lifecycle semantics instead of full reactivity
+
+**Idea**: A simple `finalize()` model — mutable during composition, read-mostly after publish.
+
+**Response**: This is the review's most original contribution. It solves the event-system complexity problem without losing safety. The idea is explored in Chapter 4, Approach C.
+
+### 7. Formal `toJSON()` introspection contract
+
+**Idea**: Keep serialization deterministic and tool-friendly.
+
+**Response**: Agreed. All approaches in Chapters 4 and 5 include `toJSON()`.
+
+---
+
+## Their Server Support Document
+
+The OpenAI agent produced a separate, detailed server-integration proposal. Key contributions:
+
+### The Normalized Manifest
+
+The document's best idea. Instead of publishers each handling different input formats, a normalization step produces a standard `normalized_website_manifest` that all publishers consume. This includes:
+
+- `normalized_page_manifest` — per-page with render_mode, cache_policy, assets, etc.
+- `normalized_api_manifest` — per-endpoint with method, auth, rate_limit, tags
+- `normalized_website_manifest` — site-level with base_path, policies, combined pages and API
+
+**Response**: Architecturally excellent. This is the right pattern. The concern is scope — the proposed manifests include many fields (`rate_limit`, `route_priority`, `cache_policy`) that don't have consumers yet. Starting with a minimal manifest and growing it is more pragmatic.
+
+### File-by-file changes
+
+The document maps out changes to `serve-factory.js`, all three publishers, `server.js`, and the wrapper modules. It also proposes five new helper modules.
+
+**Response**: The analysis is thorough. The recommendation to start with fewer helper files (one `normalize.js` rather than five) is more practical for a first pass.
+
+### Delivery phases
+
+Four phases: normalization → publisher modernization → metadata completeness → admin surface.
+
+**Response**: Well-ordered. Each phase delivers independently useful value. The first phase (normalization + compatibility) is the right starting point.
+
+---
+
+## Points of Convergence
+
+Both agents (and the conversation so far) converge on:
+
+1. Website/Webpage must be optional to the server
+2. Map-based page storage with method-based API (no `_arr`)
+3. Server should use capability checks, not `instanceof`
+4. `toJSON()` for introspection
+5. Accept multiple input formats (array, object-map, instances)
+6. API endpoints should carry metadata beyond just name+handler
+
+## Points of Disagreement
+
+| Topic | This author | OpenAI reviewer |
+|-------|-------------|-----------------|
+| `Evented_Class` as base | Yes — ecosystem norm, free capability | No — over-engineered |
+| `jsgui3-html` dependency | Required (stated constraint) | Unnecessary weight |
+| When to validate | At construction or finalize | Always at construction |
+| Number of new helper files | Start with 1 | Start with 5 |
+
+---
+
+## Synthesis Forward
+
+Chapter 11 captures the current convergence from this chapter: keep primitives optional, use normalized server manifests, avoid storage internals leakage, and ship in phased increments with strict backward compatibility.

package/docs/books/website-design/10-open-questions.md

@@ -0,0 +1,196 @@
+# Chapter 10: Open Questions
+
+These are design decisions that remain unresolved. Each one could go in multiple directions, and the right choice depends on how these modules will actually be used.
+
+---
+
+## 1. Should Webpage manage client-side JS paths?
+
+Today, `serve-factory.js` resolves client-side JavaScript paths (`src_path_client_js`). The page just says "my content is this control" and the server figures out what to bundle.
+
+**Option A**: Webpage has a `client_js` property that points to the entry file:
+```js
+new Webpage({
+    content: MyCtrl,
+    client_js: './client/main.js'
+});
+```
+
+**Option B**: The server infers the bundle path from the content control, as it does today.
+
+**Option C**: Both — `client_js` is optional, and if provided, overrides the server's inference.
+
+**The tension**: If the Webpage knows its client-side JS path, it becomes more self-contained (good for static site generation, good for admin display). But if it doesn't, the server retains full control over bundling (simpler, less to coordinate).
+
+---
+
+## 2. Static vs. dynamic pages — who decides?
+
+A static page serves the same HTML to every request. A dynamic page generates HTML per-request (SSR). Today, all pages are effectively static — bundled once, served forever.
+
+**Option A**: Page declares its render mode:
+```js
+new Webpage({ path: '/profile', content: Profile, render_mode: 'dynamic' });
+```
+
+**Option B**: Server decides based on the content type (function = dynamic, instance = static).
+
+**Option C**: Server decides, but page can opt in:
+```js
+new Webpage({ path: '/profile', content: Profile, dynamic: true });
+```
+
+**The tension**: Render mode has huge server implications (static pre-render vs. per-request handler). Should the page author decide this, or should the server operator? A page author might not understand the performance implications.
+
+---
+
+## 3. Can these modules be used without jsgui3-server?
+
+Potential non-server use cases:
+- **Static site generator** — iterate pages, render each to an HTML file
+- **Sitemap generator** — iterate pages, output XML
+- **Documentation tool** — display the website structure
+- **Testing** — assert that a website has the right pages/endpoints
+
+If the answer is "yes, these should work standalone," it pushes toward keeping them dependency-light and not assuming a server context. If the answer is "they're primarily for jsgui3-server," more coupling is acceptable.
+
+**Current position**: Both packages will depend on `jsgui3-html` (for `Evented_Class`), but nothing from `jsgui3-server`. This makes standalone use possible.
+
+---
+
+## 4. Page ordering
+
+Does the order of pages matter?
+
+**Yes**: Navigation menus, sitemaps, breadcrumbs all have a natural order. If pages are in a Map, insertion order is preserved — but is that the intended display order?
+
+**No**: Pages are identified by path, not position. A navigation component would use explicit ordering, not page insertion order.
+
+**Maybe**: An optional `order` or `nav_index` property could let pages opt into ordered display:
+```js
+new Webpage({ path: '/about', title: 'About', nav_index: 2 });
+```
+
+This doesn't need to be decided now, but the data structure choice (Map preserves order; plain objects have less predictable iteration) has implications.
+
+---
+
+## 5. Nested routes
+
+Should a Website support hierarchical page grouping?
+
+```
+/docs
+/docs/getting-started
+/docs/api
+/docs/api/server
+/docs/api/server/serve
+```
+
+**Option A**: Flat list — all pages are siblings. The path hierarchy is implicit in the path strings. Navigation code parses paths to build trees.
+
+**Option B**: Page groups — pages can have child pages:
+```js
+site.add_page({
+    path: '/docs',
+    content: DocsIndex,
+    children: [
+        { path: '/docs/getting-started', content: GettingStarted },
+        { path: '/docs/api', content: ApiDocs }
+    ]
+});
+```
+
+**Option C**: Separate concept — a `Route_Group` or `Page_Group` class that wraps related pages.
+
+**The pragmatic answer**: Start flat. If path hierarchy matters, it can be derived from path strings. Adding a formal grouping mechanism before there's a concrete use case for it adds complexity without clear benefit.
+
+---
+
+## 6. Multiple websites on one server
+
+Can a single `jsgui3-server` serve multiple websites?
+
+Today, `server.js` uses `Website_Group` (a Collection) which implies yes. But the implementation is skeletal. If this is a real requirement, it affects how websites declare their `base_path` and how the server routes between them.
+
+```js
+Server.serve({
+    websites: [
+        new Website({ name: 'main', base_path: '/', pages: [...] }),
+        new Website({ name: 'admin', base_path: '/admin', pages: [...] })
+    ]
+});
+```
+
+This might be over-engineering. The more common pattern is one server, one website, multiple pages.
+
+---
+
+## 7. Hot reloading and live updates
+
+If a Website raises `'page-added'` events, could the server react to runtime page additions?
+
+```js
+const site = new Website({ pages: [...] });
+Server.serve(site);
+
+// Later:
+site.add_page({ path: '/new', content: NewPage });
+// → server automatically adds the route?
+```
+
+This would be powerful for development workflows (add a page, see it immediately without restarting). But it requires publisher support for incremental bundling and route registration. The `finalize()` lifecycle model from Chapter 4 explicitly prevents this by design.
+
+**The safe answer**: Don't support it initially. Build for "define site → serve it" as a one-shot workflow. If hot reloading becomes important, `Evented_Class` as the base makes it possible later — the events are there, the server just needs to listen.
+
+---
+
+## 8. Error boundaries
+
+When a page's content control throws during rendering, what happens?
+
+**Option A**: The server catches it and shows an error page (500).  
+**Option B**: The Website has an `error_page` property for a custom error control.  
+**Option C**: Each page has its own error boundary.
+
+This is entirely a server concern and probably shouldn't be in the Website/Webpage API at all. But it's worth noting because the proposals don't mention it.
+
+---
+
+## 9. The `server` property in Website
+
+The current `Website.js` stores `this.server` from the spec:
+```js
+this.api = new API({ server: this.server });
+```
+
+Should the new Website know about its server?
+
+**No**: The Website is an abstract definition. It shouldn't know *how* it's being served. The server consumes the Website, not the other way around.
+
+**Yes**: Some use cases (publishing, URL generation) require knowing the server's hostname, port, or base URL. The Website could hold this as metadata.
+
+**Compromise**: Don't store a server reference. If the Website needs deployment context, that's provided at serve-time by the server, not stored in the definition.
+
+---
+
+## 10. What to implement first
+
+Given everything discussed in this book, what's the minimum viable Website + Webpage that would be useful?
+
+**Suggested first pass**:
+
+1. `Webpage` extending `Evented_Class` with explicit properties (Chapter 4, Approach B)
+2. `Website` using Map-based pages and structured API (Chapter 5, Approach B)
+3. Fix the API.js export bug (Chapter 2)
+4. Ship as 0.0.9 of both packages
+5. Add Website detection to `serve-factory.js` (Chapter 8, minimal approach)
+6. Test with an example that creates a Website and serves it
+
+Everything else — normalization layers, publisher refactoring, admin integration, lifecycle finalization — can follow once the basics work.
+
+---
+
+## Note on Chapter 11
+
+Chapter 11 (`11-converged-recommendation.md`) provides an updated synthesis and phased plan that refines this first-pass list into a more explicit baseline contract and delivery sequence.

package/docs/books/website-design/11-converged-recommendation.md

@@ -0,0 +1,205 @@
+# Chapter 11: Converged Recommendation
+
+This chapter turns the debate from previous chapters into a concrete baseline that can be implemented incrementally.
+
+The goal is not to "win" every design argument. The goal is to define a stable, useful first version that keeps `jsgui3-server` backward-compatible while unlocking richer website/webpage primitives.
+
+---
+
+## 11.1 Decision Summary
+
+| Area | Recommended baseline | Why |
+|---|---|---|
+| Base class | `Evented_Class` with plain property assignment | Ecosystem-consistent, low overhead, no per-property event boilerplate |
+| Webpage contract | Explicit known fields + `toJSON()` + lightweight validation | Improves introspection and catches common mistakes early |
+| Website pages storage | Internal `Map` keyed by normalized path | O(1) lookup, duplicate detection, insertion-order preserved |
+| Website API layer | Structured endpoint registry (Map-backed) with metadata | Supports docs/admin/auth/method semantics without server coupling |
+| Server input handling | Normalize all input shapes into one internal manifest | Keeps publishers simple and removes shape-specific branching |
+| Type detection | Capability checks with optional explicit marker | Avoids brittle `instanceof` behavior in linked/workspace installs |
+| Lifecycle | Optional `finalize()` boundary, not full reactive mutation system | Safer publishing without heavy event complexity |
+
+---
+
+## 11.2 Minimal Stable Primitive Contract
+
+The first stable release should guarantee this contract.
+
+### `Webpage`
+
+Required:
+
+1. `path` (string, normalized to leading slash)
+2. `ctrl` (control constructor / renderer)
+
+Recommended:
+
+1. `name`
+2. `title`
+3. `content` (structured content/i18n data, optional)
+4. `meta`
+5. `scripts`
+6. `stylesheets`
+7. `render_mode` (`static` or `dynamic`) — see [Chapter 4 discussion](04-webpage.md)
+8. `toJSON()` for deterministic introspection
+
+Compatibility during migration:
+
+1. Accept legacy `content: Function` as an alias for `ctrl`.
+2. Normalize to `ctrl` internally so server/publisher code has one renderer field.
+
+### `Website`
+
+Required:
+
+1. page registry (`add_page`, `get_page`, `has_page`, `remove_page`, `pages`, `routes`)
+2. API registry (`add_endpoint`/`define_endpoint`, `get_endpoint`, `api_endpoints`)
+
+Recommended:
+
+1. `name`
+2. `meta`
+3. `assets`
+4. `base_path`
+5. `toJSON()` summary for admin/tooling
+
+---
+
+## 11.3 Server Integration Baseline
+
+`jsgui3-server` should treat Website/Webpage support as additive and normalize all inputs.
+
+### Input forms to support
+
+1. `Server.serve(MyCtrl)`
+2. `Server.serve({ pages: {...}, api: {...} })`
+3. `Server.serve(new Webpage(...))`
+4. `Server.serve(new Website(...))`
+5. shape-compatible plain objects for page/site definitions
+
+### Internal flow
+
+1. Detect and normalize input into `normalized_website_manifest`
+2. Validate normalized routes/endpoints
+3. Build static and dynamic page route registrations
+4. Publish API endpoints with method/path metadata
+5. Expose publication summary for introspection/admin
+
+This keeps the publisher pipeline focused on serving, not on input parsing.
+
+---
+
+## 11.4 Validation Policy
+
+Use two-stage validation.
+
+Construction-time lightweight validation:
+
+1. route is string-like
+2. handler/content types are sane
+3. obvious shape errors fail fast
+
+Publish-time strict validation:
+
+1. duplicate routes
+2. invalid HTTP methods
+3. unresolved assets
+4. unsupported render modes
+
+This avoids over-strict constructors while still giving strong safety before serving.
+
+---
+
+## 11.5 Backward Compatibility Rules
+
+These rules should hold through the migration.
+
+1. Existing `Server.serve(...)` signatures continue to work unchanged.
+2. Existing `api: { name: fn }` remains valid.
+3. Existing control-first one-liner remains first-class.
+4. New Website/Webpage primitives are optional convenience, not mandatory API layers.
+5. Deprecations, if any, should be warning-only for at least one minor cycle.
+
+Field migration rule:
+
+1. `Webpage.ctrl` is the canonical renderer field.
+2. Legacy `Webpage.content` used as a renderer remains accepted as input during migration.
+3. Server normalization maps both shapes into one internal manifest (`ctrl` canonical, structured content separate).
+
+---
+
+## 11.6 Non-Goals for the First Release
+
+To keep scope controlled, do not block first release on:
+
+1. runtime hot-add/remove page re-publication
+2. nested page-tree abstractions
+3. full OpenAPI generation
+4. advanced per-route auth/rate-limit engines
+5. multi-website orchestration features
+
+These can be layered in after stable primitive/server interoperability is proven.
+
+---
+
+## 11.7 Suggested Phased Delivery
+
+1. **Phase A: Primitive stabilization**  
+Finalize `Webpage` and `Website` contracts, including `toJSON()` and minimal validation.
+
+2. **Phase B: Server normalization layer**  
+Add `normalize_serve_input` + manifest conversion in `serve-factory.js`.
+
+3. **Phase C: Publisher alignment**  
+Refactor publishers to consume normalized manifests and remove `._arr`/`instanceof` coupling.
+
+4. **Phase D: Tooling/admin surfacing**  
+Expose website manifest and publication summary in admin APIs/UI.
+
+Each phase should ship with targeted tests and can be released independently.
+
+---
+
+## 11.8 Recommended First Implementation Target
+
+A pragmatic first milestone:
+
+1. Webpage with explicit fields + `toJSON()` + path normalization
+2. Website with Map-backed page registry + structured API endpoint registry
+3. Server normalization for Website/Webpage inputs
+4. Route conflict detection and clear error messages
+5. No regressions for legacy `Server.serve` usage
+
+If this milestone is stable, the architecture is validated and later enhancements become straightforward rather than speculative.
+
+---
+
+## 11.9 Acceptance Criteria for the Milestone
+
+1. A `Website` with at least two pages serves both routes correctly.
+2. A `Webpage` passed directly to `Server.serve(...)` serves at its declared path.
+3. Duplicate page paths fail with a deterministic, actionable error.
+4. API endpoint metadata (`method`, `path`) is preserved through normalization and publication.
+5. Legacy usage (`Server.serve(MyCtrl)`, `Server.serve({ pages, api })`) still passes existing tests.
+
+---
+
+## 11.10 Forward References
+
+The following chapters expand this recommendation into full implementation blueprints:
+
+- **[Chapter 12](12-content-model.md)**: Defines the content model — how Webpage holds text strings, structured data, and i18n translations
+- **[Chapter 13](13-webpage-module-spec.md)**: Complete module specification for `jsgui3-webpage` (constructor, every method, validation, test plan)
+- **[Chapter 14](14-website-module-spec.md)**: Complete module specification for `jsgui3-website` (page registry, API registry, finalize cascade)
+- **[Chapter 15](15-multi-repo-plan.md)**: Multi-repo implementation coordination (dependency graph, phased delivery, version pinning)
+
+---
+
+## 11.11 Version Track Alignment
+
+To avoid ambiguity across later chapters:
+
+1. **v0.1.x** is the minimal-first track from Chapter 16 (small surface, fast adoption).
+2. **v0.3.x** is the full-spec target from Chapters 13 and 14 (lifecycle, introspection, i18n helpers, richer API).
+3. Chapter 15 coordinates both tracks as one migration path, not competing plans.
+
+**Current status (Feb 2026)**: Codex implemented the full spec (see [Ch.17](17-implementation-report-codex.md)). Implementation exists in repo source but packages have not yet been published to npm (still v0.0.8 on registry). Server integration (`normalize_serve_input`, `website_manifest`) is live in `serve-factory.js`.