caspian-utils 0.0.29 → 0.0.31

package/dist/docs/auth.md

@@ -165,8 +165,9 @@ return AuthSettings(
 
 The installed auth code reads several values from `.env` when explicit values are not passed.
 
-- `AUTH_SECRET` backs `AuthSettings.secret_key`.
-- `AUTH_COOKIE_NAME` backs `AuthSettings.cookie_name`.
+- `AUTH_SECRET` backs `AuthSettings.secret_key`.
+- App-owned startup helpers may also validate `AUTH_SECRET` and refuse production startup when the value is missing or still on a default placeholder.
+- `AUTH_COOKIE_NAME` backs `AuthSettings.cookie_name`.
 - `SESSION_LIFETIME_HOURS` controls `SessionMiddleware.max_age` in the current `main.py` bootstrap.
 - `APP_ENV=production` enables secure session cookies and the `Secure` flag on the current CSRF cookie.
 - `CASPIAN_BROWSER_SYNC_PORT` can override the development cookie scope suffix used by the current `main.py` bootstrap.
@@ -224,28 +225,29 @@ This does two things:
 - applies the centralized settings once at startup
 - registers OAuth providers once so `AuthMiddleware` can delegate signin and callback paths through `auth.auth_providers(...)`
 
-The current `main.py` also wires the session middleware directly:
-
-```python
-from starlette.middleware.sessions import SessionMiddleware
-
-
-SESSION_LIFETIME_HOURS = int(os.getenv("SESSION_LIFETIME_HOURS", 7))
-IS_PRODUCTION = os.getenv("APP_ENV") == "production"
-
-
-app.add_middleware(
-    SessionMiddleware,
-    secret_key=os.getenv("AUTH_SECRET", "change-me"),
-    session_cookie=os.getenv("AUTH_COOKIE_NAME", "session"),
-    max_age=SESSION_LIFETIME_HOURS * 3600,
-    same_site="lax",
+The current `main.py` also wires the session middleware directly, and some apps delegate the secret lookup to an app-owned helper:
+
+```python
+from starlette.middleware.sessions import SessionMiddleware
+from src.lib.security.runtime_security import get_session_secret
+
+
+SESSION_LIFETIME_HOURS = int(os.getenv("SESSION_LIFETIME_HOURS", 7))
+IS_PRODUCTION = os.getenv("APP_ENV") == "production"
+
+
+app.add_middleware(
+    SessionMiddleware,
+    secret_key=get_session_secret(is_production=IS_PRODUCTION),
+    session_cookie=os.getenv("AUTH_COOKIE_NAME", "session"),
+    max_age=SESSION_LIFETIME_HOURS * 3600,
+    same_site="lax",
     https_only=IS_PRODUCTION,
     path="/",
 )
 ```
 
-Keep this wiring in `main.py`. Keep only the policy values in `src/lib/auth/auth_config.py`.
+Keep this wiring in `main.py`. Keep only the policy values in `src/lib/auth/auth_config.py`. If the app uses an app-owned security helper for session-secret enforcement, safe public-file serving, or browser security headers, keep that helper in `src/lib/**` and keep the middleware attachment itself in `main.py`.
 
 ## Session Lifetime Rules
 
@@ -258,37 +260,40 @@ The effective authenticated session lasts only while both are valid.
 
 - If the session cookie expires first, the session disappears even if the auth payload expiration was longer.
 - If the auth payload expires first, `auth.is_authenticated()` clears it even if the session cookie still exists.
-- In the pasted `main.py`, `token_auto_refresh=True` still does nothing by itself because the request flow does not call `auth.refresh_session()`.
+- In apps where the request flow does not call `auth.refresh_session()`, `token_auto_refresh=True` still does nothing by itself.
 
 Keep `SESSION_LIFETIME_HOURS` and `default_token_validity` aligned unless you intentionally want one boundary to end earlier than the other.
 
 ## Middleware Order
 
-The current `main.py` adds middleware in this source order:
-
-```python
-app.add_middleware(RPCMiddleware)
-app.add_middleware(AuthMiddleware)
-app.add_middleware(CSRFMiddleware)
-app.add_middleware(SessionMiddleware, ...)
-```
-
-Because Starlette runs the last-added middleware first, the effective request order is:
-
-1. `SessionMiddleware`
-2. `CSRFMiddleware`
-3. `AuthMiddleware`
-4. `RPCMiddleware`
-5. route handler or RPC endpoint
-
-Current behavior by layer:
-
-- `SessionMiddleware` provides `request.session` for the rest of the stack.
-- `CSRFMiddleware` ensures `request.session["csrf_token"]` exists and emits a scoped CSRF cookie based on `pp_csrf`, for example `pp_csrf_5091` in development or plain `pp_csrf` when no dev scope is active.
-- `AuthMiddleware` sets request context with `Auth.set_request(request)`, initializes `StateManager`, runs provider callbacks, skips configured static asset paths, and enforces public, auth, private, and role-based route redirects.
-- `RPCMiddleware` handles `POST` requests with `X-PP-RPC: true` and forwards them to Caspian's RPC handler after auth and session setup are already available.
-
-Keep `SessionMiddleware` outermost. If CSRF, auth, or RPC handling runs before it, `request.session` will not be available.
+The current `main.py` adds middleware in this source order:
+
+```python
+app.add_middleware(RPCMiddleware)
+app.add_middleware(AuthMiddleware)
+app.add_middleware(CSRFMiddleware)
+app.add_middleware(SessionMiddleware, ...)
+app.add_middleware(SecurityHeadersMiddleware)
+```
+
+Because Starlette runs the last-added middleware first, the effective request order is:
+
+1. `SecurityHeadersMiddleware`
+2. `SessionMiddleware`
+3. `CSRFMiddleware`
+4. `AuthMiddleware`
+5. `RPCMiddleware`
+6. route handler or RPC endpoint
+
+Current behavior by layer:
+
+- `SecurityHeadersMiddleware` can attach browser security headers such as CSP, `X-Content-Type-Options`, framing policy, and HSTS while preserving any headers already set by the response.
+- `SessionMiddleware` provides `request.session` for the rest of the stack.
+- `CSRFMiddleware` ensures `request.session["csrf_token"]` exists and emits a scoped CSRF cookie based on `pp_csrf`, for example `pp_csrf_5091` in development or plain `pp_csrf` when no dev scope is active.
+- `AuthMiddleware` sets request context with `Auth.set_request(request)`, initializes `StateManager`, runs provider callbacks, skips configured static asset paths, and enforces public, auth, private, and role-based route redirects.
+- `RPCMiddleware` handles `POST` requests with `X-PP-RPC: true` and forwards them to Caspian's RPC handler after auth and session setup are already available.
+
+Keep `SessionMiddleware` immediately inside any outer response-header wrapper such as `SecurityHeadersMiddleware`. If CSRF, auth, or RPC handling runs before the session layer, `request.session` will not be available.
 
 ## Current `AuthMiddleware` Flow
 
@@ -760,7 +765,7 @@ Prefer `AuthSettings`, `configure_auth(...)`, and `auth.settings` in new code.
 - Expired or malformed payloads are removed during `auth.is_authenticated()` checks.
 - `auth.get_payload()` returns `None` for missing payloads, a dict for dict payloads, and `{"value": ...}` for non-dict payloads.
 - OAuth callback helpers return `None` on failure, so calling routes should be prepared for a normal page render or explicit error state when provider login fails.
-- The current app bootstrap adds `SessionMiddleware`, `CSRFMiddleware`, `AuthMiddleware`, and `RPCMiddleware`; auth-aware code depends on that ordering so sessions and CSRF state exist before route checks or RPC handling.
+- The current app bootstrap may wrap auth and session handling in an outer `SecurityHeadersMiddleware`; auth-aware code still depends on `SessionMiddleware`, `CSRFMiddleware`, `AuthMiddleware`, and `RPCMiddleware` staying in the correct relative order so sessions and CSRF state exist before route checks or RPC handling.
 - Password reset, email verification, and other account-recovery workflows are application responsibilities layered on top of `casp.auth`, not built-in auth runtime features.
 - Route lists and RBAC maps are exact-path checks, not wildcard or prefix rules.
 
@@ -803,7 +808,7 @@ If an AI agent is deciding how to handle auth in Caspian, apply these rules firs
 - Do not treat `token_auto_refresh` as a route-privacy switch. In the current app it only affects sliding-session refresh if `auth.refresh_session()` is called.
 - Use `@require_auth`, `@require_role`, and `@guest_only` for page-level access rules.
 - Use `@rpc(require_auth=True, allowed_roles=[...])` for protected browser-triggered actions.
-- Keep `SessionMiddleware` outermost so auth, CSRF, and RPC handlers can read `request.session`.
+- Keep `SessionMiddleware` immediately inside any outer response-header wrapper so auth, CSRF, and RPC handlers can read `request.session`.
 - Use [state.md](./state.md) only for transient auth-adjacent request state, not as the auth session store.
 - Align `SESSION_LIFETIME_HOURS` with `default_token_validity` unless you intentionally want different cookie and auth-payload expiry windows.
 - Prefer exact route strings in `public_routes`, `auth_routes`, `private_routes`, and `role_based_routes`.
@@ -811,6 +816,6 @@ If an AI agent is deciding how to handle auth in Caspian, apply these rules firs
 - Keep auth secrets and OAuth provider credentials in `.env`.
 - Set `AUTH_COOKIE_NAME` explicitly so the session middleware cookie name and auth settings stay aligned.
 - Use `.venv/Lib/site-packages/casp/auth.py` only when the task is about framework auth internals or debugging installed behavior.
-- Use `main.py` when the task is about auth bootstrap, session middleware, or middleware execution order.
+- Use `main.py` plus any app-owned runtime security helper when the task is about auth bootstrap, session middleware, browser security headers, or middleware execution order.
 - Pair auth work with `fetch-data.md` for RPC forms and `validation.md` for credential validation.
 - Check `routing.md` and `project-structure.md` before creating new auth routes or shared auth helpers.

package/dist/docs/core-runtime-map.md

@@ -29,16 +29,19 @@ Use it when you already know a behavior is controlled by `main.py` or `.venv/Lib
 
 | Concern | Core file | Read first | Why it matters |
 | --- | --- | --- | --- |
-| App bootstrap and request flow | [main.py](../../../../main.py) | [project-structure.md](./project-structure.md), [routing.md](./routing.md), [auth.md](./auth.md) | FastAPI app creation, static asset routes, middleware order, route registration, cache check and save, exception handlers, and final HTML transforms all live here. |
+| App bootstrap and request flow | [main.py](../../../../main.py) plus any imported app-owned runtime helpers | [project-structure.md](./project-structure.md), [routing.md](./routing.md), [auth.md](./auth.md) | FastAPI app creation, middleware wiring, route registration, cache check and save, exception handlers, and final HTML transforms live here. App-owned helper modules imported by `main.py` may own safe static-file serving, browser security headers, CSP defaults, or production session-secret enforcement. |
 | Browser runtime, SPA navigation, and scroll restoration | [public/js/pp-reactive-v2.js](../../../../public/js/pp-reactive-v2.js) | [pulsepoint.md](./pulsepoint.md), [fetch-data.md](./fetch-data.md) | The shipped `pp` runtime, same-origin SPA interception, per-history-entry scroll state, `pp-reset-scroll` behavior, and browser RPC helpers live here. |
 
 Important current `main.py` behaviors AI should keep in mind:
 
-- In development, `_scoped_cookie_name(...)` appends the active BrowserSync or dev port to both the session cookie name and the CSRF cookie name. The scope is resolved from `CASPIAN_BROWSER_SYNC_PORT`, then `settings/bs-config.json`, then `PORT`.
-- `register_single_route(...)` passes path params to `page()` as one positional dict, injects matching query params by name, and injects `request` by keyword when declared.
-- The render pipeline is `transform_components(...)`, then `render_with_nested_layouts(...)`, then `transform_scripts(...)`.
-- Route-level generators returned from `page()` are wrapped in `SSE(...)` before the response is sent.
-- Middleware is added in source order as `RPCMiddleware`, `AuthMiddleware`, `CSRFMiddleware`, `SessionMiddleware`, so the effective request order is reversed at runtime.
+- In development, `_scoped_cookie_name(...)` appends the active BrowserSync or dev port to both the session cookie name and the CSRF cookie name. The scope is resolved from `CASPIAN_BROWSER_SYNC_PORT`, then `settings/bs-config.json`, then `PORT`.
+- `register_single_route(...)` passes path params to `page()` as one positional dict, injects matching query params by name, and injects `request` by keyword when declared.
+- The render pipeline is `transform_components(...)`, then `render_with_nested_layouts(...)`, then `transform_scripts(...)`.
+- Route-level generators returned from `page()` are wrapped in `SSE(...)` before the response is sent.
+- Safe public-file helpers may be factored into an app-owned runtime module and should reject `..` traversal before serving from `public/**`.
+- Session middleware secrets may be resolved through an app-owned helper so production can fail fast when `AUTH_SECRET` is missing or still on a default placeholder.
+- Browser security headers and CSP defaults may be built in an app-owned helper and attached through `SecurityHeadersMiddleware`.
+- Middleware is added in source order as `RPCMiddleware`, `AuthMiddleware`, `CSRFMiddleware`, `SessionMiddleware`, `SecurityHeadersMiddleware`, so the effective request order is reversed at runtime.
 - `public/js/pp-reactive-v2.js` saves scroll positions per history entry, resets window scroll on push navigation, and uses `pp-reset-scroll="true"` to opt specific containers or the whole body into reset behavior.
 
 ## Caspian Core Feature Map
@@ -53,6 +56,7 @@ Use this table when the task names a framework feature but the owning file is no
 | Metadata | route or layout `metadata`, runtime metadata helpers | `casp.layout`, `main.py` | [metadata.md](./metadata.md), [routing.md](./routing.md) |
 | Component imports and `x-*` tags | `src/components/**`, `src/app/**/*.html` | `casp.components_compiler`, `casp.component_decorator` | [components.md](./components.md), [routing.md](./routing.md) |
 | Auth and route protection | `src/lib/auth/auth_config.py`, `main.py`, route decorators | `casp.auth`, `main.py` middleware | [auth.md](./auth.md) |
+| Browser security headers, CSP defaults, and safe public-file serving | app-owned helpers imported by `main.py` | app-owned runtime modules | [auth.md](./auth.md), [project-structure.md](./project-structure.md) |
 | RPC and server actions | route or component Python modules with `@rpc()` | `casp.rpc`, `main.py` middleware | [fetch-data.md](./fetch-data.md), [file-uploads.md](./file-uploads.md) |
 | Streaming | route `page()` generators, RPC generators | `casp.streaming`, `casp.rpc`, `main.py` | [fetch-data.md](./fetch-data.md) |
 | Server state | request handlers and RPC actions | `casp.state_manager`, `main.py` middleware | [state.md](./state.md) |
@@ -106,7 +110,7 @@ Use these behavior checkpoints when AI needs the fastest verification path for a
 
 | Runtime area | Verify these behaviors |
 | --- | --- |
-| `main.py` routing and request flow | route registration, path and query injection, static asset handling, and exception rendering |
+| `main.py` routing and request flow | route registration, path and query injection, static asset handling, session-middleware wiring, browser security headers, and exception rendering |
 | `public/js/pp-reactive-v2.js` browser runtime | SPA interception, history-vs-push scroll behavior, `pp-reset-scroll` semantics, and browser-side `pp.rpc()` behavior |
 | `casp.auth` | auth settings, signin and signout flow, provider wiring, and page protection behavior |
 | `casp.rpc` and streamed RPC responses | middleware interception, CSRF and session expectations, registry behavior, and helper-level RPC contracts |

package/dist/docs/file-conventions.md

@@ -0,0 +1,259 @@
+---
+title: File Conventions
+description: Use this page when deciding what belongs in `index.html`, `index.py`, `layout.html`, `layout.py`, `loading.html`, `not-found.html`, or `error.html` in a Caspian app.
+related:
+  title: Related docs
+  description: Use the routing guide for URL and subtree behavior, the structure guide for placement, the metadata guide for SEO fields, the cache guide for route-level caching, and the runtime maps when the task crosses into framework internals.
+  links:
+    - /docs/routing
+    - /docs/project-structure
+    - /docs/metadata
+    - /docs/cache
+    - /docs/pulsepoint
+    - /docs/core-runtime-map
+    - /docs/pulsepoint-runtime-map
+    - /docs/index
+---
+
+This page is the quick decision guide for the special file conventions under `src/app`.
+
+Use it when a task names `index.html`, `index.py`, `layout.html`, `layout.py`, `loading.html`, `not-found.html`, or `error.html`, or when the question is where a page shell, route logic, loading UI, 404 page, or 500 page should live.
+
+Treat `caspian.config.json` and the actual project tree as the source of truth for which features exist in the current workspace. For runtime ownership, verify these rules against:
+
+- `main.py` for global `not-found.html` and `error.html` handling
+- `.venv/Lib/site-packages/casp/layout.py` for `render_page(...)`, `render_layout(...)`, and nested layout behavior
+- `.venv/Lib/site-packages/casp/loading.py` plus `public/js/pp-reactive-v2.js` for `loading.html` collection and SPA loading behavior
+- `.venv/Lib/site-packages/casp/caspian_config.py` for how `index.*`, `layout.html`, and `loading.html` are indexed under `src/app`
+
+## Quick Map
+
+| File | Purpose | Add it when | Verify against |
+| --- | --- | --- | --- |
+| `index.html` | Authored visible page template for a route | The route renders UI | `src/app/**`, `routing.md`, `pulsepoint.md` |
+| `index.py` | Backend companion for route logic and metadata | The route needs `page()`, metadata, auth checks, redirects, caching, or route-owned `@rpc()` actions | `main.py`, `.venv/Lib/site-packages/casp/layout.py` |
+| `layout.html` | Shared shell for a route subtree | Multiple child routes share wrapper markup | `.venv/Lib/site-packages/casp/layout.py`, `routing.md` |
+| `layout.py` | Shared synchronous props and metadata defaults for a subtree | The shared shell needs Python-provided values or metadata | `.venv/Lib/site-packages/casp/layout.py`, `metadata.md` |
+| `loading.html` | Route-scope loading UI used during SPA navigation | A section or page needs an immediate loading state before the next route finishes rendering | `.venv/Lib/site-packages/casp/loading.py`, `public/js/pp-reactive-v2.js` |
+| `not-found.html` | Global 404 page | The app needs a branded fallback for unmatched URLs | `main.py` |
+| `error.html` | Global 500 page | The app needs a safe fallback for unhandled exceptions | `main.py` |
+
+## Authored HTML Rule
+
+For authored route, layout, loading, not-found, and error HTML files, keep exactly one top-level parent HTML element or one imported `x-*` component root.
+
+- Keep any `<!-- @import ... -->` directives above that root.
+- Keep any owned plain `<script>` inside that root, not after it.
+- Do not handwrite `pp-component` or `type="text/pp"`.
+
+This is a runtime requirement, not a style preference.
+
+## `index.html`
+
+`index.html` is the authored page template for a route.
+
+Use it for:
+
+- visible page markup
+- HTML-first `x-*` component usage
+- PulsePoint state, refs, effects, and directives that belong to that route
+- route-local plain `<script>` blocks that stay inside the same root
+
+For any route that renders UI, keep the visible markup here even when the route also has an `index.py` companion.
+
+Example:
+
+```html
+<!-- @import Button from "../../components/Button.py" -->
+
+<section class="space-y-4 p-6">
+  <h1 class="text-2xl font-semibold">Dashboard</h1>
+
+  <x-button onclick="setFilter('open')">
+    Show Open
+  </x-button>
+
+  <script>
+    const [filter, setFilter] = pp.state("all");
+  </script>
+</section>
+```
+
+Use `index.html` by itself when the route is UI-only.
+
+## `index.py`
+
+`index.py` is the backend companion for a route.
+
+Add it when the route needs:
+
+- `page()`
+- metadata
+- auth checks or redirects
+- route-level `Cache(...)`
+- route-owned `@rpc()` actions
+- server-side context before rendering the sibling template
+
+For UI routes, `index.py` does not replace `index.html`. It prepares data and calls `render_page(__file__, ...)` so Caspian renders the sibling template.
+
+Example:
+
+```python
+from casp.layout import Metadata, render_page
+
+metadata = Metadata(
+    title="Dashboard | Caspian",
+    description="Overview page for the dashboard.",
+)
+
+
+async def page(params: dict, request=None):
+    return render_page(__file__, {
+        "slug": params.get("slug"),
+        "has_request": request is not None,
+    })
+```
+
+Current route-parameter behavior:
+
+- path params arrive as a single `params` dict
+- query params can be injected by name
+- `request` is injected by keyword when declared
+
+When one page needs to influence a wrapping layout, `page()` can return `(page_html, layout_props_dict)`.
+
+## `layout.html`
+
+`layout.html` is the shared wrapper for a route subtree.
+
+Use it for:
+
+- shared shell markup such as sidebars, headers, docs rails, or dashboard frames
+- the `[[children|safe]]` insertion point for child routes
+- shared layout props consumed as `[[ layout.* ]]`
+- shared metadata fields consumed as `[[ metadata.* ]]`
+
+Example:
+
+```html
+<div class="docs-shell">
+  <aside class="docs-nav">Docs navigation</aside>
+
+  <main class="docs-content" pp-reset-scroll="true">
+    [[children|safe]]
+  </main>
+</div>
+```
+
+Use nested `layout.html` files for sections like `dashboard/`, `docs/`, `account/`, or route groups such as `(marketing)/`.
+
+In grouped shells with separate shell and content scrolling, put `pp-reset-scroll="true"` on the content pane that should reset on child-route navigation. Leave persistent shell scrollers such as sidebars unmarked when they should keep their own scroll position.
+
+## `layout.py`
+
+`layout.py` is the Python companion for `layout.html`.
+
+Use it for:
+
+- shared synchronous props
+- metadata defaults for everything below that folder
+- small shared layout decisions that belong to the subtree rather than one page
+
+Return `render_layout(__file__), props` so the sibling `layout.html` stays the authored wrapper.
+
+Example:
+
+```python
+from casp.layout import Metadata, render_layout
+
+metadata = Metadata(
+    title="Docs Section | Caspian",
+    description="Shared metadata for the docs subtree.",
+)
+
+
+def layout():
+    return render_layout(__file__), {
+        "shell_class": "docs-shell",
+        "content_class": "docs-shell__content",
+    }
+```
+
+Important runtime detail: `layout()` is synchronous in the installed runtime. Put async I/O in route `page()` functions or route-owned `@rpc()` actions instead of awaiting inside `layout.py`.
+
+## `loading.html`
+
+`loading.html` provides route-scope loading UI for SPA navigation.
+
+The runtime indexes `src/app/**/loading.html` separately from routes and layouts. During navigation, the browser runtime looks for the closest matching loading file by URL scope and falls back up the path toward `/`.
+
+Examples:
+
+- `src/app/loading.html` can act as the root fallback loader
+- `src/app/dashboard/loading.html` applies to `/dashboard` and its descendants unless a closer loading file exists
+- `src/app/(marketing)/loading.html` applies to the grouped subtree even though `(marketing)` does not appear in the URL
+
+To preserve the surrounding shell during navigation, put `pp-loading-content="true"` on the live container that should be swapped during loading, usually the main content pane in a layout.
+
+Example layout shell:
+
+```html
+<main class="docs-content" pp-loading-content="true" pp-reset-scroll="true">
+  [[children|safe]]
+</main>
+```
+
+Example loader:
+
+```html
+<div class="space-y-4 p-6">
+  <div class="h-4 rounded bg-muted animate-pulse"></div>
+  <div class="h-24 rounded bg-muted/70 animate-pulse"></div>
+</div>
+```
+
+If no `pp-loading-content="true"` container exists, the browser runtime falls back to `document.body`.
+
+## `not-found.html`
+
+`src/app/not-found.html` is the global 404 page.
+
+When `main.py` catches a `404`, it looks for this exact root-level file, renders it through the nested layout pipeline, and returns it with a 404 status.
+
+Important scope rule:
+
+- this is a global root-level file, not a per-folder `not-found.html` convention in the current runtime
+
+The runtime passes `request` into the template context and sets default page metadata for the response.
+
+Use this file for branded invalid-URL fallbacks instead of leaving the app on the generic FastAPI or Starlette message.
+
+## `error.html`
+
+`src/app/error.html` is the global 500 page for unhandled exceptions.
+
+When `main.py` catches a general exception, it looks for this exact root-level file, renders it through the nested layout pipeline, and returns it with a 500 status.
+
+The runtime provides these template values:
+
+- `request`
+- `error_message`
+- `error_trace`
+
+In production, `error_trace` is suppressed. Keep this page safe for users and avoid exposing sensitive internals outside development environments.
+
+If rendering `error.html` fails, the runtime falls back to a minimal plain HTML 500 response.
+
+## Decision Rule
+
+Use this order when deciding where a concern belongs:
+
+1. Put visible route markup in `index.html`.
+2. Add `index.py` only when the same route needs backend work.
+3. Put shared subtree wrapper markup in `layout.html`.
+4. Add `layout.py` only when that shared shell needs synchronous Python props or metadata.
+5. Add `loading.html` when SPA navigation needs an immediate scoped loading state.
+6. Use root `not-found.html` for unmatched URLs.
+7. Use root `error.html` for unhandled exceptions.
+
+Use [routing.md](./routing.md) for the broader file-based routing model, [metadata.md](./metadata.md) for page and layout metadata, [cache.md](./cache.md) for route caching in `index.py`, and [pulsepoint.md](./pulsepoint.md) for the authored template contract and browser runtime behavior.

package/dist/docs/index.md

@@ -8,10 +8,11 @@ related:
     - /docs/ai-validation-checklist
     - /docs/installation
     - /docs/commands
-    - /docs/core-runtime-map
-    - /docs/pulsepoint-runtime-map
-    - /docs/mcp
+    - /docs/core-runtime-map
+    - /docs/pulsepoint-runtime-map
+    - /docs/mcp
     - /docs/file-uploads
+    - /docs/file-conventions
     - /docs/project-structure
     - /docs/components
 ---
@@ -24,41 +25,41 @@ The docs can mention optional features even when those features are disabled in
 
 Before making feature, tooling, or file-placement decisions in a Caspian project, read `./caspian.config.json` almost immediately. That file tells you which optional capabilities are enabled, such as Prisma, MCP, TypeScript, Tailwind, backend-only mode, and component scan directories.
 
-## Default Stack
-
-When generating or editing a Caspian app, treat these as the default choices unless the task explicitly requires something else:
+## Default Stack
+
+When generating or editing a Caspian app, treat these as the default choices unless the task explicitly requires something else:
 
 - Use PulsePoint for reactive frontend behavior.
 - Treat every authored route, layout, and component HTML file like a React component return value: exactly one top-level parent HTML element or one imported `x-*` root, with any owned plain `<script>` kept inside that same root.
 - When `caspian.config.json` has `tailwindcss: true`, use Python `merge_classes(...)` plus browser `twMerge(...)` as the only supported Tailwind class-merging path.
-- Use `@rpc()` plus `pp.rpc()` for browser-triggered reads, writes, streaming, and uploads.
-- Use `Validate` and `Rule` from `casp.validate` for server-side input validation and sanitization.
-
-## AI Doc Shape
-
-Each packaged feature doc should help AI answer the same five questions quickly:
-
-- When does this doc apply?
-- Which `caspian.config.json` flag, if any, gates the feature?
-- Which app-owned files usually change?
-- Which `main.py` or installed `casp` runtime files own the behavior?
-- Which behavior should be verified before editing or explaining the feature?
-
-Keep project-specific facts, temporary file inventory, and current app feature flags out of packaged docs. Put those in `AGENTS.md`, `.github/copilot-instructions.md`, or the project code instead.
-
-Use [core-runtime-map.md](./core-runtime-map.md) for Python-side Caspian runtime ownership and [pulsepoint-runtime-map.md](./pulsepoint-runtime-map.md) for browser-side PulsePoint runtime ownership.
-
-## Redundancy Rule
-
-Prefer one canonical explanation plus links from related docs.
-
-- Put the full authored-template and PulsePoint runtime contract in [pulsepoint.md](./pulsepoint.md).
-- Put the full route and layout placement contract in [routing.md](./routing.md).
-- Put the full reusable component contract in [components.md](./components.md).
-- Put the full Python runtime ownership map in [core-runtime-map.md](./core-runtime-map.md).
-- Put the full PulsePoint feature lookup map in [pulsepoint-runtime-map.md](./pulsepoint-runtime-map.md).
-
-Other docs should summarize these rules only when the reminder prevents a common mistake, then link to the canonical page instead of restating the whole rule block.
+- Use `@rpc()` plus `pp.rpc()` for browser-triggered reads, writes, streaming, and uploads.
+- Use `Validate` and `Rule` from `casp.validate` for server-side input validation and sanitization.
+
+## AI Doc Shape
+
+Each packaged feature doc should help AI answer the same five questions quickly:
+
+- When does this doc apply?
+- Which `caspian.config.json` flag, if any, gates the feature?
+- Which app-owned files usually change?
+- Which `main.py` or installed `casp` runtime files own the behavior?
+- Which behavior should be verified before editing or explaining the feature?
+
+Keep project-specific facts, temporary file inventory, and current app feature flags out of packaged docs. Put those in `AGENTS.md`, `.github/copilot-instructions.md`, or the project code instead.
+
+Use [core-runtime-map.md](./core-runtime-map.md) for Python-side Caspian runtime ownership and [pulsepoint-runtime-map.md](./pulsepoint-runtime-map.md) for browser-side PulsePoint runtime ownership.
+
+## Redundancy Rule
+
+Prefer one canonical explanation plus links from related docs.
+
+- Put the full authored-template and PulsePoint runtime contract in [pulsepoint.md](./pulsepoint.md).
+- Put the full route and layout placement contract in [routing.md](./routing.md).
+- Put the full reusable component contract in [components.md](./components.md).
+- Put the full Python runtime ownership map in [core-runtime-map.md](./core-runtime-map.md).
+- Put the full PulsePoint feature lookup map in [pulsepoint-runtime-map.md](./pulsepoint-runtime-map.md).
+
+Other docs should summarize these rules only when the reminder prevents a common mistake, then link to the canonical page instead of restating the whole rule block.
 
 ## Docs Location
 
@@ -72,11 +73,12 @@ The packaged Caspian docs referenced by this index live here:
 - `ai-validation-checklist.md` - workflow and representative prompts for checking whether AI can find the correct Caspian docs, core files, and verification checkpoints
 - `installation.md` - First-time setup flow for creating a new Caspian application
 - `commands.md` - Main Caspian CLI workflows for project creation, generation, updates, and config-aware maintenance
-- `core-runtime-map.md` - map of `main.py` plus installed `casp` modules to the packaged docs that explain them and the behaviors AI should trace there
-- `pulsepoint-runtime-map.md` - fast feature-to-runtime lookup for PulsePoint state, effects, refs, context, portals, lists, events, RPC, uploads, streaming, SPA navigation, scroll restoration, and Tailwind merge behavior
-- `mcp.md` - MCP-specific layout, launch flow, and AI routing rules for projects where `caspian.config.json` enables MCP
+- `core-runtime-map.md` - map of `main.py` plus installed `casp` modules to the packaged docs that explain them and the behaviors AI should trace there
+- `pulsepoint-runtime-map.md` - fast feature-to-runtime lookup for PulsePoint state, effects, refs, context, portals, lists, events, RPC, uploads, streaming, SPA navigation, scroll restoration, and Tailwind merge behavior
+- `mcp.md` - MCP-specific layout, launch flow, and AI routing rules for projects where `caspian.config.json` enables MCP
 - `database.md` - Prisma schema, migration, seed, and client-generation workflow for projects where `caspian.config.json` enables Prisma, plus Python-side helper caveats
 - `auth.md` - Session-backed authentication with `casp.auth`, centralized `auth_config.py`, public-vs-private route mode guidance, RPC-first signout guidance, RBAC, and OAuth provider helpers
+- `file-conventions.md` - quick decision guide for `index.html`, `index.py`, `layout.html`, `layout.py`, `loading.html`, `not-found.html`, and `error.html`, plus the owning runtime files to verify
 - `components.md` - Create reusable Python components, template-backed UI, HTML-first `x-*` component tags, the single-parent authored-root rule for component HTML files, and the Python-side `merge_classes(...)` contract when Tailwind CSS is enabled
 - `pulsepoint.md` - Default reactive frontend runtime contract for component scripts, state, effects, directives, SPA navigation scroll restoration, `pp-reset-scroll`, and direct browser `twMerge(...)` usage when Tailwind CSS is enabled
 - `fetch-data.md` - Initial server-side data loading and browser-triggered RPC flows with `pp.rpc()`, streaming, uploads, and auth-aware actions
@@ -98,13 +100,13 @@ Preferred lookup order:
 2. Read `./caspian.config.json` before making any feature assumption. A doc existing in the package does not mean that feature is enabled in the current project.
 3. Before authoring or editing any `src/app/**` or component HTML template, apply the single-root invariant: one authored root only, any owned `<script>` inside that root, and no handwritten `pp-component` or `type="text/pp"`.
 4. Treat `caspian.config.json` as the single source of truth for optional features. Use feature-specific docs only when the matching flag is enabled. If a feature is disabled and the user wants it, ask first, then update `caspian.config.json` and follow the update workflow in `commands.md`.
-5. If the task touches `main.py` or `.venv/Lib/site-packages/casp/**`, read `core-runtime-map.md` to jump to the controlling runtime file and the matching feature doc.
-6. If the task names a PulsePoint feature or directive, read `pulsepoint-runtime-map.md` for the fastest feature-to-runtime lookup, then read `pulsepoint.md` for authoring rules.
-7. After the feature is confirmed, inspect the actual project files that decide behavior, such as `package.json`, `main.py`, `src/app/**`, `src/lib/**`, `settings/**`, `prisma/**`, and the installed `casp` runtime.
-8. Use `commands.md` for scaffold and update workflows, `project-structure.md` for placement decisions, and the feature docs such as `mcp.md`, `database.md`, `auth.md`, `fetch-data.md`, and `file-uploads.md` for task-specific guidance.
-9. Prefer packaged Caspian docs before upstream documentation when generating code, commands, or migration guidance.
-10. Use `ai-validation-checklist.md` when you want to verify that the docs lead AI to the correct files and behavior checkpoints.
-11. Keep `index.md` and cross-links aligned so AI can quickly discover the right doc.
+5. If the task touches `main.py` or `.venv/Lib/site-packages/casp/**`, read `core-runtime-map.md` to jump to the controlling runtime file and the matching feature doc.
+6. If the task names a PulsePoint feature or directive, read `pulsepoint-runtime-map.md` for the fastest feature-to-runtime lookup, then read `pulsepoint.md` for authoring rules.
+7. After the feature is confirmed, inspect the actual project files that decide behavior, such as `package.json`, `main.py`, `src/app/**`, `src/lib/**`, `settings/**`, `prisma/**`, and the installed `casp` runtime.
+8. Use `file-conventions.md` for quick decisions about `index.html`, `index.py`, `layout.html`, `layout.py`, `loading.html`, `not-found.html`, and `error.html`; use `commands.md` for scaffold and update workflows, `project-structure.md` for placement decisions, and the feature docs such as `mcp.md`, `database.md`, `auth.md`, `fetch-data.md`, and `file-uploads.md` for task-specific guidance.
+9. Prefer packaged Caspian docs before upstream documentation when generating code, commands, or migration guidance.
+10. Use `ai-validation-checklist.md` when you want to verify that the docs lead AI to the correct files and behavior checkpoints.
+11. Keep `index.md` and cross-links aligned so AI can quickly discover the right doc.
 
 ## Maintenance
 

package/dist/docs/project-structure.md

@@ -185,16 +185,27 @@ If the local BrowserSync stack is running, keep that upload directory in `settin
 
 The application entry point for the project.
 
-In the current Caspian app shape, this file is where startup wiring happens:
-
-- load environment variables
-- call `configure_auth(build_auth_settings())`
-- register OAuth providers with `Auth.set_providers(...)`
-- create the FastAPI app
-- register routes and RPC handlers
-- add `SessionMiddleware`, CSRF middleware, auth middleware, and RPC middleware
-
-Use `main.py` for auth bootstrap and middleware-order changes. Use `src/lib/auth/auth_config.py` for auth policy values such as public routes, redirects, and RBAC maps.
+In the current Caspian app shape, this file is where startup wiring happens:
+
+- load environment variables
+- call `configure_auth(build_auth_settings())`
+- register OAuth providers with `Auth.set_providers(...)`
+- create the FastAPI app
+- register routes and RPC handlers
+- add security headers middleware, `SessionMiddleware`, CSRF middleware, auth middleware, and RPC middleware
+
+Use `main.py` for auth bootstrap and middleware-order changes. Use `src/lib/auth/auth_config.py` for auth policy values such as public routes, redirects, and RBAC maps.
+
+### `src/lib/security/runtime_security.py`
+
+When the app factors request hardening into an app-owned helper module, keep browser security headers, CSP defaults, safe public-file helpers, and production session-secret enforcement there instead of bloating `main.py`.
+
+Use this module when the task is about:
+
+- safe serving of files from `public/**`
+- browser security headers or content-security-policy defaults
+- production session-secret enforcement
+- user-facing vs production-safe exception messaging helpers
 
 ### `caspian.config.json`
 

package/dist/docs/routing.md

@@ -33,7 +33,7 @@ Start with these rules:
 - Use a standalone `index.py` only for non-visual routes such as redirects or action-only handlers.
 - When a folder owns child routes, use `layout.html` to wrap them. This is the default pattern for dashboards, admin sections, account areas, settings trees, and route groups.
 - In grouped shells with separate shell and content scrolling, put `pp-reset-scroll="true"` on the content pane in `layout.html` when that pane should reset on child-route navigation while the shell sidebar or rail keeps its own scroll.
-- Use `layout.py` when a layout needs shared synchronous props or metadata before rendering.
+- Use `layout.py` when a layout needs shared props or metadata before rendering. The `layout()` function may be synchronous or async.
 - Keep visible route and layout markup in `index.html` and `layout.html`. Treat `index.py` and `layout.py` as backend companions, not as places to author visible HTML.
 - Treat every authored route and layout template like a React component body: it must have exactly one top-level parent HTML element or one imported `x-*` root, and any owned plain `<script>` must live inside that same root.
 
@@ -86,7 +86,7 @@ When a user asks for a dashboard, admin area, account section, docs section, or
 
 - Create a parent folder for the section.
 - Add `layout.html` in that folder for the shared shell.
-- Add `layout.py` only when that shared shell needs synchronous props or metadata.
+- Add `layout.py` only when that shared shell needs shared props or metadata.
 - Put each child page in its own route folder with `index.html` and an optional `index.py` companion.
 - Use a normal folder name such as `dashboard/` when the section name should appear in the URL.
 - Use a route-group folder such as `(marketing)/` when the folder should organize code and own a layout without adding a URL segment.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "caspian-utils",
-  "version": "0.0.29",
+  "version": "0.0.31",
   "description": "Caspian tooling",
   "main": "index.js",
   "scripts": {