arcane-os 0.11.2 → 0.12.0

package/docs/reference/pwa.md

@@ -83,31 +83,90 @@ the [existing asset version contract](asset-versioning.md).
 
 ## Development and hosting
 
+Use the ordinary `arcane dev --app <id>` command after editing the selected
+app's `arcane-app.json`. Startup refreshes that app's generated
+`arcane-package.json` from the authored descriptor before refreshing its import
+maps and starting the server. No packaging or `dist` output is required.
+Package-only applications retain their existing descriptor workflow.
+
+Add a file or directory to `package.include` to make it part of the app's
+resources. A new file inside an already included directory needs no separate
+entry. If `package.pwa.offline.include` is nonempty, the resource must also
+match that offline selection and must not match `offline.exclude`. Adding a
+path only to the offline selection does not add it to the app's resources.
+Restart after changing descriptor settings. Edits to selected source files are
+picked up by the next due page-load check while the server remains running.
+
 For an enabled application, `arcane dev` serves the generated PWA files at the
 origin root and starts at the selected app page under `/apps/<id>/`. Use one
-selected app per development origin. Its worker uses network revalidation first
-and retains successful selected resource responses for offline use, so saved
-source edits remain visible on an ordinary online refresh.
-
-Source inventory work begins when the browser requests the worker update, after
-the page can start. It traverses the selected route inventory once for that
-request, follows application page and runtime resource references to retain
-selected query variants, and shares an in-flight traversal with concurrent
-manifest requests. Each referenced source file is read once per traversal;
+selected app per development origin. The SDK uses `node-http-server` for source
+and packaged-preview serving, including conditional resource responses.
+Every Arcane development server and packaged browser preview serves HTTPS,
+including localhost. Configure the workspace certificate pair before starting
+the ordinary command; see [development HTTPS setup](cli.md#development-https-setup).
+
+Source inventory work begins when the browser requests the worker or current
+offline manifest, after the page can start. It traverses the selected route
+inventory once for that request, follows page and runtime resource references
+to retain selected query variants, and shares an in-flight traversal with
+concurrent requests. Each referenced source file is read once per traversal;
 document corpus bodies remain under their existing owner. It does not rebuild
-the application. Generated metadata and bootstrap requests reuse the current
-bundle.
+the application. Installation metadata and bootstrap requests reuse the current
+generated bundle.
+
+The SDK owns version information in `arcane-offline.json`:
+
+| Field | Owner and update rule |
+| --- | --- |
+| `schemaVersion` | SDK offline-manifest format; currently `1`. |
+| `appVersion` | The app descriptor's top-level `version`. |
+| `sdkVersion` | The selected installed SDK or explicit live SDK source version. |
+| `revision` | `development` for the source server; a fresh generated deployment ID for each packaged output. |
 
-Packaged workers use their selected cache generation first. During installation,
-at most four resource requests run together; each response is fetched from the
-network with the browser's reload cache mode before it enters the candidate
-cache. The candidate must finish
-installation before the browser activates it. Failures retain the prior active
-worker and remain observable through native worker state and SDK diagnostics.
+Do not hand-edit generated manifests or bump a version for every source edit.
+`arcane.webmanifest` holds installation metadata and has no separate SDK-managed
+release counter. Resource freshness uses each response's `Last-Modified` header.
+
+On each page load, the SDK reads one `lastChecked` value for the app and worker
+scope from DBOPFS in the background. When that value is missing or older than
+the delivery mode's interval, it checks the current offline manifest and every
+selected resource:
+
+| Mode | Page-load check interval |
+| --- | --- |
+| Development | 120 seconds |
+| Packaged browser delivery | 15 minutes |
+
+These intervals schedule revalidation; they never expire a cached file. The
+worker sends `GET` with `If-Modified-Since` using the cached response's
+`Last-Modified`. A `304 Not Modified` retains the complete cached response. A
+successful `200` replaces it after the new response is stored. Missing cached
+resources are downloaded. Network and server failures retain an existing
+offline copy and remain observable through SDK diagnostics. A host without
+modification headers must send the current response because freshness cannot
+be established from a missing header.
+
+Complete resource responses remain in browser CacheStorage. DBOPFS stores one
+successful whole-cycle timestamp, updated only after the manifest and every
+selected resource have been checked successfully. A partial failure preserves
+the previous timestamp so the next page load can retry. Each cached response
+retains its own `Last-Modified` header, but there are no per-file check times.
+The SDK imposes no age-based cache deletion and
+retains resource bodies across app and SDK version changes. Requests for a page
+do not wait for the complete resource inventory to finish checking. The SDK
+uses at most four concurrent background resource requests and starts no timer
+or polling loop between page loads.
+
+During initial installation, selected resources are cached before the browser
+activates the worker. Failures remain observable through native worker state
+and SDK diagnostics. HTTP modification dates have second-level precision;
+hosts must report changes to the served representation, including generated
+output, rather than only the date of an unrelated source file.
 
 The SDK development server sends `Cache-Control: no-cache` for PWA resources.
-An independent static host must also revalidate stable HTML, module, style,
-import-map and worker URLs, and serve JavaScript with a JavaScript content type.
+An independent static host should support `Last-Modified` and conditional GET
+for stable HTML, module, style and import-map URLs, revalidate worker URLs, and
+serve JavaScript with a JavaScript content type.
 Keep the worker beside the deployment root it controls. The browser requires a
 supported secure context, such as trusted HTTPS or localhost, to register it.
 The SDK does not change certificates or browser permissions.
@@ -132,9 +191,15 @@ worker while they are open. Closing those pages allows activation; a refresh
 can leave overlapping document clients and keep the update waiting.
 
 The SDK does not call `skipWaiting`, claim the initial page, reload a page,
-restart a model or poll for updates. Activation retires only obsolete resource
-caches belonging to that exact app and registration scope. Saved application
-data and caches owned by other capabilities are untouched.
+restart a model or poll for updates. Worker activation preserves cached
+resources and the DBOPFS check history for the same app and registration scope.
+Saved application data and caches owned by other capabilities are untouched.
+
+When importing caches from an older SDK worker, the new worker fetches the
+SDK-owned registration bootstrap and PWA client once so the page can use the
+current cache-check protocol. Other cached resource bodies retain the normal
+check cadence. This transition follows native worker installation and
+activation without forcing a page reload.
 
 Switching a server from a packaged release to live development does not replace
 an already active release worker inside an open document. The same native

package/docs/reference/sdk-api.md

@@ -3459,6 +3459,15 @@ async function useselectApp(...arguments_) {
 
 Starts one owned browser development server with exact runtime/app route mappings and a caller-selected bind address.
 
+HTTPS serving uses the published `node-http-server` module. The SDK
+selects source routes and supplies generated representations; the module owns
+static-file conditional GET/HEAD handling and response delivery. The SDK
+retains modification dates for its generated representations. Unchanged
+resources can return `304` with no body while changed resources return their
+complete current representation. PEM-path listeners use module deployment;
+explicit raw `tls` options retain native HTTPS transport with the same module
+response and static-file operations.
+
 ### Signature, modes, and result
 
 ```text
@@ -3466,53 +3475,64 @@ async startDevServer(options={})
 ```
 
 Import it from `arcane-os`. Source mode accepts
-`{workspaceRoot=process.cwd(), appId, mode='source', host='127.0.0.1', port=0,
-https=false, certPath, keyPath, tls, signal, onEvent}` and serves one validated
+`{workspaceRoot=process.cwd(), appId, mode='source', host='127.0.0.1', port=0, httpPort=0,
+certPath, keyPath, tls, signal, onEvent}` and serves one validated
 workspace application plus its complete SDK or integrated runtime. Packaged mode uses
-`{mode:'packaged', releaseRoot, host, port, signal, onEvent}` and serves the
-complete selected release files. `host` defaults to `127.0.0.1` and accepts an
+`{mode:'packaged', releaseRoot, workspaceRoot, host, port, httpPort, certPath, keyPath, tls,
+signal, onEvent}` and serves the complete selected release files.
+`host` defaults to `127.0.0.1` and accepts an
 explicit network address or hostname. Use `0.0.0.0` for all IPv4 interfaces or
-`::` for the platform's IPv6 wildcard listener; port `0` asks the operating
-system for an available port.
-
-`https:true` reads `.arcane/dev/server-cert.pem` and
+`::` for the platform's IPv6 wildcard listeners. `port` selects the HTTPS
+application port, and `httpPort` selects the paired HTTP redirect port. Each
+defaults to `0`, which asks the operating system for an available port.
+The SDK request hook returns `308` for HTTP with a `Location` pointing to the
+actual HTTPS port while preserving the original request path and query. Both listeners
+use the selected host.
+
+Every source server and packaged browser preview enforces HTTPS, including
+localhost. The legacy `https` option is accepted but cannot disable it.
+Startup reads `.arcane/dev/server-cert.pem` and
 `.arcane/dev/server-key.pem` relative to `workspaceRoot` unless explicit
-`certPath` and `keyPath` are supplied together. The path pair also selects
-HTTPS without `https:true`; relative paths resolve from the workspace. A
-direct `tls` object instead supplies Node HTTPS server options, including
+`certPath` and `keyPath` are supplied together; relative paths resolve from the
+workspace. A direct `tls` object instead supplies Node HTTPS server options, including
 `cert` and `key`, without reading certificate files. Keep private material
 server-side. Missing PEM files and Node certificate/key parse errors reject
 startup without falling back to HTTP. Node owns TLS option handling and the
 handshake; browser trust and address matching are evaluated when a client
-connects. The CLI's `--public` selects HTTPS and the wildcard bind;
+connects. The CLI's `--public` selects the wildcard bind;
 the API's `host` option alone changes only the bind address.
 
-The promise settles after the listener is ready and resolves to
+The promise settles after both listeners are ready and resolves to
 `{server, protocol, mode, workspaceRoot, appId, host, port, origin, cleanUrl, url,
-networkUrls, close, closed, lifecycle}`. `server` is the raw Node HTTP or HTTPS
-server; `protocol` is `'http:'` or `'https:'`.
+networkUrls, httpPort, httpOrigin, httpUrl, close, closed, lifecycle}`. `server` is the raw Node HTTPS
+server; `protocol` is `'https:'`.
 `url` and `cleanUrl` are the same application URL. Wildcard listeners use
 `localhost` in that local URL; `host` retains the actual bound address.
+`httpPort` is the actual HTTP listener port, `httpOrigin` is its HTTP origin,
+and `httpUrl` combines that origin with the application start path. These
+fields identify the redirect endpoint; `origin`, `url`, `cleanUrl`, and
+`networkUrls` identify HTTPS application endpoints.
 `networkUrls` lists application URLs for applicable non-loopback interface
 addresses discovered once at startup. These URLs are connection candidates,
 not evidence of reachability from another device. The server adds no session
 capability or authentication. In packaged mode, `workspaceRoot` and `appId`
 are `null`.
 
-All returned application URLs use the selected transport's scheme. The server
-reads one PEM pair per startup and does not create certificates or modify trust
-stores. Each client must trust the issuing CA and open an address covered by the
+PEM-path startup reads the selected pair. The server does not create
+certificates or modify trust stores. Each client must trust the issuing CA and open an address covered by the
 server certificate. Lifecycle events and CLI summaries exclude TLS options and
 private key contents. See [development HTTPS setup](cli.md#development-https-setup).
 
-Starting the server opens the selected listener and emits awaited,
-backpressured `server.starting` and `server.started` events. Request failures
-emit `server.request.failed`; shutdown emits `server.stopped` after owned
+Starting the server opens both selected listeners and emits awaited,
+backpressured `server.starting` and `server.started` events. `server.started`
+includes `httpPort`, `httpOrigin`, and `httpUrl` alongside the HTTPS endpoint.
+Request failures emit `server.request.failed`; shutdown emits `server.stopped` after owned
 requests and event delivery drain. Call `await result.close()` in a `finally`
 block, or abort `signal`; `close()` is idempotent and returns the
-same settlement represented by both `closed` and `lifecycle`. A listener error
-or event-callback failure closes the server and rejects its lifecycle. Invalid
-mode/host/port, malformed workspace or release content, an occupied port,
+same settlement represented by both `closed` and `lifecycle`. Closing the
+operation closes both listeners. An error from either listener or an
+event-callback failure closes both and rejects the lifecycle. Invalid
+mode/host/port/httpPort, malformed workspace or release content, an occupied port,
 or an already-aborted signal rejects startup.
 
 ### Availability and normalization
@@ -3907,9 +3927,16 @@ async function usedescribeTargets(...arguments_) {
 Starts one owned browser development server for the selected application.
 
 `https`, `certPath`, `keyPath`, and `tls` follow the
-[`startDevServer()` TLS contract](#startdevserver), alongside `host` and `port`.
-The operation refreshes the selected app's managed import maps once, then owns
-one source listener and returns its protocol, URLs, and shutdown lifecycle.
+[`startDevServer()` TLS contract](#startdevserver), alongside `host`, `port`, and
+`httpPort`. `port` selects HTTPS and `httpPort` selects the HTTP `308` redirect
+listener; each defaults to an available port.
+The operation refreshes the selected authored descriptor's `arcane-package.json`
+projection and managed import maps under one development-refresh lock, then
+releases that lock before opening the HTTPS source listener and its HTTP
+redirect listener. It returns both endpoints and their shared shutdown
+lifecycle. Legacy package-only apps remain unchanged. The operation generates
+no packaged output; enabled PWA manifests
+are served directly from the selected source resources.
 
 ### Signature and result
 
@@ -6075,6 +6102,13 @@ registration and returns a synchronous owner with `ready`, `state`, `subscribe`,
 `update` and `dispose`. It does not block page rendering or load models.
 Unsupported environments receive an explicit unsupported state.
 
+The generated page bootstrap also starts the SDK's background resource check.
+It stores one `lastChecked` timestamp per app/cache in DBOPFS, advancing it only
+after the full resource check succeeds, and revalidates on page load after
+120 seconds in development or 15 minutes in packaged delivery. Complete
+responses remain cached without SDK expiration; `304` retains the response
+and a successful replacement updates it. See the [PWA guide](pwa.md).
+
 ### Example
 
 ```javascript

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "arcane-os",
-  "version": "0.11.2",
+  "version": "0.12.0",
   "description": "Arcane OS JavaScript SDK, project-local CLI, browser runtime, and repository-portable application packager.",
   "type": "module",
   "main": "./src/index.mjs",
@@ -105,6 +105,7 @@
   },
   "dependencies": {
     "event-pubsub": "6.1.0",
+    "node-http-server": "9.1.1",
     "strong-type": "2.0.1",
     "vanilla-test": "2.1.3"
   },

package/src/app-descriptor.mjs

@@ -1,7 +1,8 @@
 import Is from 'strong-type';
 import {isDeepStrictEqual} from 'node:util';
-import {readFile} from 'node:fs/promises';
+import {readFile, writeFile} from 'node:fs/promises';
 import path from 'node:path';
+import {throwIfAborted} from './errors.mjs';
 import {normalizePwaConfig} from './pwa.mjs';
 import {
     ARCANE_MACHINE_BUNDLE_VERSION,
@@ -424,6 +425,47 @@ async function readJsonFile(filePath,label,{optional=false}={}){
     }
 }
 
+export async function refreshAppPackageProjection({workspaceRoot, appId, signal, onEvent}) {
+    throwIfAborted(signal);
+    const appRoot = path.join(workspaceRoot, 'apps', appId);
+    const descriptorPath = path.join(appRoot, APP_DESCRIPTOR_NAME);
+    const authored = await readJsonFile(
+        descriptorPath,
+        `apps/${appId}/${APP_DESCRIPTOR_NAME}`,
+        {optional: true}
+    );
+    if (authored === null) return {updated: false};
+
+    validateAppDescriptor(
+        authored,
+        {appId}
+    );
+    const projection = projectPackageManifest(authored);
+    const packagePath = path.join(appRoot, 'arcane-package.json');
+    const packageManifest = await readJsonFile(
+        packagePath,
+        `apps/${appId}/arcane-package.json`
+    );
+    const packageProjection = packageManifest.pwa === undefined ? packageManifest : {
+        ...packageManifest,
+        pwa: normalizePwaConfig(packageManifest.pwa)
+    };
+    if (isDeepStrictEqual(projection, packageProjection)) return {updated: false};
+
+    // Development updates the generated projection before strict workspace readers run.
+    throwIfAborted(signal);
+    await writeFile(packagePath, `${JSON.stringify(projection, null, 2)}\n`, 'utf8');
+    await onEvent?.(
+        {
+            type: 'workspace.application.projection.updated',
+            workspaceRoot,
+            appId,
+            path: packagePath
+        }
+    );
+    return {updated: true};
+}
+
 export async function loadAppDescriptor({workspaceRoot,appRoot,appId,packageManifest}){
     const descriptorPath=path.join(appRoot,APP_DESCRIPTOR_NAME);
     const authored=await readJsonFile(descriptorPath,`apps/${appId}/${APP_DESCRIPTOR_NAME}`,{optional:true});

package/src/cli/main.mjs

@@ -18,6 +18,7 @@ const VALUE_OPTIONS=new Set([
     'arcane-root',
     'host',
     'port',
+    'http-port',
     'cert',
     'key',
     'target',
@@ -64,7 +65,7 @@ Usage:
   ${CLI_NAME} upgrade [--workspace <directory>] [--app <id>]
   ${CLI_NAME} doctor [--workspace <directory>] [--arcane-root <directory>]
   ${CLI_NAME} import-map [--workspace <directory>] [--app <id>]
-  ${CLI_NAME} dev [--app <id>] [--public] [--https] [--cert <pem>] [--key <pem>] [--host <address>] [--port 8000] [--sdk-runtime-source <sdk-root>]
+  ${CLI_NAME} dev [--app <id>] [--public] [--https] [--cert <pem>] [--key <pem>] [--host <address>] [--port 8000] [--http-port 0] [--sdk-runtime-source <sdk-root>]
   ${CLI_NAME} test [--app <id>] [--scope app]
   ${CLI_NAME} test --scope shared --test-file <repo-relative.test.mjs>
   ${CLI_NAME} check [--app <id>] [--scope app] [--skip-tests]
@@ -76,7 +77,7 @@ Usage:
   ${CLI_NAME} native-doctor --target <native-target> --arcane-root <directory>
   ${CLI_NAME} native-prepare --target <native-target> --arcane-root <directory>
   ${CLI_NAME} build --target <target> [--arcane-root <directory>] [--output-root <directory>] [--format <format>] [--signing <mode>]
-  ${CLI_NAME} run [--target <target>] [--app <id>] [--arcane-root <directory>] [--output-root <directory>] [--format <format>] [--signing <mode>]
+  ${CLI_NAME} run [--target <target>] [--app <id>] [--cert <pem>] [--key <pem>] [--port 8000] [--http-port 0] [--arcane-root <directory>] [--output-root <directory>] [--format <format>] [--signing <mode>]
   ${CLI_NAME} update-check
   ${CLI_NAME} targets
   ${CLI_NAME} repo status|pull|push
@@ -88,9 +89,10 @@ Usage:
 
 Development:
   --public                      Serve HTTPS on all IPv4 interfaces (0.0.0.0) and print network URLs.
-  --https                       Use HTTPS with .arcane/dev/server-cert.pem and server-key.pem.
+  --https                       Accepted for compatibility; Arcane browser serving always uses HTTPS.
   --cert <pem> --key <pem>        Use an existing certificate pair; paths are relative to the workspace.
   --host <address>               Override the bind address; takes precedence over --public.
+  --http-port <port>             Browser dev/run HTTP redirect port; 0 selects an available port (default).
   --sdk-runtime-source <sdk-root>  Dev-only live SDK checkout; omitted preserves the workspace runtime mode.
 
 Global:
@@ -456,9 +458,25 @@ function operationOptions(command,parsed,cwd){
     if(flags.has('public')&&command!=='dev'){
         usage('--public is supported only by dev.');
     }
-    if((flags.has('https')||values.cert!==undefined||values.key!==undefined)&&command!=='dev'){
-        usage('--https, --cert, and --key are supported only by dev.');
+    const browserServing = command === 'dev'
+        || (command === 'run' && (values.target ?? 'browser') === 'browser');
+    if ((flags.has('https') || values.cert !== undefined || values.key !== undefined) && !browserServing) {
+        usage('--https, --cert, and --key are supported only by dev and run --target browser.');
     }
+    if (values['http-port'] !== undefined && !browserServing) {
+        usage('--http-port is supported only by dev and run --target browser.');
+    }
+    if (browserServing && (values.cert === undefined) !== (values.key === undefined)) {
+        usage('Arcane HTTPS serving requires --cert and --key together.');
+    }
+    const browserServerOptions = browserServing ? {
+        https: true,
+        httpPort: readPort(values['http-port'], 0),
+        ...(values.cert === undefined ? {} : {
+            certPath: path.resolve(workspaceRoot, values.cert),
+            keyPath: path.resolve(workspaceRoot, values.key)
+        })
+    } : {};
     if(flags.has('overwrite')&&command!=='bundle'){
         usage('--overwrite is supported only by bundle.');
     }
@@ -518,18 +536,11 @@ function operationOptions(command,parsed,cwd){
     }
     if(command==='dev'){
         noExtraPositionals(command,positionals);
-        if((values.cert===undefined)!==(values.key===undefined)){
-            usage('HTTPS development requires --cert and --key together.');
-        }
         return {
             ...common,
+            ...browserServerOptions,
             host:values.host??(flags.has('public')?'0.0.0.0':'127.0.0.1'),
             port:readPort(values.port,8000),
-            https:flags.has('public')||flags.has('https')||values.cert!==undefined,
-            ...(values.cert===undefined?{}:{
-                certPath:path.resolve(workspaceRoot,values.cert),
-                keyPath:path.resolve(workspaceRoot,values.key)
-            }),
             ...(values['sdk-runtime-source']===undefined?{}:{
                 sdkRuntimeSourceRoot:path.resolve(cwd,values['sdk-runtime-source'])
             })
@@ -609,6 +620,7 @@ function operationOptions(command,parsed,cwd){
         noExtraPositionals(command,positionals);
         return {
             ...common,
+            ...browserServerOptions,
             target:values.target??'browser',
             arcaneRoot:values['arcane-root']?path.resolve(cwd,values['arcane-root']):undefined,
             outputRoot:values['output-root']?path.resolve(cwd,values['output-root']):undefined,
@@ -853,6 +865,9 @@ function serverSummary(result){
         host:result.host,
         port:result.port,
         url:result.url,
+        ...(result.httpPort===undefined?{}:{httpPort:result.httpPort}),
+        ...(result.httpOrigin===undefined?{}:{httpOrigin:result.httpOrigin}),
+        ...(result.httpUrl===undefined?{}:{httpUrl:result.httpUrl}),
         ...(result.protocol===undefined?{}:{protocol:result.protocol}),
         ...(result.networkUrls===undefined?{}:{networkUrls:result.networkUrls}),
         ...(result.callerAuthentication
@@ -867,6 +882,7 @@ function serverSummary(result){
 async function waitForServer(result,signal,reporter){
     const readyMessage=[
         `Development server ready at ${result.url}`,
+        ...(result.httpUrl ? [`HTTP redirect: ${result.httpUrl}`] : []),
         ...(result.networkUrls??[]).map(function networkAddress(url){return `Network: ${url}`;})
     ].join('\n');
     reporter.emit('server.ready',serverSummary(result),readyMessage);