@_linked/server 3.8.0 → 3.10.0

package/CHANGELOG.md

@@ -1,5 +1,76 @@
 # @\_linked/server
 
+## 3.10.0
+
+### Minor Changes
+
+- [#208](https://github.com/linked-fw/server/pull/208) [`94a7776`](https://github.com/linked-fw/server/commit/94a77765eb3009fd531e0db2bfdfd1b59a9a2d79) Thanks [@flyon](https://github.com/flyon)! - The host control channel no longer reads the deprecated `CN_APP_ADMIN_SECRET` and
+  `x-cn-admin-secret` names that 3.9.0 still accepted. Its secret comes from `APP_ADMIN_SECRET` and
+  the `x-app-admin-secret` header only.
+
+  Breaking changes:
+
+  - `CN_APP_ADMIN_SECRET` is no longer read: a process started with only that variable has no
+    control channel (every `/admin/*` route answers 404). Set `APP_ADMIN_SECRET`.
+  - The `x-cn-admin-secret` header is no longer read: a request that carries the secret only there
+    answers 401. Send `x-app-admin-secret`.
+  - `LEGACY_ADMIN_SECRET_ENV`, `LEGACY_ADMIN_SECRET_HEADER` and the `warn` handler of
+    `registerAdminRoutes` are removed, and `resolveAdminSecret` takes only the environment.
+
+## 3.9.0
+
+### Minor Changes
+
+- [#206](https://github.com/linked-fw/server/pull/206) [`dc25a49`](https://github.com/linked-fw/server/commit/dc25a490a9a115e366d99f8db644d236043d609e) Thanks [@flyon](https://github.com/flyon)! - The host control channel reads its secret from `APP_ADMIN_SECRET` and its requests carry it in the
+  `x-app-admin-secret` header. New route `GET /admin/routes` answers the graph routes the process
+  runs with: `{datasets: [{store, id?, default, routes, defaultGraphShapes}]}`, one entry per dataset
+  the router knows, `routes` as `{<shape IRI>: <graph IRI>}` as core's routing holds them once the
+  app's storage file has run, and the shapes each dataset writes to its default graph. Like every
+  `/admin/*` route it answers 404 when the process has no admin secret.
+
+  Renamed, with a fallback (each old name logs one deprecation warning per process):
+
+  - `CN_APP_ADMIN_SECRET` → `APP_ADMIN_SECRET`; the old variable is read when the new one is unset.
+  - The `x-cn-admin-secret` header → `x-app-admin-secret`; the old header is read when the new one
+    is absent.
+
+  The secret is compared through sha256 digests of both sides, so the comparison does not reveal
+  its length.
+
+  No breaking changes: an app or host that sets only the old names keeps working. Requires
+  `@_linked/core` `^2.39.1` (`getShapeGraphs` and `graphRoutingFor` in `utils/graphRouting`).
+
+- [#206](https://github.com/linked-fw/server/pull/206) [`fe012b1`](https://github.com/linked-fw/server/commit/fe012b1291d45763d779e8a8b0e9c18900efbca4) Thanks [@flyon](https://github.com/flyon)! - Shape graphs in the app. The boot and `POST /admin/sync-shapes` sync the registered shapes with
+  core's `syncShapeGraphs`: each package's shapes in its own named graph
+  (`{package IRI}/graph/shapes`, a `ShapesGraph` record in the graph registry), skipped when the
+  stored content hash matches. An installed package's graph comes from its built
+  `lib/esm/data/shapes.ttl`; the app's own packages (the app and its workspace packages) are
+  generated in process, in dev through Vite's loader; the app's single-owner shapes go in
+  `{DATA_ROOT}/graph/shapes`. The boot logs the graphs it writes, and the route reports them as
+  `shapeGraphs`. A graph refusal `graph-not-routed`, and any graph refusal code the server does not
+  know, is answered 400.
+
+  Breaking changes:
+
+  - The default-graph shape sync is replaced: shapes are written into shape graphs, and the shapes an
+    earlier server wrote into the default graph are removed there once their package's graph holds
+    them. A store without graph queries still gets the default-graph sync.
+  - An installed package without `lib/esm/data/shapes.ttl` gets no shape graph, and its shapes are
+    not generated: packages need a release built with a cli that writes `shapes.ttl`.
+  - `ShapeSyncResult.written` (the route's `written`) counts the shape graphs written, not shapes;
+    `shapeGraphs` holds the detail.
+  - `LINKED_SYNC_SHAPES_PRUNE_ORPHANS=true` drops the shape graphs of packages no longer registered
+    (never the app's own graph), instead of default-graph node shapes; it is skipped, with a warning,
+    while a registered package has no `shapes.ttl`.
+  - Requires `@_linked/core` `^2.39.1` (`shapes/syncShapeGraphs`, `shapes/shapesToTurtle`).
+
+  Deprecated: `materializeShapes` and `SyncShapesFn` in `utils/syncShapes` stay as they were, for
+  code built against an earlier server; `syncShapeGraphsInto` replaces them.
+
+  An own package whose shapes could not be generated (the dev loader failed and the server's core
+  holds none of them) is listed in `withoutTurtle`, with a warning, instead of being left out of the
+  report.
+
 ## 3.8.0
 
 ### Minor Changes

package/lib/esm/data/shapes.ttl

@@ -0,0 +1,26 @@
+@prefix core: <https://linked.cm/ont/core/> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix sh: <http://www.w3.org/ns/shacl#> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+
+<https://linked.cm/shape/server/LincdAPI>
+  a sh:NodeShape ;
+  sh:targetClass <https://linked.cm/ont/server/LincdAPI> .
+
+<https://linked.cm/shape/server/LincdWebApp>
+  a sh:NodeShape ;
+  sh:property <https://linked.cm/shape/server/LincdWebApp/api> ;
+  sh:targetClass <https://linked.cm/ont/server/LincdWebApp> .
+
+<https://linked.cm/shape/server/LincdWebApp/api>
+  a sh:PropertyShape ;
+  rdfs:label "api" ;
+  sh:maxCount "1"^^xsd:integer ;
+  sh:node <https://linked.cm/shape/server-utils/Lincd_API_Client> ;
+  sh:nodeKind sh:IRI ;
+  sh:path <https://linked.cm/ont/server/hasAPI> .
+
+<https://linked.cm/shape/server/LinkedServer>
+  a sh:NodeShape ;
+  sh:targetClass <https://linked.cm/ont/server/LincdServer> .

package/lib/esm/shapes/LinkedServer.d.ts

@@ -84,6 +84,8 @@ export declare class LinkedServer extends Shape {
     private linkedPackageDirs;
     /** The app's own workspace packages (`linkedPackage: true`), by real path. */
     private workspacePackageDirs;
+    /** The names of the app's own workspace packages. */
+    private workspacePackageNames;
     private warnedDuplicateInstalls;
     private resizePathsMap;
     private api;
@@ -105,30 +107,32 @@ export declare class LinkedServer extends Shape {
     initOnly(): Promise<this>;
     private initOnlyBoot;
     /**
-     * Materialize this server's registered shapes into the RDF store as SHACL
-     * (core `syncShapes` → pure `sh:NodeShape` + property shapes), so a running
-     * app's app-data holds the shapes its instances validate/query against.
-     * Whatever shapes the app package registers (published re-exports +
-     * generated) are written on boot, and again when the host asks for it over
-     * the control channel ({@link resyncShapes}). A source reload (HMR) does not
-     * sync shapes by itself.
+     * Sync this server's registered shapes into app data, one named graph per package (see
+     * utils/syncShapes): each package's shapes in `{package IRI}/graph/shapes`, from its built
+     * `shapes.ttl` (an installed package without one gets no shape graph), the app's own packages
+     * generated in process in dev (through Vite's loader), and the app's generated single-owner
+     * shapes in `{DATA_ROOT}/graph/shapes`. Runs at boot, after the ontology sync, and again when
+     * the host asks for it over the control channel ({@link resyncShapes}). A source reload (HMR)
+     * does not sync shapes by itself.
      *
-     * Core writes only the shapes whose stored hash differs, so a sync of
-     * unchanged shapes writes nothing.
+     * A graph whose stored content hash matches is not written, so a sync of unchanged shapes
+     * writes nothing. Shapes an earlier sync wrote into the default graph are removed there once
+     * their package's graph holds them; a store without graph queries gets that default-graph sync
+     * instead.
      *
-     * Additive only: store shapes this process did not register are left alone
-     * unless the operator sets `LINKED_SYNC_SHAPES_PRUNE_ORPHANS=true` — see
-     * `bootOrphanScope` in utils/syncShapes for why.
+     * Shape graphs of packages no longer registered stay unless the operator sets
+     * `LINKED_SYNC_SHAPES_PRUNE_ORPHANS=true` — see `bootOrphanScope` in utils/syncShapes.
      *
      * Opt-out via `linked.config` `syncShapesOnBoot` (default true); env
-     * LINKED_SYNC_SHAPES_ON_BOOT overrides for deployment. CN sets
-     * `syncShapesOnBoot: false` in its own linked.config — its default dataset is
-     * a context-router that can't materialize context-free, and it syncs its own
-     * pinned shapes separately (linked.backend.storage.js).
+     * LINKED_SYNC_SHAPES_ON_BOOT overrides for deployment. A host whose default
+     * dataset is a context router (one that can't write context-free) sets
+     * `syncShapesOnBoot: false` and syncs its shapes itself.
      */
     private materializeShapesIntoStore;
     /** One shape sync into the default dataset; throws when it fails. */
     private syncRegisteredShapes;
+    /** The root directory of a package with shapes: the app, a discovered linked package, or one the app resolves. */
+    private shapePackageDir;
     /**
      * Boot: sync the registered ontologies into their graphs in app data and run the shape
      * check, before the shape sync (see utils/ontologySync). The sync follows the shape sync's

package/lib/esm/shapes/LinkedServer.js

@@ -137,9 +137,9 @@ import { resolveStaticAccessURL, staticAssetURL, } from '../utils/releaseManifes
 import { resolveBootstrapEntry, resolveViteMainEntry, resolveViteServingMode, } from '../utils/bootstrapEntry.js';
 import { resolveRouteAssets } from '../utils/routeAssets.js';
 import { resolveRouteComponent, ssrCssInlineUrls, ssrCssScope, } from '../utils/ssrCss.js';
-import { bootOrphanScope, materializeShapes } from '../utils/syncShapes.js';
+import { appShapesGraph, bootOrphanScope, loadShapeGraphCore, syncShapeGraphsInto, viteShapesTurtleGenerator, } from '../utils/syncShapes.js';
 import { bootSyncsOntologies, loadOntologyCore, runOntologyPhase, ontologyPackageDirs, sourceModuleImporter, shapeSyncOn, syncOntologiesThenShapes, } from '../utils/ontologySync.js';
-import { AdminRequestError, assertReloadablePackages, registerAdminRoutes, } from '../utils/adminChannel.js';
+import { AdminRequestError, assertReloadablePackages, registerAdminRoutes, resolveAdminSecret, } from '../utils/adminChannel.js';
 import { envMs, withTimeout } from '../utils/withTimeout.js';
 import { invalidateSourceUnder } from '../utils/reloadSource.js';
 import { installSpaFallback, repinSpaFallback, } from '../utils/spaFallback.js';
@@ -286,6 +286,8 @@ let LinkedServer = class LinkedServer extends Shape {
         this.linkedPackageDirs = new Map();
         /** The app's own workspace packages (`linkedPackage: true`), by real path. */
         this.workspacePackageDirs = [];
+        /** The names of the app's own workspace packages. */
+        this.workspacePackageNames = new Set();
         this.warnedDuplicateInstalls = new Set();
         //from resizedFileName to full resized path (CDN or similar)
         this.resizePathsMap = new Map();
@@ -366,49 +368,73 @@ let LinkedServer = class LinkedServer extends Shape {
         return this;
     }
     /**
-     * Materialize this server's registered shapes into the RDF store as SHACL
-     * (core `syncShapes` → pure `sh:NodeShape` + property shapes), so a running
-     * app's app-data holds the shapes its instances validate/query against.
-     * Whatever shapes the app package registers (published re-exports +
-     * generated) are written on boot, and again when the host asks for it over
-     * the control channel ({@link resyncShapes}). A source reload (HMR) does not
-     * sync shapes by itself.
+     * Sync this server's registered shapes into app data, one named graph per package (see
+     * utils/syncShapes): each package's shapes in `{package IRI}/graph/shapes`, from its built
+     * `shapes.ttl` (an installed package without one gets no shape graph), the app's own packages
+     * generated in process in dev (through Vite's loader), and the app's generated single-owner
+     * shapes in `{DATA_ROOT}/graph/shapes`. Runs at boot, after the ontology sync, and again when
+     * the host asks for it over the control channel ({@link resyncShapes}). A source reload (HMR)
+     * does not sync shapes by itself.
      *
-     * Core writes only the shapes whose stored hash differs, so a sync of
-     * unchanged shapes writes nothing.
+     * A graph whose stored content hash matches is not written, so a sync of unchanged shapes
+     * writes nothing. Shapes an earlier sync wrote into the default graph are removed there once
+     * their package's graph holds them; a store without graph queries gets that default-graph sync
+     * instead.
      *
-     * Additive only: store shapes this process did not register are left alone
-     * unless the operator sets `LINKED_SYNC_SHAPES_PRUNE_ORPHANS=true` — see
-     * `bootOrphanScope` in utils/syncShapes for why.
+     * Shape graphs of packages no longer registered stay unless the operator sets
+     * `LINKED_SYNC_SHAPES_PRUNE_ORPHANS=true` — see `bootOrphanScope` in utils/syncShapes.
      *
      * Opt-out via `linked.config` `syncShapesOnBoot` (default true); env
-     * LINKED_SYNC_SHAPES_ON_BOOT overrides for deployment. CN sets
-     * `syncShapesOnBoot: false` in its own linked.config — its default dataset is
-     * a context-router that can't materialize context-free, and it syncs its own
-     * pinned shapes separately (linked.backend.storage.js).
+     * LINKED_SYNC_SHAPES_ON_BOOT overrides for deployment. A host whose default
+     * dataset is a context router (one that can't write context-free) sets
+     * `syncShapesOnBoot: false` and syncs its shapes itself.
      */
     async materializeShapesIntoStore() {
         try {
-            await this.syncRegisteredShapes();
+            await runAsSystem(() => this.syncRegisteredShapes(), 'shape sync at boot');
         }
         catch (err) {
-            console.warn('[LinkedServer] shape materialization failed (non-fatal):', err);
+            console.warn('[LinkedServer] shape sync failed (non-fatal):', err);
         }
     }
     /** One shape sync into the default dataset; throws when it fails. */
     async syncRegisteredShapes() {
-        var _a;
+        var _a, _b, _c, _d, _e;
         if (!shapeSyncOn(process.env, (_a = this.config) === null || _a === void 0 ? void 0 : _a.syncShapesOnBoot)) {
             return { written: 0, skipped: 'disabled' };
         }
         const appData = LinkedStorage.getDefaultDataset();
         if (!appData)
             return { written: 0, skipped: 'no-default-dataset' };
-        const { syncShapes } = await import('@_linked/core');
-        // Explicit target: materialize EVERY registered shape into the app's own
-        // data store regardless of per-shape routing/pins.
-        const { count, orphanScope } = await materializeShapes(appData, syncShapes);
-        return { written: count, orphanScope };
+        const orphanScope = bootOrphanScope();
+        const appName = (_b = this.package) === null || _b === void 0 ? void 0 : _b.name;
+        const vite = (_c = this.config.server) === null || _c === void 0 ? void 0 : _c.vite;
+        const generate = viteShapesTurtleGenerator(vite);
+        // Explicit target: every registered shape goes into the app's own data store, regardless of
+        // per-shape routing or pins.
+        const shapeGraphs = await syncShapeGraphsInto(appData, {
+            core: await loadShapeGraphCore(),
+            orphanScope,
+            isOwn: (name) => name === appName || this.workspacePackageNames.has(name),
+            dev: !!generate,
+            generate,
+            packageDirOf: (name) => this.shapePackageDir(name),
+            appGraph: appShapesGraph(),
+            appPackage: appName,
+        });
+        const written = (_d = shapeGraphs.defaultGraph) !== null && _d !== void 0 ? _d : shapeGraphs.written.length + (((_e = shapeGraphs.appGraph) === null || _e === void 0 ? void 0 : _e.status) === 'written' ? 1 : 0);
+        return { written, orphanScope, shapeGraphs };
+    }
+    /** The root directory of a package with shapes: the app, a discovered linked package, or one the app resolves. */
+    shapePackageDir(name) {
+        var _a, _b;
+        if (name === ((_a = this.package) === null || _a === void 0 ? void 0 : _a.name))
+            return process.cwd();
+        const known = (_b = this.linkedPackageDirs) === null || _b === void 0 ? void 0 : _b.get(name);
+        if (known)
+            return known;
+        const installed = readInstalledPackage(name, process.cwd());
+        return installed === null || installed === void 0 ? void 0 : installed.root;
     }
     /**
      * Boot: sync the registered ontologies into their graphs in app data and run the shape
@@ -726,8 +752,8 @@ let LinkedServer = class LinkedServer extends Shape {
             res.sendStatus(200);
         });
         // The host's control channel (/admin/*), open only when the host started
-        // this process with CN_APP_ADMIN_SECRET; see utils/adminChannel.
-        registerAdminRoutes(this.server, process.env.CN_APP_ADMIN_SECRET, {
+        // this process with APP_ADMIN_SECRET; see utils/adminChannel.
+        registerAdminRoutes(this.server, resolveAdminSecret(), {
             // The host's provisioner starts the app again after it exits.
             exit: () => process.exit(0),
             syncShapes: (request) => this.resyncShapes(request),
@@ -829,6 +855,7 @@ let LinkedServer = class LinkedServer extends Shape {
         // Local workspace packages in this app's dependency tree. These are also
         // imported whole at boot (indexLincdPackage), not only their /backend.
         const allLocalPackages = getLincdPackages();
+        this.workspacePackageNames = new Set(allLocalPackages.map((pkg) => pkg.packageName));
         this.workspacePackageDirs = allLocalPackages.map((pkg) => {
             try {
                 return fsNative.realpathSync(pkg.path);