@factoidal/core 0.15.0 → 0.17.0

package/CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## Unreleased
+
+## 0.17.0 (2026-10-27)
+
+- **Breaking:** `canonicalHash(graph)` now returns the SHA-256 (64 lowercase hex digits) of the RDFC-1.0 canonical N-Quads, as the RDF Dataset Canonicalization specification uses it. Before, it returned the canonical text. Use `canonicalize()` for the text. The engine computes the digest in Lean (new op `datasetCanonicalHash`) (https://github.com/danbri/factoidal/issues/1040, https://github.com/danbri/factoidal-core/pull/127).
+- SPARQL string functions follow SPARQL 1.1 §17.4.3 argument types: STRLEN, SUBSTR, UCASE, LCASE, STRSTARTS, STRENDS, CONTAINS, ENCODE_FOR_URI, MD5 and SHA* give a type error on a number, boolean or IRI argument (MD5/SHA* also on a language-tagged literal). STRSTARTS/STRENDS/CONTAINS check argument compatibility. SUBSTR follows `fn:substring`: `SUBSTR("12345", 0, 3)` is now `"12"` (was `"123"`) (https://github.com/danbri/factoidal/issues/1036, https://github.com/danbri/factoidal-core/pull/126).
+- `factoidal pack --layout sbm11` writes an SBM11 generation: the same files as `l4sbm11-pack`, byte for byte (605 of 605 W3C census inputs). This layout also reads RDF/XML and JSON-LD. `factoidal activate`, `inspect` and `query` read SBM11 stores. `activate` checks the length, SHA-256 and Merkle root of every file before it moves `CURRENT`. `serve` does not read SBM11 stores yet. IBK5 stays the default layout. New wasm ops `storeLayout` and `sbm11Verify` (https://github.com/danbri/factoidal/issues/1009, https://github.com/danbri/factoidal-core/pull/131).
+- Store open in Node is faster: on the londat store (142,314 quads, 1,046 blocks) `openStoreHandle` takes 562 ms, not 1485 ms (median of 7 runs, Apple M1, Node 22). The engine decodes the hexadecimal manifest over bytes, reads `JSON.stringify` arguments in place, verifies artifacts with HACL* SHA-256 in `storeOpen`, and decodes the arguments once per call. What remains is the manifest parse, which grows with the number of blocks; an SBM11 store opens in about 5 ms (https://github.com/danbri/factoidal/issues/1039, https://github.com/danbri/factoidal-core/pull/135).
+- The chunked Turtle reader (the default `l4sbm11-pack` path and the source layer's split read) reads a subject such as `prefix:Atto`, `base:x` or `version:v` as a prefixed name. Before, it took the name for a `PREFIX`/`BASE`/`VERSION` directive and refused the document (QUDT-all-in-one-OWL.ttl) (https://github.com/danbri/factoidal/issues/1028, https://github.com/danbri/factoidal-core/pull/128).
+- RML: a reference to a SPARQL result binding (`formats:SPARQL_Results_CSV`) keeps the literal's datatype when the term map states none. `"5"^^xsd:integer` gives `"5"^^xsd:integer`, not `"5"` (https://github.com/danbri/factoidal/issues/1034, https://github.com/danbri/factoidal-core/pull/129).
+- Tests: 996 Apache Jena ARQ expression tests run as SPARQL on the package (`test/jena-arq-expr.test.mjs`). Each case that differs from Jena has a verdict: a Factoidal defect with an open issue (https://github.com/danbri/factoidal/issues/1041, https://github.com/danbri/factoidal/issues/1042, https://github.com/danbri/factoidal/issues/573), or a Jena extension or difference from the SPARQL specification (https://github.com/danbri/factoidal-core/pull/132). Graph names that are blank nodes: `test/graph-names-876.test.mjs` (https://github.com/danbri/factoidal/issues/876 part 2, fixed in 0.16.0 by https://github.com/danbri/factoidal-core/pull/75; https://github.com/danbri/factoidal-core/pull/130).
+- Tests and release process: no npm test skips when the wasm lacks an op; each test file asserts the ops it needs, so a wasm without an op fails. The `serve --tdb2` test, which needs Apache Jena, moved to `npm run test:jena`. Core PR CI runs `tests/dataflow/run.sh` and `npm test`. `tools/npm-release-check.mjs` checks the version in package.json, package-lock.json and lib/version.mjs, and a dated changelog heading; the publish workflow runs it. `tools/fetch-builds.sh` warns when the installed release was built from a commit whose source differs from HEAD; `--strict` fails (https://github.com/danbri/factoidal/issues/1038, https://github.com/danbri/factoidal-core/pull/124, https://github.com/danbri/factoidal-core/pull/125).
+- LWS: `factoidal lws-serve DIR --authz ASDIR` serves the storage and an authorization server on one port, the issuer the storage root without its trailing slash. The token endpoint and the JWK Set are under the storage root's `.well-known/` (`lws-token`, `lws-jwks`), so no storage resource name is taken from clients; the RFC 8414 metadata is at `/.well-known/lws-configuration` + the issuer's path. The engine decides which requests are the authorization server's (new op `authzOwns`, Lean `L4Factoidal/Authz/Mount.lean`, theorem `endpointOf?_mem`); the storage trusts the issuer with the JWK Set the engine publishes. JS: `listen({ authz: { dir, ... } })` from `@factoidal/core/lws/server`. `authzOpen` takes `mountedOn`, `tokenPath` and `jwksPath`, and answers `resources` and `jwks`.
+- LWS and the authorization server read both forms of their well-known paths: the whole path (a client that reaches the server directly) and the path with the storage root's or issuer's path removed (a reverse proxy that serves the server at a path prefix and removes the prefix, as the router of Touchstone's `targetservers` does). Up to 0.16.0 the protected resource metadata named the wrong resource behind such a proxy (`L4Factoidal.LWS.resourceOfMetadataPath`). Test: `tests/authz/mounted.mjs`. Docker target for third-party testing: `deploy/lws-target/`.
+
+## 0.16.0 (2026-10-26)
+
+- Node settings in the evidence (https://github.com/danbri/factoidal/issues/975). The params of `node()` are the node's settings. With a wasm module that has the new op `pipelineCheckNodes` (Lean `Wasm/Ops/PipelineNodes.lean`, `Dataflow/NodeSettings.lean`), each evidence record carries `settings` (canonical: keys sorted, defaults of absent settings that narrow the contract such as `lenient: false` and `entail: 'none'`, a Dataset value as `{nquadsSha256}`) and `settingsDigest` (`sha256:` and the hex SHA-256 of the canonical text). A setting outside the proved domain of a proved stage (a lenient reader, an evaluator with `entail` other than `none` or with `output`, a key the stage's settings table does not name) makes the node's status `unproved`, with `registryStatus` and `outside` (each setting and the reason) in the record, so `run()` reports `proved: false`. A reader's `format` must name the stage's syntax (`ttl12` selects RDF 1.2 mode, reported as `derived.mode`); an evaluator's query must parse, else `node()` rejects. Design record: `docs/designissues/2026-10-25-pipeline-settings-and-provenance.md`.
+- The output wire of a `shippedEvaluator` node follows its query form: SELECT gives `Table`, ASK `Answer`, CONSTRUCT and DESCRIBE `Dataset` (owner, 2026-10-25: "select give table? Sure"). So reader, CONSTRUCT, CONSTRUCT is well typed, and reader, SELECT, CONSTRUCT is not. Proved: every answer that meets the evaluator contract has the wire of its query form (`Dataflow.evalContract_wire`) (https://github.com/danbri/factoidal/issues/1026). The Lean wire enumeration has a new wire `table`.
+- CSVW is a registered, bound stage: `node('CSVW (tabular data to RDF)', { metadata, mode, base, url })` runs `csvwToRdf` on CSV text and gives a Dataset and an evidence record with status `tested` (the W3C csv2rdf suite; no Lean contract, so `checked: false`) (https://github.com/danbri/factoidal/issues/975). `stages()` marks a stage `bound` only when the engine also reports it runnable.
+- With the wasm of 0.15.0 or older (for example a mismatched local install), records have no settings, the evaluator's wire is the registry's `Answer`, and the CSVW stage stays `missing` and unbound.
+
+- `packSbm11(text, { syntax, base })` packs an RDF document into the files of one SBM11 generation in memory and resolves to a `Map` from file name to bytes (data files, `.merkle` sidecars, `manifest.sbm11`; no `CURRENT`); `mapReader(files)` is a reader over such files for `openSbm11(source, { reader })`. The wasm op is `sbm11PackFiles` (Lean `Storage/Sbm11/PackBytes.lean`), one call per file. The files are byte for byte those of the native `l4sbm11-pack` for the same quads: 23 of 23 files for `test/fixtures/sbm11-tiny`, and all files of 605 of 605 W3C SPARQL 1.0 and 1.1 inputs (`SBM11_PACKFILES=1 tests/sbm11/census.sh`) (https://github.com/danbri/factoidal/issues/1009 stage 2). On the wasm of 0.15.0 or older, `packSbm11` rejects with `code: 'SBM11_NO_PACK_OP'`. The whole document and every file are in memory at once. Not yet: `factoidal pack` on SBM11, and a pack that streams its input.
+- SPARQL: the inverse path `^p` keeps a pair whose object is a literal. SPARQL 1.1 §9.3 and §18.4 make `?x ^p ?y` the same as `?y p ?x`, so `?x` can bind a literal; up to 0.15.0 both evaluators (the in-memory one and the SBM11 id path) dropped those pairs, and `SELECT ?x ?y WHERE { ?x ^<http://e/p> ?y }` over `<http://e/s> <http://e/p> "v" .` gave no row. Apache Jena ARQ 5.6.0 gives the same rows as the fix (https://github.com/danbri/factoidal/issues/1023). Tests: `test/inverse-path-literal-1023.test.mjs` and the `#guard`s in `SPARQL/QueryTests.lean`. The F* tree has the same fix.
+- SPARQL: a `FROM` or `FROM NAMED` IRI that names no graph of the dataset gives a warning of class `dataset-iri-not-found` in the `warnings` member of the query envelope (`class`, `clause` (`FROM` or `FROM NAMED`), `iri`, `message`), and on standard error in the native `l4factoidal query` command. The rows do not change: the clause adds no graph, as before (https://github.com/danbri/factoidal/issues/1032). Lean: `SPARQL.datasetWarnings` beside `applyDataset`, whose merge theorem is unchanged; `#guard`s in `SPARQL/QueryTests.lean` and `Wasm/Ops/Query.lean`. Tests: `test/from-iri-not-found.test.mjs` (plain assertions; https://github.com/danbri/factoidal/issues/1038 records why an expected failure that waits for a rebuild blocked the build of this release).
+- Tests, no engine change: GeoSPARQL `geo:wktLiteral` values made with `STRDT(CONCAT("POINT(", STR(?lon), " ", STR(?lat), ")"), geo:wktLiteral)` in SELECT, in a sub-SELECT that feeds `geof:sfWithin`, and by an INSERT whose stored literal `geof:sfWithin` then tests; from a CSVW column whose datatype object is `{"@id": "http://www.opengis.net/ont/geosparql#wktLiteral", "base": "string"}`; and from an RML object map with `rml:datatype` or a `rml:datatypeMap` with `rml:constant`. `test/wkt-generation.test.mjs` (6 pass on the 0.15.0 wasm) and `#guard`s in `Wasm/Ops/Query.lean` and `Wasm/Ops/Formats.lean`.
+
+- Tests, no engine change: `test/sparql-string-arg-types.test.mjs` holds 6 expected failures for SPARQL 1.1 string functions that accept arguments the specification makes type errors, and for SUBSTR rounding (https://github.com/danbri/factoidal/issues/1036), found by porting Apache Jena ARQ expression tests (`docs/designissues/2026-10-25-jena-style-facade-feasibility.md`).
+
 ## 0.15.0
 
 - `openSbm11(source)` and `registerSbm11Source(source, reader)` are back, with the wasm ops `sbm11Open`, `sbm11Query`, `sbm11Close`: SPARQL over an SBM11 generation (the Shardborough store) in Node and in the browser, reading each file at an offset through a host reader (https://github.com/danbri/factoidal/issues/1009, https://github.com/danbri/factoidal-core/pull/93). Fix: in 0.14.0 `sbm11Open` trapped (wasm `unreachable`) because the C host-read functions had a signature that differs from the generated one; the wasm build now fails on any such mismatch. Not yet: the Merkle check of each read, a generation with a delta log (refused at open), packing SBM11 from npm.

package/README.md

@@ -8,14 +8,15 @@ npm install @factoidal/core
 ```
 
 ```js
-import { parse, csvwToRdf } from "@factoidal/core";
+import { parse, csvwToRdf, canonicalHash } from "@factoidal/core";
 
 const ds = await parse('<http://x/a> <http://x/n> "A" .', { format: "ntriples" });
 const rows = await ds.query("SELECT ?n WHERE { ?s <http://x/n> ?n }");
 console.log(rows[0].get("n").value);              // A
 console.log(ds.toString("turtle"));
 console.log(await ds.serialize({ format: "rdfxml" }));
-console.log(await ds.canonicalize());              // RDFC-1.0
+console.log(await ds.canonicalize());              // RDFC-1.0 canonical N-Quads
+console.log(await canonicalHash(ds));              // its SHA-256, 64 hex digits
 
 const table = await csvwToRdf("id,name\n1,a\n", "", { mode: "minimal", url: "http://x/t.csv" });
 console.log(table.size);                           // 2

package/authz/server.mjs

@@ -22,6 +22,7 @@
 //   authzFetched [handle, { url, status, body, now } | { host, addresses, now }] -> { ok }
 //   authzKeys    [handle, { nonce?, private? }] -> { ok, keys } | { ok, jwks }
 //   authzRotate  [handle, { now, random }]  -> { ok, changed, kid }
+//   authzOwns    [handle, target]          -> { ok, owns }
 //   authzClose   [handle]                   -> { ok }
 //
 // FETCHES. `resolve` names a host: this host resolves it (`dns.lookup`, every
@@ -38,7 +39,7 @@ import { loadEngine } from '../bin/engine.mjs'
 import { storageKeyText } from '../lws/persist.mjs'
 
 /** The ops this host needs from the engine. */
-export const AUTHZ_OPS = ['authzOpen', 'authzStep', 'authzFetched', 'authzKeys', 'authzRotate', 'authzClose']
+export const AUTHZ_OPS = ['authzOpen', 'authzStep', 'authzFetched', 'authzKeys', 'authzRotate', 'authzOwns', 'authzClose']
 
 /** The key file in DIR. */
 export const KEY_FILE = 'authz-keys.json'
@@ -174,12 +175,16 @@ export async function openAuthorizationServer (options = {}) {
   const engine = options.engine !== undefined ? options.engine : await loadEngine()
   const dir = typeof options.dir === 'string' ? options.dir : null
   const config = {
-    issuer: options.issuer,
     resources: Array.isArray(options.resources) ? options.resources : [],
     now: nowSeconds(),
     entropy: randomBase64(64)
   }
-  for (const name of ['alg', 'tokenLifetime', 'leeway', 'rotateAfter', 'docTtl']) {
+  // `mountedOn` (an LWS storage root that shares this issuer) or `issuer`:
+  // passed unread; the engine derives the issuer, the resource and the
+  // endpoint paths from `mountedOn` (L4Factoidal/Authz/Mount.lean).
+  if (typeof options.mountedOn === 'string') config.mountedOn = options.mountedOn
+  else config.issuer = options.issuer
+  for (const name of ['alg', 'tokenLifetime', 'leeway', 'rotateAfter', 'docTtl', 'tokenPath', 'jwksPath', 'audience']) {
     if (options[name] !== undefined) config[name] = options[name]
   }
   if (options.allowHttp === true) config.allowHttp = true
@@ -208,6 +213,15 @@ export async function openAuthorizationServer (options = {}) {
     engine,
     handle,
     issuer: opened.issuer,
+    /** The resources tokens may be issued for, as the engine set them. */
+    resources: opened.resources,
+    /** The published JWK Set when the handle was opened. */
+    jwks: opened.jwks,
+    /** Whether a request target is this server's (the engine decides:
+     *  L4Factoidal.Authz.owns). */
+    owns (target) {
+      return engine.call('authzOwns', [handle, String(target)]).owns === true
+    },
     /**
      * One request.
      * @param {{method: string, target: string, headers: Array<[string, string]>, body: string}} request

package/bin/engine.mjs

@@ -120,13 +120,8 @@ export function loadEngine (name = 'l4factoidal') {
   return promise
 }
 
-const HEX = (() => {
-  const table = new Array(256)
-  for (let byte = 0; byte < 256; byte += 1) {
-    table[byte] = byte.toString(16).padStart(2, '0')
-  }
-  return table
-})()
+const HEX_DIGIT = Uint8Array.from('0123456789abcdef', (c) => c.charCodeAt(0))
+const HEX_DECODER = new TextDecoder()
 
 /**
  * Lowercase hexadecimal of a byte array. This is a transport encoding for
@@ -137,7 +132,14 @@ const HEX = (() => {
  * @returns {string}
  */
 export function hexOfBytes (bytes) {
-  const parts = new Array(bytes.length)
-  for (let index = 0; index < bytes.length; index += 1) parts[index] = HEX[bytes[index]]
-  return parts.join('')
+  // One byte buffer of ASCII digits, decoded once: about 20 times faster
+  // than joining one string per byte for a 753 kB store manifest
+  // (https://github.com/danbri/factoidal/issues/1039).
+  const out = new Uint8Array(bytes.length * 2)
+  for (let index = 0, at = 0; index < bytes.length; index += 1, at += 2) {
+    const byte = bytes[index]
+    out[at] = HEX_DIGIT[byte >> 4]
+    out[at + 1] = HEX_DIGIT[byte & 15]
+  }
+  return HEX_DECODER.decode(out)
 }

package/bin/factoidal.mjs

@@ -35,6 +35,8 @@ import {
   STACK_REMEDY, StoreOperationError, inspectManifest, openStore, planQuery,
   queryStore, stackLimitAdvice, turtleOfNQuads
 } from './store.mjs'
+import { sbm11Api, setDefaultSbm11Reader } from '../lib/sbm11.mjs'
+import { nodeDefaultReaderFactory } from '../lib/sbm11-node.mjs'
 
 const EXIT_OK = 0
 const EXIT_FAILURE = 1
@@ -45,7 +47,12 @@ const EXIT_NOT_WIRED = 3
 // a time, so this is the window that catches one feed per report.
 const FEED_PROGRESS = 65536
 
-const PACK_LAYOUTS = ['ibk3', 'ibk4', 'ibk5']
+const PACK_LAYOUTS = ['ibk3', 'ibk4', 'ibk5', 'sbm11']
+// https://github.com/danbri/factoidal/issues/1009: an SBM11 generation is
+// packed by the wasm op sbm11PackFiles, verified by sbm11Verify and queried
+// by sbm11Open / sbm11Query (formal/lean4/Wasm/Ops/Sbm11*.lean). The op
+// storeLayout says which store a directory holds; this file does not decide
+// it from file names (iron rule 7).
 // The syntaxes the packer reads and the file-name suffixes of each are the
 // engine's tables (op engineTables: packSyntaxes, packSuffixes,
 // packUnsupportedSuffixes), https://github.com/danbri/factoidal/issues/894.
@@ -226,8 +233,11 @@ normally STORE/gen-N. Packing does not activate: run activate next.
 options:
   --layout LAYOUT    ibk5 (quads, wire version 10: RDF 1.2 terms,
                      out-of-line literals, zone maps; default), ibk4
-                     (quads, wire version 9) or ibk3 (triples)
-  --syntax SYNTAX    turtle, trig or nquads; default from the file extension
+                     (quads, wire version 9), ibk3 (triples) or sbm11
+                     (the files l4sbm11-pack writes; the document is read
+                     whole, in memory; serve does not read it yet)
+  --syntax SYNTAX    turtle, trig or nquads, and with sbm11 also rdfxml or
+                     jsonld; default from the file extension
   --chunk-bytes N    Merkle chunk size; default is the engine's
   --batch-bytes N    ibk4 only: source bytes between publication batches;
                      default is the engine's. Blocks are published during
@@ -248,7 +258,10 @@ usage: factoidal activate STORE GENERATION [--json]
 
 Verifies every artifact of GENERATION and every cross-artifact relation,
 then atomically replaces STORE/CURRENT. A generation that fails
-verification never becomes current.`,
+verification never becomes current. For an SBM11 generation the engine
+checks the length, SHA-256 and Merkle root of every file and every
+.merkle sidecar against the manifest (op sbm11Verify); it does not run
+the decoder checks of l4sbm11 activate.`,
 
   update: `factoidal update - apply a SPARQL Update through the delta log
 
@@ -379,8 +392,10 @@ private keys (with d): for a conformance harness that mints test tokens
 
   'lws-serve': `factoidal lws-serve - serve Linked Web Storage 1.0 over HTTP
 
-usage: factoidal lws-serve DIR (--issuer IRI | --auth-config FILE | --insecure-no-auth)
+usage: factoidal lws-serve DIR (--authz DIR | --issuer IRI | --auth-config FILE | --insecure-no-auth)
                          [--owner WEBID] [--root-acl FILE] [--audience IRI]
+                         [--alg ES256|EdDSA] [--token-lifetime SECONDS] [--rotate-after SECONDS]
+                         [--allow-http] [--allow-private-fetch]
                          [--port N] [--host ADDRESS] [--base IRI] [--page-size N]
                          [--paginate-always] [--moves all|data|off]
                          [--no-linkset-put] [--unstable-search-results] [--merkle]
@@ -494,8 +509,21 @@ It protects the file on disk only (formal/lean4/L4Factoidal/LWS/SnapshotSeal.lea
   --no-cors       send no CORS fields; a page on another origin cannot read
                   any answer
 
-Authorization (LWS editor's draft, section Authorization; one of the three
+Authorization (LWS editor's draft, section Authorization; one of the four
 is required):
+  --authz DIR           also serve an authorization server on this port, its
+                        issuer the storage root without the trailing slash
+                        (BASE less "/"): RFC 8414 metadata at
+                        /.well-known/lws-configuration, the token endpoint at
+                        BASE.well-known/lws-token, the JWK Set at
+                        BASE.well-known/lws-jwks, signing keys in DIR (as
+                        authz-serve; --alg, --token-lifetime, --rotate-after,
+                        --allow-http and --allow-private-fetch as there). The
+                        storage trusts it, and any --issuer too. Each request
+                        is the authorization server's when the engine says so
+                        (L4Factoidal/Authz/Mount.lean); every other is the
+                        storage's. A proxy may serve BASE at a path prefix and
+                        remove the prefix: both forms of every path are read.
   --issuer IRI          trust this authorization server; several may be
                         given separated by commas. Its metadata is read from
                         IRI/.well-known/lws-configuration and its keys from
@@ -750,7 +778,7 @@ const VALUE_OPTIONS = {
   serve: new Set(['port', 'host', 'query-path', 'gsp-path', 'file', 'layout', 'tdb2']),
   'lws-serve': new Set(['port', 'host', 'base', 'issuer', 'auth-config', 'owner', 'root-acl',
     'audience', 'page-size', 'moves', 'controller-inbox', 'delivery-limit', 'delivery-window',
-    'redirects-max', 'storage', 'cors-origin']),
+    'redirects-max', 'storage', 'cors-origin', 'authz', 'alg', 'token-lifetime', 'rotate-after']),
   'solid-serve': new Set(['port', 'host', 'base', 'owner', 'issuer', 'auth-config']),
   'solid-client': new Set(['body', 'file', 'content-type', 'interpret']),
   'lws-client': new Set(['body', 'file', 'content-type', 'slug', 'type', 'link', 'if-match',
@@ -931,14 +959,154 @@ function printTable (headers, rows) {
   for (const row of rows) out(line(row))
 }
 
+// --------------------------------------------------------------- SBM11
+
+let sbm11 = null
+
+/** The SBM11 functions of lib/sbm11.mjs over this command's engine, with
+ *  the Node directory reader as the default reader (it moves bytes only). */
+function sbm11Functions () {
+  if (sbm11 === null) {
+    setDefaultSbm11Reader(nodeDefaultReaderFactory())
+    sbm11 = sbm11Api(loadEngine)
+  }
+  return sbm11
+}
+
+/**
+ * The engine's answer (op storeLayout) when `source` holds an SBM11
+ * generation, or names one through CURRENT: `{layout, generation,
+ * manifest}`. null for an IBK store, for a directory the op does not
+ * recognise, and for an engine built before the op; the IBK path then
+ * runs, and reports a missing store in its own words.
+ */
+function sbm11Layout (engine, source) {
+  sbm11Functions()
+  let answer
+  try {
+    answer = engine.call('storeLayout', [source])
+  } catch (_error) {
+    return null
+  }
+  return answer && answer.layout === 'sbm11' ? answer : null
+}
+
+/** The directory a command reads: STORE, or STORE/GENERATION when
+ *  --generation names one. */
+function storeSource (root, named) {
+  return named === null ? root : joinPath(root, named)
+}
+
+/** The engine's words of a refusal, without the loader's prefix. */
+function engineWords (error) {
+  const message = error && error.message ? String(error.message) : String(error)
+  return message.replace(/^l4factoidal:\s*/, '')
+}
+
+async function inspectSbm11 (root, named, layout, options) {
+  let store
+  try {
+    store = await sbm11Functions().openSbm11(storeSource(root, named))
+  } catch (error) {
+    err(`factoidal inspect: ${engineWords(error)}`)
+    return EXIT_FAILURE
+  }
+  const terms = store.terms
+  store.close()
+  const generation = named !== null ? named : layout.generation
+  const generationDir = named !== null || layout.generation === ''
+    ? storeSource(root, named)
+    : joinPath(root, layout.generation)
+  const files = listGeneration(generationDir)
+  let directoryBytes = 0
+  for (const file of files) directoryBytes += file.size
+  if (options.json) {
+    out(JSON.stringify({
+      ok: true,
+      layout: 'sbm11',
+      generation,
+      activated: named === null && layout.generation !== '',
+      manifest: layout.manifest,
+      terms,
+      files: files.length,
+      bytes: directoryBytes
+    }, null, 2))
+    return EXIT_OK
+  }
+  out(`store ${root}`)
+  if (named !== null) out(`generation ${generation} (named on the command line)`)
+  else if (layout.generation !== '') out(`generation ${generation} (activated through CURRENT)`)
+  else out('generation directory (no CURRENT)')
+  out(`manifest ${layout.manifest}`)
+  out('layout sbm11')
+  out(plural(terms, 'term'))
+  out(`generation directory holds ${files.length} files, ${directoryBytes} bytes`)
+  return EXIT_OK
+}
+
+/**
+ * Answer a query on an SBM11 store (ops sbm11Open / sbm11Query /
+ * sbm11Close) and print it as `query` prints an IBK answer. The engine
+ * answers SPARQL Query Results JSON for SELECT and ASK and N-Triples for
+ * CONSTRUCT and DESCRIBE; this only chooses which renderer prints it.
+ */
+async function querySbm11 (engine, root, named, sparql, format, limit, options) {
+  const quiet = options.quiet === true
+  if (options.explain === true) {
+    err('factoidal query: --explain prints the plan of an IBK store; there is ' +
+      'no plan operation for an SBM11 store yet ' +
+      '(https://github.com/danbri/factoidal/issues/1009)')
+    return EXIT_FAILURE
+  }
+  let answer
+  try {
+    const store = await sbm11Functions().openSbm11(storeSource(root, named))
+    try {
+      const got = store.query(sparql)
+      // read the bytes while the handle is open
+      answer = { path: got.path, format: got.format, reads: got.reads,
+        bytesRead: got.bytesRead, text: got.result }
+    } finally {
+      store.close()
+    }
+  } catch (error) {
+    err(`factoidal query: ${engineWords(error)}`)
+    if (isStackOverflow(error)) {
+      for (const line of stackLimitAdvice(STACK_REMEDY.query)) err(line)
+    }
+    return EXIT_FAILURE
+  }
+  if (!quiet) {
+    err(`${answer.path}, ${plural(answer.reads, 'read')}, ${answer.bytesRead} bytes read`)
+  }
+  let result
+  if (answer.format === 'srj') {
+    const document = JSON.parse(answer.text)
+    if (typeof document.boolean === 'boolean') {
+      if (format === 'json') {
+        out(JSON.stringify(document, null, 2))
+        return EXIT_OK
+      }
+      result = { kind: 'ask', boolean: document.boolean }
+    } else {
+      result = { kind: 'select', srj: document }
+    }
+  } else {
+    result = { kind: 'construct', nquads: answer.text }
+  }
+  return renderQueryResult(engine, result, format, limit, quiet)
+}
+
 // ------------------------------------------------------------- inspect
 
 async function commandInspect (positional, options) {
   if (positional.length !== 1) throw new UsageError('inspect needs exactly one STORE')
   const root = positional[0]
   const named = typeof options.generation === 'string' ? options.generation : null
-  const store = openStore(root, named)
   const engine = await loadEngine()
+  const layout = sbm11Layout(engine, storeSource(root, named))
+  if (layout !== null) return inspectSbm11(root, named, layout, options)
+  const store = openStore(root, named)
   const envelope = inspectManifest(engine, store)
 
   if (options.json) {
@@ -1102,8 +1270,11 @@ async function commandQuery (positional, options) {
   const quiet = options.quiet === true
 
   const named = typeof options.generation === 'string' ? options.generation : null
-  const store = openStore(root, named)
   const engine = await loadEngine()
+  if (sbm11Layout(engine, storeSource(root, named)) !== null) {
+    return querySbm11(engine, root, named, sparql, format, limit, options)
+  }
+  const store = openStore(root, named)
 
   if (options.explain === true) {
     let plan
@@ -1208,9 +1379,15 @@ function readHead (path, length) {
 /** The syntax tag for an input: --syntax when given, else the engine's
  *  suffix table (op engineTables), else the engine's content detection
  *  (op detectFormat) on the first 64 KiB of the file. */
-function packSyntax (engine, input, options) {
+function packSyntax (engine, input, options, layout) {
   const tables = engine.call('engineTables', []).tables
-  const syntaxes = tables.packSyntaxes
+  // The SBM11 pack reads the whole document (op sbm11PackFiles), so it also
+  // reads the syntaxes of packWholeSuffixes (RDF/XML, JSON-LD).
+  const whole = layout === 'sbm11' ? (tables.packWholeSuffixes ?? {}) : {}
+  const syntaxes = tables.packSyntaxes.slice()
+  for (const syntax of Object.values(whole)) {
+    if (syntaxes.indexOf(syntax) < 0) syntaxes.push(syntax)
+  }
   if (typeof options.syntax === 'string') {
     const syntax = options.syntax.toLowerCase()
     if (syntaxes.indexOf(syntax) >= 0) return syntax
@@ -1220,11 +1397,14 @@ function packSyntax (engine, input, options) {
   for (const [suffix, syntax] of Object.entries(tables.packSuffixes)) {
     if (lower.endsWith(suffix)) return syntax
   }
+  for (const [suffix, syntax] of Object.entries(whole)) {
+    if (lower.endsWith(suffix)) return syntax
+  }
   for (const [suffix, name] of Object.entries(tables.packUnsupportedSuffixes)) {
     if (lower.endsWith(suffix)) {
       throw new UsageError(
-        `pack does not read ${name}. The packer's streaming fold reads ` +
-        `${syntaxes.join(', ')} only. Convert the file first, for ` +
+        `pack does not read ${name}. The packer reads ` +
+        `${syntaxes.join(', ')} only${layout === 'sbm11' ? '' : ' (--layout sbm11 also reads rdfxml and jsonld)'}. Convert the file first, for ` +
         "example with: factoidal parse FILE --out nquads")
     }
   }
@@ -1253,8 +1433,11 @@ function packSyntax (engine, input, options) {
  * `--base ''` asks for no base, which turns a relative IRI into a parse
  * error rather than a silently different term.
  */
-function packBase (input, options) {
+function packBase (input, options, layout) {
   if (typeof options.base === 'string') return options.base
+  // `l4sbm11-pack` uses no base unless --base names one; the same default
+  // gives the same bytes.
+  if (layout === 'sbm11') return ''
   const absolute = input.startsWith('/') ? input : joinPath(currentDirectory(), input)
   return 'file://' + absolute
 }
@@ -1275,8 +1458,8 @@ function packLayout (options) {
 async function commandPack (positional, options) {
   if (positional.length !== 2) throw new UsageError('pack needs INPUT and OUTPUT')
   const [input, output] = positional
-  const syntax = packSyntax(await loadEngine(), input, options)
   const layout = packLayout(options)
+  const syntax = packSyntax(await loadEngine(), input, options, layout)
   // The pack fold needs a bigger call stack than either runtime gives by
   // default (https://github.com/danbri/factoidal/issues/649). Under Node
   // the work runs on a worker thread with a raised stack; under Deno the
@@ -1291,8 +1474,10 @@ async function commandPack (positional, options) {
   let answer
   try {
     answer = await runPack(
-      { kind: 'pack', input, output, syntax, layout, base: packBase(input, options),
-        batchBytes: options['batch-bytes'] },
+      layout === 'sbm11'
+        ? { kind: 'packSbm11', input, output, syntax, base: packBase(input, options, layout) }
+        : { kind: 'pack', input, output, syntax, layout, base: packBase(input, options, layout),
+            batchBytes: options['batch-bytes'] },
       quiet
         ? undefined
         : (progress) => {
@@ -1331,7 +1516,7 @@ async function commandPack (positional, options) {
     return EXIT_OK
   }
   out(`packed ${report.bytesRead} bytes of ${syntax} into ${output}`)
-  out(`${report.written.length} artifacts, ${report.bytesWritten} bytes, layout ${layout}`)
+  out(`${report.written.length} ${layout === 'sbm11' ? 'files' : 'artifacts'}, ${report.bytesWritten} bytes, layout ${layout}`)
   if (typeof report.rows === 'number') out(`${plural(report.rows, 'row')}`)
   out(`activate it with: factoidal activate ${dirOf(output)} ${nameOf(output)}`)
   return EXIT_OK
@@ -1382,6 +1567,10 @@ async function commandActivate (positional, options) {
   const host = { worker: options['no-worker'] !== true }
   const reexec = await denoReexec(host)
   if (reexec !== null) return reexec
+  const engine = await loadEngine()
+  if (sbm11Layout(engine, joinPath(root, generation)) !== null) {
+    return activateSbm11(engine, root, generation, options)
+  }
   let answer
   try {
     answer = await runPack({ kind: 'activate', root, generation }, undefined, host)
@@ -1425,6 +1614,32 @@ async function commandActivate (positional, options) {
   return EXIT_OK
 }
 
+/**
+ * Activate an SBM11 generation: the engine checks every file against the
+ * manifest (op sbm11Verify), and only after its "ok" does CURRENT move.
+ */
+function activateSbm11 (engine, root, generation, options) {
+  let verdict
+  try {
+    verdict = engine.call('sbm11Verify', [joinPath(root, generation)])
+  } catch (error) {
+    err(`factoidal activate: ${engineWords(error)}`)
+    err('The generation is NOT activated; CURRENT is unchanged.')
+    return EXIT_FAILURE
+  }
+  const synced = atomicReplace(joinPath(root, 'CURRENT'), new TextEncoder().encode(generation))
+  if (options.json === true) {
+    out(JSON.stringify({ ...verdict, layout: 'sbm11', generation, directorySynced: synced }, null, 2))
+    return EXIT_OK
+  }
+  out(`activated ${generation}: ${verdict.artifacts} artifacts verified (${verdict.files} files, ${verdict.bytes} bytes), layout sbm11`)
+  if (!synced) {
+    err('CURRENT was replaced, but the directory entry was not synced; a ' +
+        'crash now could lose the pointer update.')
+  }
+  return EXIT_OK
+}
+
 function commandUpdate (positional, options) {
   if (positional.length < 1) throw new UsageError('update needs a STORE')
   const text = positional.length > 1
@@ -1529,6 +1744,10 @@ async function commandServe (positional, options) {
     const { join } = await import('node:path')
     const root = mkdtempSync(join(tmpdir(), 'factoidal-serve-'))
     const packOptions = { quiet: true, layout: packLayout(options) }
+    if (packOptions.layout === 'sbm11') {
+      throw new UsageError('serve --file packs a store that accepts updates; --layout sbm11 ' +
+        'packs a store that serve does not read yet. Use ibk3, ibk4 or ibk5.')
+    }
     let rc = await commandPack([options.file, join(root, 'gen-1')], packOptions)
     if (rc !== 0) return rc
     rc = await commandActivate([root, 'gen-1'], { quiet: true })
@@ -1549,6 +1768,12 @@ async function commandServe (positional, options) {
     options = { ...options, 'query-path': `${dsName}/sparql`, 'gsp-path': `${dsName}/data` }
   }
   if (positional.length !== 1) throw new UsageError('serve needs one STORE')
+  if (!tdb2 && sbm11Layout(await loadEngine(), positional[0]) !== null) {
+    err(`factoidal serve: ${positional[0]} is an SBM11 store, and serve reads IBK ` +
+      'stores only (the storeServe operations). Query it with: factoidal query ' +
+      `${positional[0]} QUERY (https://github.com/danbri/factoidal/issues/1009)`)
+    return EXIT_FAILURE
+  }
   const { listen, listenOnWorker, ServeHostError } = await import('./serve.mjs')
   const serveOptions = {
     port: servePort(options, 3030),
@@ -1590,12 +1815,25 @@ async function commandLwsServe (positional, options) {
   const insecure = options['insecure-no-auth'] === true
   const issuerOption = typeof options.issuer === 'string' ? options.issuer : null
   const authFile = typeof options['auth-config'] === 'string' ? options['auth-config'] : null
-  if (!insecure && issuerOption === null && authFile === null) {
-    throw new UsageError('lws-serve needs --issuer IRI or --auth-config FILE, ' +
+  const authzDir = typeof options.authz === 'string' ? options.authz : null
+  if (!insecure && issuerOption === null && authFile === null && authzDir === null) {
+    throw new UsageError('lws-serve needs --authz DIR, --issuer IRI or --auth-config FILE, ' +
       'or --insecure-no-auth for development')
   }
-  if (insecure && (issuerOption !== null || authFile !== null)) {
-    throw new UsageError('--insecure-no-auth cannot be combined with --issuer or --auth-config')
+  if (insecure && (issuerOption !== null || authFile !== null || authzDir !== null)) {
+    throw new UsageError('--insecure-no-auth cannot be combined with --authz, --issuer or --auth-config')
+  }
+  // --authz DIR: an authorization server on the same port, its issuer the
+  // storage root (L4Factoidal/Authz/Mount.lean). Its options are passed
+  // unread; the engine checks them (Wasm/Ops/Authz.lean).
+  const authz = authzDir === null ? undefined : {
+    dir: authzDir,
+    ...(typeof options.alg === 'string' ? { alg: options.alg } : {}),
+    ...(options['token-lifetime'] !== undefined ? { tokenLifetime: natOption('token-lifetime', options['token-lifetime']) } : {}),
+    ...(options['rotate-after'] !== undefined ? { rotateAfter: natOption('rotate-after', options['rotate-after']) } : {}),
+    ...(options['allow-http'] === true ? { allowHttp: true } : {}),
+    ...(options['allow-private-fetch'] === true ? { allowPrivateFetch: true } : {}),
+    ...corsConfig(options)
   }
   let auth
   if (authFile !== null) {
@@ -1628,6 +1866,7 @@ async function commandLwsServe (positional, options) {
       ...(typeof options.storage === 'string' ? { storage: options.storage } : {}),
       ...corsConfig(options),
       ...(insecure ? { insecureNoAuth: true } : { auth }),
+      ...(authz !== undefined ? { authz } : {}),
       ...(typeof options.owner === 'string' ? { owner: options.owner } : {}),
       ...(typeof options['root-acl'] === 'string'
         ? { rootAcl: new TextDecoder('utf-8', { fatal: true }).decode(readWhole(options['root-acl'])) }
@@ -1651,6 +1890,9 @@ async function commandLwsServe (positional, options) {
   }
   if (!quiet) {
     err(`lws-serve: listening on ${running.origin} for ${positional[0]}`)
+    if (running.authz() !== null) err(`lws-serve: authorization server ${running.authz().issuer} (keys in ${authzDir})`)
+    if (options['allow-http'] === true) err('lws-serve: WARNING: --allow-http: http subject identifiers are accepted')
+    if (options['allow-private-fetch'] === true) err('lws-serve: WARNING: --allow-private-fetch: private addresses may be fetched')
     if (insecure) err('lws-serve: WARNING: --insecure-no-auth: every request is served without authorization')
   }
   out(running.origin)