@factoidal/core 0.16.0 → 0.17.0

package/CHANGELOG.md

@@ -2,6 +2,19 @@
 
 ## Unreleased
 
+## 0.17.0 (2026-10-27)
+
+- **Breaking:** `canonicalHash(graph)` now returns the SHA-256 (64 lowercase hex digits) of the RDFC-1.0 canonical N-Quads, as the RDF Dataset Canonicalization specification uses it. Before, it returned the canonical text. Use `canonicalize()` for the text. The engine computes the digest in Lean (new op `datasetCanonicalHash`) (https://github.com/danbri/factoidal/issues/1040, https://github.com/danbri/factoidal-core/pull/127).
+- SPARQL string functions follow SPARQL 1.1 §17.4.3 argument types: STRLEN, SUBSTR, UCASE, LCASE, STRSTARTS, STRENDS, CONTAINS, ENCODE_FOR_URI, MD5 and SHA* give a type error on a number, boolean or IRI argument (MD5/SHA* also on a language-tagged literal). STRSTARTS/STRENDS/CONTAINS check argument compatibility. SUBSTR follows `fn:substring`: `SUBSTR("12345", 0, 3)` is now `"12"` (was `"123"`) (https://github.com/danbri/factoidal/issues/1036, https://github.com/danbri/factoidal-core/pull/126).
+- `factoidal pack --layout sbm11` writes an SBM11 generation: the same files as `l4sbm11-pack`, byte for byte (605 of 605 W3C census inputs). This layout also reads RDF/XML and JSON-LD. `factoidal activate`, `inspect` and `query` read SBM11 stores. `activate` checks the length, SHA-256 and Merkle root of every file before it moves `CURRENT`. `serve` does not read SBM11 stores yet. IBK5 stays the default layout. New wasm ops `storeLayout` and `sbm11Verify` (https://github.com/danbri/factoidal/issues/1009, https://github.com/danbri/factoidal-core/pull/131).
+- Store open in Node is faster: on the londat store (142,314 quads, 1,046 blocks) `openStoreHandle` takes 562 ms, not 1485 ms (median of 7 runs, Apple M1, Node 22). The engine decodes the hexadecimal manifest over bytes, reads `JSON.stringify` arguments in place, verifies artifacts with HACL* SHA-256 in `storeOpen`, and decodes the arguments once per call. What remains is the manifest parse, which grows with the number of blocks; an SBM11 store opens in about 5 ms (https://github.com/danbri/factoidal/issues/1039, https://github.com/danbri/factoidal-core/pull/135).
+- The chunked Turtle reader (the default `l4sbm11-pack` path and the source layer's split read) reads a subject such as `prefix:Atto`, `base:x` or `version:v` as a prefixed name. Before, it took the name for a `PREFIX`/`BASE`/`VERSION` directive and refused the document (QUDT-all-in-one-OWL.ttl) (https://github.com/danbri/factoidal/issues/1028, https://github.com/danbri/factoidal-core/pull/128).
+- RML: a reference to a SPARQL result binding (`formats:SPARQL_Results_CSV`) keeps the literal's datatype when the term map states none. `"5"^^xsd:integer` gives `"5"^^xsd:integer`, not `"5"` (https://github.com/danbri/factoidal/issues/1034, https://github.com/danbri/factoidal-core/pull/129).
+- Tests: 996 Apache Jena ARQ expression tests run as SPARQL on the package (`test/jena-arq-expr.test.mjs`). Each case that differs from Jena has a verdict: a Factoidal defect with an open issue (https://github.com/danbri/factoidal/issues/1041, https://github.com/danbri/factoidal/issues/1042, https://github.com/danbri/factoidal/issues/573), or a Jena extension or difference from the SPARQL specification (https://github.com/danbri/factoidal-core/pull/132). Graph names that are blank nodes: `test/graph-names-876.test.mjs` (https://github.com/danbri/factoidal/issues/876 part 2, fixed in 0.16.0 by https://github.com/danbri/factoidal-core/pull/75; https://github.com/danbri/factoidal-core/pull/130).
+- Tests and release process: no npm test skips when the wasm lacks an op; each test file asserts the ops it needs, so a wasm without an op fails. The `serve --tdb2` test, which needs Apache Jena, moved to `npm run test:jena`. Core PR CI runs `tests/dataflow/run.sh` and `npm test`. `tools/npm-release-check.mjs` checks the version in package.json, package-lock.json and lib/version.mjs, and a dated changelog heading; the publish workflow runs it. `tools/fetch-builds.sh` warns when the installed release was built from a commit whose source differs from HEAD; `--strict` fails (https://github.com/danbri/factoidal/issues/1038, https://github.com/danbri/factoidal-core/pull/124, https://github.com/danbri/factoidal-core/pull/125).
+- LWS: `factoidal lws-serve DIR --authz ASDIR` serves the storage and an authorization server on one port, the issuer the storage root without its trailing slash. The token endpoint and the JWK Set are under the storage root's `.well-known/` (`lws-token`, `lws-jwks`), so no storage resource name is taken from clients; the RFC 8414 metadata is at `/.well-known/lws-configuration` + the issuer's path. The engine decides which requests are the authorization server's (new op `authzOwns`, Lean `L4Factoidal/Authz/Mount.lean`, theorem `endpointOf?_mem`); the storage trusts the issuer with the JWK Set the engine publishes. JS: `listen({ authz: { dir, ... } })` from `@factoidal/core/lws/server`. `authzOpen` takes `mountedOn`, `tokenPath` and `jwksPath`, and answers `resources` and `jwks`.
+- LWS and the authorization server read both forms of their well-known paths: the whole path (a client that reaches the server directly) and the path with the storage root's or issuer's path removed (a reverse proxy that serves the server at a path prefix and removes the prefix, as the router of Touchstone's `targetservers` does). Up to 0.16.0 the protected resource metadata named the wrong resource behind such a proxy (`L4Factoidal.LWS.resourceOfMetadataPath`). Test: `tests/authz/mounted.mjs`. Docker target for third-party testing: `deploy/lws-target/`.
+
 ## 0.16.0 (2026-10-26)
 
 - Node settings in the evidence (https://github.com/danbri/factoidal/issues/975). The params of `node()` are the node's settings. With a wasm module that has the new op `pipelineCheckNodes` (Lean `Wasm/Ops/PipelineNodes.lean`, `Dataflow/NodeSettings.lean`), each evidence record carries `settings` (canonical: keys sorted, defaults of absent settings that narrow the contract such as `lenient: false` and `entail: 'none'`, a Dataset value as `{nquadsSha256}`) and `settingsDigest` (`sha256:` and the hex SHA-256 of the canonical text). A setting outside the proved domain of a proved stage (a lenient reader, an evaluator with `entail` other than `none` or with `output`, a key the stage's settings table does not name) makes the node's status `unproved`, with `registryStatus` and `outside` (each setting and the reason) in the record, so `run()` reports `proved: false`. A reader's `format` must name the stage's syntax (`ttl12` selects RDF 1.2 mode, reported as `derived.mode`); an evaluator's query must parse, else `node()` rejects. Design record: `docs/designissues/2026-10-25-pipeline-settings-and-provenance.md`.

package/README.md

@@ -8,14 +8,15 @@ npm install @factoidal/core
 ```
 
 ```js
-import { parse, csvwToRdf } from "@factoidal/core";
+import { parse, csvwToRdf, canonicalHash } from "@factoidal/core";
 
 const ds = await parse('<http://x/a> <http://x/n> "A" .', { format: "ntriples" });
 const rows = await ds.query("SELECT ?n WHERE { ?s <http://x/n> ?n }");
 console.log(rows[0].get("n").value);              // A
 console.log(ds.toString("turtle"));
 console.log(await ds.serialize({ format: "rdfxml" }));
-console.log(await ds.canonicalize());              // RDFC-1.0
+console.log(await ds.canonicalize());              // RDFC-1.0 canonical N-Quads
+console.log(await canonicalHash(ds));              // its SHA-256, 64 hex digits
 
 const table = await csvwToRdf("id,name\n1,a\n", "", { mode: "minimal", url: "http://x/t.csv" });
 console.log(table.size);                           // 2

package/authz/server.mjs

@@ -22,6 +22,7 @@
 //   authzFetched [handle, { url, status, body, now } | { host, addresses, now }] -> { ok }
 //   authzKeys    [handle, { nonce?, private? }] -> { ok, keys } | { ok, jwks }
 //   authzRotate  [handle, { now, random }]  -> { ok, changed, kid }
+//   authzOwns    [handle, target]          -> { ok, owns }
 //   authzClose   [handle]                   -> { ok }
 //
 // FETCHES. `resolve` names a host: this host resolves it (`dns.lookup`, every
@@ -38,7 +39,7 @@ import { loadEngine } from '../bin/engine.mjs'
 import { storageKeyText } from '../lws/persist.mjs'
 
 /** The ops this host needs from the engine. */
-export const AUTHZ_OPS = ['authzOpen', 'authzStep', 'authzFetched', 'authzKeys', 'authzRotate', 'authzClose']
+export const AUTHZ_OPS = ['authzOpen', 'authzStep', 'authzFetched', 'authzKeys', 'authzRotate', 'authzOwns', 'authzClose']
 
 /** The key file in DIR. */
 export const KEY_FILE = 'authz-keys.json'
@@ -174,12 +175,16 @@ export async function openAuthorizationServer (options = {}) {
   const engine = options.engine !== undefined ? options.engine : await loadEngine()
   const dir = typeof options.dir === 'string' ? options.dir : null
   const config = {
-    issuer: options.issuer,
     resources: Array.isArray(options.resources) ? options.resources : [],
     now: nowSeconds(),
     entropy: randomBase64(64)
   }
-  for (const name of ['alg', 'tokenLifetime', 'leeway', 'rotateAfter', 'docTtl']) {
+  // `mountedOn` (an LWS storage root that shares this issuer) or `issuer`:
+  // passed unread; the engine derives the issuer, the resource and the
+  // endpoint paths from `mountedOn` (L4Factoidal/Authz/Mount.lean).
+  if (typeof options.mountedOn === 'string') config.mountedOn = options.mountedOn
+  else config.issuer = options.issuer
+  for (const name of ['alg', 'tokenLifetime', 'leeway', 'rotateAfter', 'docTtl', 'tokenPath', 'jwksPath', 'audience']) {
     if (options[name] !== undefined) config[name] = options[name]
   }
   if (options.allowHttp === true) config.allowHttp = true
@@ -208,6 +213,15 @@ export async function openAuthorizationServer (options = {}) {
     engine,
     handle,
     issuer: opened.issuer,
+    /** The resources tokens may be issued for, as the engine set them. */
+    resources: opened.resources,
+    /** The published JWK Set when the handle was opened. */
+    jwks: opened.jwks,
+    /** Whether a request target is this server's (the engine decides:
+     *  L4Factoidal.Authz.owns). */
+    owns (target) {
+      return engine.call('authzOwns', [handle, String(target)]).owns === true
+    },
     /**
      * One request.
      * @param {{method: string, target: string, headers: Array<[string, string]>, body: string}} request

package/bin/engine.mjs

@@ -120,13 +120,8 @@ export function loadEngine (name = 'l4factoidal') {
   return promise
 }
 
-const HEX = (() => {
-  const table = new Array(256)
-  for (let byte = 0; byte < 256; byte += 1) {
-    table[byte] = byte.toString(16).padStart(2, '0')
-  }
-  return table
-})()
+const HEX_DIGIT = Uint8Array.from('0123456789abcdef', (c) => c.charCodeAt(0))
+const HEX_DECODER = new TextDecoder()
 
 /**
  * Lowercase hexadecimal of a byte array. This is a transport encoding for
@@ -137,7 +132,14 @@ const HEX = (() => {
  * @returns {string}
  */
 export function hexOfBytes (bytes) {
-  const parts = new Array(bytes.length)
-  for (let index = 0; index < bytes.length; index += 1) parts[index] = HEX[bytes[index]]
-  return parts.join('')
+  // One byte buffer of ASCII digits, decoded once: about 20 times faster
+  // than joining one string per byte for a 753 kB store manifest
+  // (https://github.com/danbri/factoidal/issues/1039).
+  const out = new Uint8Array(bytes.length * 2)
+  for (let index = 0, at = 0; index < bytes.length; index += 1, at += 2) {
+    const byte = bytes[index]
+    out[at] = HEX_DIGIT[byte >> 4]
+    out[at + 1] = HEX_DIGIT[byte & 15]
+  }
+  return HEX_DECODER.decode(out)
 }

package/bin/factoidal.mjs

@@ -35,6 +35,8 @@ import {
   STACK_REMEDY, StoreOperationError, inspectManifest, openStore, planQuery,
   queryStore, stackLimitAdvice, turtleOfNQuads
 } from './store.mjs'
+import { sbm11Api, setDefaultSbm11Reader } from '../lib/sbm11.mjs'
+import { nodeDefaultReaderFactory } from '../lib/sbm11-node.mjs'
 
 const EXIT_OK = 0
 const EXIT_FAILURE = 1
@@ -45,7 +47,12 @@ const EXIT_NOT_WIRED = 3
 // a time, so this is the window that catches one feed per report.
 const FEED_PROGRESS = 65536
 
-const PACK_LAYOUTS = ['ibk3', 'ibk4', 'ibk5']
+const PACK_LAYOUTS = ['ibk3', 'ibk4', 'ibk5', 'sbm11']
+// https://github.com/danbri/factoidal/issues/1009: an SBM11 generation is
+// packed by the wasm op sbm11PackFiles, verified by sbm11Verify and queried
+// by sbm11Open / sbm11Query (formal/lean4/Wasm/Ops/Sbm11*.lean). The op
+// storeLayout says which store a directory holds; this file does not decide
+// it from file names (iron rule 7).
 // The syntaxes the packer reads and the file-name suffixes of each are the
 // engine's tables (op engineTables: packSyntaxes, packSuffixes,
 // packUnsupportedSuffixes), https://github.com/danbri/factoidal/issues/894.
@@ -226,8 +233,11 @@ normally STORE/gen-N. Packing does not activate: run activate next.
 options:
   --layout LAYOUT    ibk5 (quads, wire version 10: RDF 1.2 terms,
                      out-of-line literals, zone maps; default), ibk4
-                     (quads, wire version 9) or ibk3 (triples)
-  --syntax SYNTAX    turtle, trig or nquads; default from the file extension
+                     (quads, wire version 9), ibk3 (triples) or sbm11
+                     (the files l4sbm11-pack writes; the document is read
+                     whole, in memory; serve does not read it yet)
+  --syntax SYNTAX    turtle, trig or nquads, and with sbm11 also rdfxml or
+                     jsonld; default from the file extension
   --chunk-bytes N    Merkle chunk size; default is the engine's
   --batch-bytes N    ibk4 only: source bytes between publication batches;
                      default is the engine's. Blocks are published during
@@ -248,7 +258,10 @@ usage: factoidal activate STORE GENERATION [--json]
 
 Verifies every artifact of GENERATION and every cross-artifact relation,
 then atomically replaces STORE/CURRENT. A generation that fails
-verification never becomes current.`,
+verification never becomes current. For an SBM11 generation the engine
+checks the length, SHA-256 and Merkle root of every file and every
+.merkle sidecar against the manifest (op sbm11Verify); it does not run
+the decoder checks of l4sbm11 activate.`,
 
   update: `factoidal update - apply a SPARQL Update through the delta log
 
@@ -379,8 +392,10 @@ private keys (with d): for a conformance harness that mints test tokens
 
   'lws-serve': `factoidal lws-serve - serve Linked Web Storage 1.0 over HTTP
 
-usage: factoidal lws-serve DIR (--issuer IRI | --auth-config FILE | --insecure-no-auth)
+usage: factoidal lws-serve DIR (--authz DIR | --issuer IRI | --auth-config FILE | --insecure-no-auth)
                          [--owner WEBID] [--root-acl FILE] [--audience IRI]
+                         [--alg ES256|EdDSA] [--token-lifetime SECONDS] [--rotate-after SECONDS]
+                         [--allow-http] [--allow-private-fetch]
                          [--port N] [--host ADDRESS] [--base IRI] [--page-size N]
                          [--paginate-always] [--moves all|data|off]
                          [--no-linkset-put] [--unstable-search-results] [--merkle]
@@ -494,8 +509,21 @@ It protects the file on disk only (formal/lean4/L4Factoidal/LWS/SnapshotSeal.lea
   --no-cors       send no CORS fields; a page on another origin cannot read
                   any answer
 
-Authorization (LWS editor's draft, section Authorization; one of the three
+Authorization (LWS editor's draft, section Authorization; one of the four
 is required):
+  --authz DIR           also serve an authorization server on this port, its
+                        issuer the storage root without the trailing slash
+                        (BASE less "/"): RFC 8414 metadata at
+                        /.well-known/lws-configuration, the token endpoint at
+                        BASE.well-known/lws-token, the JWK Set at
+                        BASE.well-known/lws-jwks, signing keys in DIR (as
+                        authz-serve; --alg, --token-lifetime, --rotate-after,
+                        --allow-http and --allow-private-fetch as there). The
+                        storage trusts it, and any --issuer too. Each request
+                        is the authorization server's when the engine says so
+                        (L4Factoidal/Authz/Mount.lean); every other is the
+                        storage's. A proxy may serve BASE at a path prefix and
+                        remove the prefix: both forms of every path are read.
   --issuer IRI          trust this authorization server; several may be
                         given separated by commas. Its metadata is read from
                         IRI/.well-known/lws-configuration and its keys from
@@ -750,7 +778,7 @@ const VALUE_OPTIONS = {
   serve: new Set(['port', 'host', 'query-path', 'gsp-path', 'file', 'layout', 'tdb2']),
   'lws-serve': new Set(['port', 'host', 'base', 'issuer', 'auth-config', 'owner', 'root-acl',
     'audience', 'page-size', 'moves', 'controller-inbox', 'delivery-limit', 'delivery-window',
-    'redirects-max', 'storage', 'cors-origin']),
+    'redirects-max', 'storage', 'cors-origin', 'authz', 'alg', 'token-lifetime', 'rotate-after']),
   'solid-serve': new Set(['port', 'host', 'base', 'owner', 'issuer', 'auth-config']),
   'solid-client': new Set(['body', 'file', 'content-type', 'interpret']),
   'lws-client': new Set(['body', 'file', 'content-type', 'slug', 'type', 'link', 'if-match',
@@ -931,14 +959,154 @@ function printTable (headers, rows) {
   for (const row of rows) out(line(row))
 }
 
+// --------------------------------------------------------------- SBM11
+
+let sbm11 = null
+
+/** The SBM11 functions of lib/sbm11.mjs over this command's engine, with
+ *  the Node directory reader as the default reader (it moves bytes only). */
+function sbm11Functions () {
+  if (sbm11 === null) {
+    setDefaultSbm11Reader(nodeDefaultReaderFactory())
+    sbm11 = sbm11Api(loadEngine)
+  }
+  return sbm11
+}
+
+/**
+ * The engine's answer (op storeLayout) when `source` holds an SBM11
+ * generation, or names one through CURRENT: `{layout, generation,
+ * manifest}`. null for an IBK store, for a directory the op does not
+ * recognise, and for an engine built before the op; the IBK path then
+ * runs, and reports a missing store in its own words.
+ */
+function sbm11Layout (engine, source) {
+  sbm11Functions()
+  let answer
+  try {
+    answer = engine.call('storeLayout', [source])
+  } catch (_error) {
+    return null
+  }
+  return answer && answer.layout === 'sbm11' ? answer : null
+}
+
+/** The directory a command reads: STORE, or STORE/GENERATION when
+ *  --generation names one. */
+function storeSource (root, named) {
+  return named === null ? root : joinPath(root, named)
+}
+
+/** The engine's words of a refusal, without the loader's prefix. */
+function engineWords (error) {
+  const message = error && error.message ? String(error.message) : String(error)
+  return message.replace(/^l4factoidal:\s*/, '')
+}
+
+async function inspectSbm11 (root, named, layout, options) {
+  let store
+  try {
+    store = await sbm11Functions().openSbm11(storeSource(root, named))
+  } catch (error) {
+    err(`factoidal inspect: ${engineWords(error)}`)
+    return EXIT_FAILURE
+  }
+  const terms = store.terms
+  store.close()
+  const generation = named !== null ? named : layout.generation
+  const generationDir = named !== null || layout.generation === ''
+    ? storeSource(root, named)
+    : joinPath(root, layout.generation)
+  const files = listGeneration(generationDir)
+  let directoryBytes = 0
+  for (const file of files) directoryBytes += file.size
+  if (options.json) {
+    out(JSON.stringify({
+      ok: true,
+      layout: 'sbm11',
+      generation,
+      activated: named === null && layout.generation !== '',
+      manifest: layout.manifest,
+      terms,
+      files: files.length,
+      bytes: directoryBytes
+    }, null, 2))
+    return EXIT_OK
+  }
+  out(`store ${root}`)
+  if (named !== null) out(`generation ${generation} (named on the command line)`)
+  else if (layout.generation !== '') out(`generation ${generation} (activated through CURRENT)`)
+  else out('generation directory (no CURRENT)')
+  out(`manifest ${layout.manifest}`)
+  out('layout sbm11')
+  out(plural(terms, 'term'))
+  out(`generation directory holds ${files.length} files, ${directoryBytes} bytes`)
+  return EXIT_OK
+}
+
+/**
+ * Answer a query on an SBM11 store (ops sbm11Open / sbm11Query /
+ * sbm11Close) and print it as `query` prints an IBK answer. The engine
+ * answers SPARQL Query Results JSON for SELECT and ASK and N-Triples for
+ * CONSTRUCT and DESCRIBE; this only chooses which renderer prints it.
+ */
+async function querySbm11 (engine, root, named, sparql, format, limit, options) {
+  const quiet = options.quiet === true
+  if (options.explain === true) {
+    err('factoidal query: --explain prints the plan of an IBK store; there is ' +
+      'no plan operation for an SBM11 store yet ' +
+      '(https://github.com/danbri/factoidal/issues/1009)')
+    return EXIT_FAILURE
+  }
+  let answer
+  try {
+    const store = await sbm11Functions().openSbm11(storeSource(root, named))
+    try {
+      const got = store.query(sparql)
+      // read the bytes while the handle is open
+      answer = { path: got.path, format: got.format, reads: got.reads,
+        bytesRead: got.bytesRead, text: got.result }
+    } finally {
+      store.close()
+    }
+  } catch (error) {
+    err(`factoidal query: ${engineWords(error)}`)
+    if (isStackOverflow(error)) {
+      for (const line of stackLimitAdvice(STACK_REMEDY.query)) err(line)
+    }
+    return EXIT_FAILURE
+  }
+  if (!quiet) {
+    err(`${answer.path}, ${plural(answer.reads, 'read')}, ${answer.bytesRead} bytes read`)
+  }
+  let result
+  if (answer.format === 'srj') {
+    const document = JSON.parse(answer.text)
+    if (typeof document.boolean === 'boolean') {
+      if (format === 'json') {
+        out(JSON.stringify(document, null, 2))
+        return EXIT_OK
+      }
+      result = { kind: 'ask', boolean: document.boolean }
+    } else {
+      result = { kind: 'select', srj: document }
+    }
+  } else {
+    result = { kind: 'construct', nquads: answer.text }
+  }
+  return renderQueryResult(engine, result, format, limit, quiet)
+}
+
 // ------------------------------------------------------------- inspect
 
 async function commandInspect (positional, options) {
   if (positional.length !== 1) throw new UsageError('inspect needs exactly one STORE')
   const root = positional[0]
   const named = typeof options.generation === 'string' ? options.generation : null
-  const store = openStore(root, named)
   const engine = await loadEngine()
+  const layout = sbm11Layout(engine, storeSource(root, named))
+  if (layout !== null) return inspectSbm11(root, named, layout, options)
+  const store = openStore(root, named)
   const envelope = inspectManifest(engine, store)
 
   if (options.json) {
@@ -1102,8 +1270,11 @@ async function commandQuery (positional, options) {
   const quiet = options.quiet === true
 
   const named = typeof options.generation === 'string' ? options.generation : null
-  const store = openStore(root, named)
   const engine = await loadEngine()
+  if (sbm11Layout(engine, storeSource(root, named)) !== null) {
+    return querySbm11(engine, root, named, sparql, format, limit, options)
+  }
+  const store = openStore(root, named)
 
   if (options.explain === true) {
     let plan
@@ -1208,9 +1379,15 @@ function readHead (path, length) {
 /** The syntax tag for an input: --syntax when given, else the engine's
  *  suffix table (op engineTables), else the engine's content detection
  *  (op detectFormat) on the first 64 KiB of the file. */
-function packSyntax (engine, input, options) {
+function packSyntax (engine, input, options, layout) {
   const tables = engine.call('engineTables', []).tables
-  const syntaxes = tables.packSyntaxes
+  // The SBM11 pack reads the whole document (op sbm11PackFiles), so it also
+  // reads the syntaxes of packWholeSuffixes (RDF/XML, JSON-LD).
+  const whole = layout === 'sbm11' ? (tables.packWholeSuffixes ?? {}) : {}
+  const syntaxes = tables.packSyntaxes.slice()
+  for (const syntax of Object.values(whole)) {
+    if (syntaxes.indexOf(syntax) < 0) syntaxes.push(syntax)
+  }
   if (typeof options.syntax === 'string') {
     const syntax = options.syntax.toLowerCase()
     if (syntaxes.indexOf(syntax) >= 0) return syntax
@@ -1220,11 +1397,14 @@ function packSyntax (engine, input, options) {
   for (const [suffix, syntax] of Object.entries(tables.packSuffixes)) {
     if (lower.endsWith(suffix)) return syntax
   }
+  for (const [suffix, syntax] of Object.entries(whole)) {
+    if (lower.endsWith(suffix)) return syntax
+  }
   for (const [suffix, name] of Object.entries(tables.packUnsupportedSuffixes)) {
     if (lower.endsWith(suffix)) {
       throw new UsageError(
-        `pack does not read ${name}. The packer's streaming fold reads ` +
-        `${syntaxes.join(', ')} only. Convert the file first, for ` +
+        `pack does not read ${name}. The packer reads ` +
+        `${syntaxes.join(', ')} only${layout === 'sbm11' ? '' : ' (--layout sbm11 also reads rdfxml and jsonld)'}. Convert the file first, for ` +
         "example with: factoidal parse FILE --out nquads")
     }
   }
@@ -1253,8 +1433,11 @@ function packSyntax (engine, input, options) {
  * `--base ''` asks for no base, which turns a relative IRI into a parse
  * error rather than a silently different term.
  */
-function packBase (input, options) {
+function packBase (input, options, layout) {
   if (typeof options.base === 'string') return options.base
+  // `l4sbm11-pack` uses no base unless --base names one; the same default
+  // gives the same bytes.
+  if (layout === 'sbm11') return ''
   const absolute = input.startsWith('/') ? input : joinPath(currentDirectory(), input)
   return 'file://' + absolute
 }
@@ -1275,8 +1458,8 @@ function packLayout (options) {
 async function commandPack (positional, options) {
   if (positional.length !== 2) throw new UsageError('pack needs INPUT and OUTPUT')
   const [input, output] = positional
-  const syntax = packSyntax(await loadEngine(), input, options)
   const layout = packLayout(options)
+  const syntax = packSyntax(await loadEngine(), input, options, layout)
   // The pack fold needs a bigger call stack than either runtime gives by
   // default (https://github.com/danbri/factoidal/issues/649). Under Node
   // the work runs on a worker thread with a raised stack; under Deno the
@@ -1291,8 +1474,10 @@ async function commandPack (positional, options) {
   let answer
   try {
     answer = await runPack(
-      { kind: 'pack', input, output, syntax, layout, base: packBase(input, options),
-        batchBytes: options['batch-bytes'] },
+      layout === 'sbm11'
+        ? { kind: 'packSbm11', input, output, syntax, base: packBase(input, options, layout) }
+        : { kind: 'pack', input, output, syntax, layout, base: packBase(input, options, layout),
+            batchBytes: options['batch-bytes'] },
       quiet
         ? undefined
         : (progress) => {
@@ -1331,7 +1516,7 @@ async function commandPack (positional, options) {
     return EXIT_OK
   }
   out(`packed ${report.bytesRead} bytes of ${syntax} into ${output}`)
-  out(`${report.written.length} artifacts, ${report.bytesWritten} bytes, layout ${layout}`)
+  out(`${report.written.length} ${layout === 'sbm11' ? 'files' : 'artifacts'}, ${report.bytesWritten} bytes, layout ${layout}`)
   if (typeof report.rows === 'number') out(`${plural(report.rows, 'row')}`)
   out(`activate it with: factoidal activate ${dirOf(output)} ${nameOf(output)}`)
   return EXIT_OK
@@ -1382,6 +1567,10 @@ async function commandActivate (positional, options) {
   const host = { worker: options['no-worker'] !== true }
   const reexec = await denoReexec(host)
   if (reexec !== null) return reexec
+  const engine = await loadEngine()
+  if (sbm11Layout(engine, joinPath(root, generation)) !== null) {
+    return activateSbm11(engine, root, generation, options)
+  }
   let answer
   try {
     answer = await runPack({ kind: 'activate', root, generation }, undefined, host)
@@ -1425,6 +1614,32 @@ async function commandActivate (positional, options) {
   return EXIT_OK
 }
 
+/**
+ * Activate an SBM11 generation: the engine checks every file against the
+ * manifest (op sbm11Verify), and only after its "ok" does CURRENT move.
+ */
+function activateSbm11 (engine, root, generation, options) {
+  let verdict
+  try {
+    verdict = engine.call('sbm11Verify', [joinPath(root, generation)])
+  } catch (error) {
+    err(`factoidal activate: ${engineWords(error)}`)
+    err('The generation is NOT activated; CURRENT is unchanged.')
+    return EXIT_FAILURE
+  }
+  const synced = atomicReplace(joinPath(root, 'CURRENT'), new TextEncoder().encode(generation))
+  if (options.json === true) {
+    out(JSON.stringify({ ...verdict, layout: 'sbm11', generation, directorySynced: synced }, null, 2))
+    return EXIT_OK
+  }
+  out(`activated ${generation}: ${verdict.artifacts} artifacts verified (${verdict.files} files, ${verdict.bytes} bytes), layout sbm11`)
+  if (!synced) {
+    err('CURRENT was replaced, but the directory entry was not synced; a ' +
+        'crash now could lose the pointer update.')
+  }
+  return EXIT_OK
+}
+
 function commandUpdate (positional, options) {
   if (positional.length < 1) throw new UsageError('update needs a STORE')
   const text = positional.length > 1
@@ -1529,6 +1744,10 @@ async function commandServe (positional, options) {
     const { join } = await import('node:path')
     const root = mkdtempSync(join(tmpdir(), 'factoidal-serve-'))
     const packOptions = { quiet: true, layout: packLayout(options) }
+    if (packOptions.layout === 'sbm11') {
+      throw new UsageError('serve --file packs a store that accepts updates; --layout sbm11 ' +
+        'packs a store that serve does not read yet. Use ibk3, ibk4 or ibk5.')
+    }
     let rc = await commandPack([options.file, join(root, 'gen-1')], packOptions)
     if (rc !== 0) return rc
     rc = await commandActivate([root, 'gen-1'], { quiet: true })
@@ -1549,6 +1768,12 @@ async function commandServe (positional, options) {
     options = { ...options, 'query-path': `${dsName}/sparql`, 'gsp-path': `${dsName}/data` }
   }
   if (positional.length !== 1) throw new UsageError('serve needs one STORE')
+  if (!tdb2 && sbm11Layout(await loadEngine(), positional[0]) !== null) {
+    err(`factoidal serve: ${positional[0]} is an SBM11 store, and serve reads IBK ` +
+      'stores only (the storeServe operations). Query it with: factoidal query ' +
+      `${positional[0]} QUERY (https://github.com/danbri/factoidal/issues/1009)`)
+    return EXIT_FAILURE
+  }
   const { listen, listenOnWorker, ServeHostError } = await import('./serve.mjs')
   const serveOptions = {
     port: servePort(options, 3030),
@@ -1590,12 +1815,25 @@ async function commandLwsServe (positional, options) {
   const insecure = options['insecure-no-auth'] === true
   const issuerOption = typeof options.issuer === 'string' ? options.issuer : null
   const authFile = typeof options['auth-config'] === 'string' ? options['auth-config'] : null
-  if (!insecure && issuerOption === null && authFile === null) {
-    throw new UsageError('lws-serve needs --issuer IRI or --auth-config FILE, ' +
+  const authzDir = typeof options.authz === 'string' ? options.authz : null
+  if (!insecure && issuerOption === null && authFile === null && authzDir === null) {
+    throw new UsageError('lws-serve needs --authz DIR, --issuer IRI or --auth-config FILE, ' +
       'or --insecure-no-auth for development')
   }
-  if (insecure && (issuerOption !== null || authFile !== null)) {
-    throw new UsageError('--insecure-no-auth cannot be combined with --issuer or --auth-config')
+  if (insecure && (issuerOption !== null || authFile !== null || authzDir !== null)) {
+    throw new UsageError('--insecure-no-auth cannot be combined with --authz, --issuer or --auth-config')
+  }
+  // --authz DIR: an authorization server on the same port, its issuer the
+  // storage root (L4Factoidal/Authz/Mount.lean). Its options are passed
+  // unread; the engine checks them (Wasm/Ops/Authz.lean).
+  const authz = authzDir === null ? undefined : {
+    dir: authzDir,
+    ...(typeof options.alg === 'string' ? { alg: options.alg } : {}),
+    ...(options['token-lifetime'] !== undefined ? { tokenLifetime: natOption('token-lifetime', options['token-lifetime']) } : {}),
+    ...(options['rotate-after'] !== undefined ? { rotateAfter: natOption('rotate-after', options['rotate-after']) } : {}),
+    ...(options['allow-http'] === true ? { allowHttp: true } : {}),
+    ...(options['allow-private-fetch'] === true ? { allowPrivateFetch: true } : {}),
+    ...corsConfig(options)
   }
   let auth
   if (authFile !== null) {
@@ -1628,6 +1866,7 @@ async function commandLwsServe (positional, options) {
       ...(typeof options.storage === 'string' ? { storage: options.storage } : {}),
       ...corsConfig(options),
       ...(insecure ? { insecureNoAuth: true } : { auth }),
+      ...(authz !== undefined ? { authz } : {}),
       ...(typeof options.owner === 'string' ? { owner: options.owner } : {}),
       ...(typeof options['root-acl'] === 'string'
         ? { rootAcl: new TextDecoder('utf-8', { fatal: true }).decode(readWhole(options['root-acl'])) }
@@ -1651,6 +1890,9 @@ async function commandLwsServe (positional, options) {
   }
   if (!quiet) {
     err(`lws-serve: listening on ${running.origin} for ${positional[0]}`)
+    if (running.authz() !== null) err(`lws-serve: authorization server ${running.authz().issuer} (keys in ${authzDir})`)
+    if (options['allow-http'] === true) err('lws-serve: WARNING: --allow-http: http subject identifiers are accepted')
+    if (options['allow-private-fetch'] === true) err('lws-serve: WARNING: --allow-private-fetch: private addresses may be fetched')
     if (insecure) err('lws-serve: WARNING: --insecure-no-auth: every request is served without authorization')
   }
   out(running.origin)

package/bin/pack-host.mjs

@@ -42,11 +42,12 @@
 // route rescues an in-page packer. See the closing section of
 // `docs/designissues/2026-09-03-npm-pack-in-wasm.md`.
 
-import { StoreHostError, listGeneration, readWhole } from '../store-host/index.mjs'
+import { StoreHostError, listGeneration, readWhole, writeNew } from '../store-host/index.mjs'
 import { fileUrlToPath, joinPath } from '../store-host/paths.mjs'
 import { loadEngine } from './engine.mjs'
 import { PackError, packFile, packSupported, verifyGeneration } from './pack.mjs'
 import { StoreOperationError, openStore } from './store.mjs'
+import { sbm11Api } from '../lib/sbm11.mjs'
 
 const isDeno = typeof globalThis.Deno !== 'undefined'
 
@@ -207,6 +208,7 @@ export function isStackOverflow (error) {
  */
 export async function packHere (task, onProgress) {
   const engine = await loadEngine()
+  if (task.kind === 'packSbm11') return await packSbm11Here(engine, task)
   if (!packSupported(engine)) return { notWired: true }
   if (task.kind === 'activate') {
     return { report: activateHere(engine, task) }
@@ -221,6 +223,34 @@ export async function packHere (task, onProgress) {
   return { report }
 }
 
+/**
+ * Pack one document into the files of one SBM11 generation (op
+ * sbm11PackFiles, through lib/sbm11.mjs packSbm11) and write each file
+ * under the name the engine gave it. The document is read whole: the op
+ * takes the whole document, and refuses one above its limit.
+ *
+ * @returns {{notWired: true}|{report: object}}
+ */
+async function packSbm11Here (engine, task) {
+  const { packSbm11 } = sbm11Api(async () => engine)
+  const bytes = readWhole(task.input)
+  let files
+  try {
+    files = await packSbm11(bytes, { syntax: task.syntax, base: task.base })
+  } catch (error) {
+    if (error && error.code === 'SBM11_NO_PACK_OP') return { notWired: true }
+    throw error
+  }
+  const written = []
+  let bytesWritten = 0
+  for (const [name, fileBytes] of files) {
+    writeNew(joinPath(task.output, name), fileBytes)
+    written.push(name)
+    bytesWritten += fileBytes.length
+  }
+  return { report: { bytesRead: bytes.length, written, bytesWritten } }
+}
+
 /**
  * Verify one generation and answer the engine's verdict.
  *

package/index.d.ts

@@ -529,14 +529,15 @@ export function canonicalize(
 export function graphs(dataset: Dataset): Array<[name: NamedNode | BlankNode, graph: Dataset]>;
 
 /**
- * RDFC-1.0 canonical form of a single graph -- the graph-scoped
- * sibling of canonicalize(). It returns the canonical N-Quads TEXT, not
- * a digest: hash the string yourself (for example `vcSha256Hex`) when a
- * fixed-length value is needed. Every quad's graph component is dropped
- * before canonicalizing, so isomorphic graphs (including under
- * blank-node relabeling) give the same string regardless of what
- * graph name they were extracted from. Typically called with one
- * entry of graphs()'s output.
+ * The RDFC-1.0 canonical hash of a single graph: the SHA-256 of its
+ * canonical N-Quads document, as 64 lowercase hexadecimal digits (RDF
+ * Dataset Canonicalization 1.0, section 4.4 note 2). The engine computes
+ * the canonical form and the digest. Every quad's graph component is
+ * dropped first, so isomorphic graphs (including under blank-node
+ * relabeling) give the same hash whatever graph name they were extracted
+ * from. Typically called with one entry of graphs()'s output. For the
+ * canonical text, use canonicalize(). Rejects when the input holds an
+ * RDF 1.2 triple term or the section 4.4 call budget is exceeded.
  */
 export function canonicalHash(datasetOrGraph: Dataset): Promise<string>;
 

package/l4-assets/l4factoidal.js

@@ -37,7 +37,7 @@
 // bytes change.
 
 // Stamped by formal/lean4/Wasm/build-wasm.sh step 9 -- do not hand-edit.
-const WASM_VERSION = "1b10444e4416";
+const WASM_VERSION = "ebac38a482d0";
 
 import createModule from './l4factoidal.mjs';
 

package/l4-assets/version.json

@@ -2,17 +2,17 @@
   "engine": "lean4",
   "note": "The Lean 4 (L4Factoidal) engine build provenance and claims: the only engine of @factoidal/core since 0.9.0 (issue 713). This committed file is a placeholder. tools/fetch-builds.sh --set web (through tools/stamp-l4-version.mjs) writes the provenance members from the danbri/factoidal-builds release that built the wasm; formal/lean4/Wasm/build-wasm.sh step 9 writes them for a local build. Do not commit a stamped copy.",
   "stamped": true,
-  "release": "danbri/factoidal-builds build-0069da82efd7-web",
-  "releaseRun": "https://github.com/danbri/factoidal-builds/actions/runs/40281606727",
-  "gitSha": "0069da82efd75cc3bfe10785ef8853bd84931957",
-  "builtAt": "2026-10-26T08:11:18Z",
+  "release": "danbri/factoidal-builds build-fa72ba5bc828-web",
+  "releaseRun": "https://github.com/danbri/factoidal-builds/actions/runs/40439252690",
+  "gitSha": "fa72ba5bc828c1eb75572c3436e9d739f76fdd57",
+  "builtAt": "2026-10-27T03:13:16Z",
   "leanToolchain": "leanprover/lean4:v4.33.1",
   "emscripten": "6.0.8",
   "abiVersion": "1",
   "engineVersion": "l4factoidal-wasm 0.2.0 (dispatch: RDF parse/serialize, SPARQL query+update, RDFS/OWL-RL closures, RDFC-1.0, CL/IKL)",
   "version": "l4factoidal-wasm 0.2.0 (dispatch: RDF parse/serialize, SPARQL query+update, RDFS/OWL-RL closures, RDFC-1.0, CL/IKL)",
-  "wasmSha256": "1b10444e44160b8689ebd5375c074509196930b1927ce4c9fbc829a73d9b58ac",
-  "wasmBytes": 17034490,
+  "wasmSha256": "ebac38a482d03d4a6ce1cf4a59164288628fed6957483dd55180743e7a3daf40",
+  "wasmBytes": 17071472,
   "claims": {
     "source": "formal/lean4 (L4Factoidal): no sorry, no user axioms, no native_decide; W3C behaviour pinned by build-time #guard",
     "suitesAtBuildSha": "see docs/test-results and formal/lean4/PORT_NOTES.md at gitSha"

package/lib/api.mjs

@@ -884,9 +884,12 @@ function buildApi(driver) {
   }
 
   /**
-   * RDFC-1.0 canonical N-Quads of one graph: every quad's graph name is
-   * dropped first (op datasetGraphAsDefault with the union of the graphs).
-   * Text, not a digest.
+   * The RDFC-1.0 canonical hash of one graph: the SHA-256, as 64 lowercase
+   * hexadecimal digits, of the canonical N-Quads document (RDF Dataset
+   * Canonicalization 1.0, section 4.4 note 2). Every quad's graph name is
+   * dropped first (op datasetGraphAsDefault with the union of the graphs);
+   * then op datasetCanonicalHash computes the canonical form and the digest
+   * in Lean. canonicalize() answers the canonical text.
    * @param {Dataset} datasetOrGraph
    * @returns {Promise<string>}
    */
@@ -897,10 +900,11 @@ function buildApi(driver) {
     datasetOrGraph.size; // throws the disposed error
     const c = datasetOrGraph._ctx;
     requireEntryFn(c.e, 'datasetGraphAsDefault', 'canonicalHash');
+    requireEntryFn(c.e, 'datasetCanonicalHash', 'canonicalHash');
     const merged = c.wrap(entryResult(c.e.datasetGraphAsDefault(datasetOrGraph.handle,
       JSON.stringify({ type: 'union' })), 'canonicalHash'), {});
     try {
-      return merged.toString('rdfc');
+      return entryResult(c.e.datasetCanonicalHash(merged.handle), 'canonicalHash').hash;
     } finally {
       merged.dispose();
     }

package/lib/store.mjs

@@ -99,16 +99,18 @@ export function manifestKeys(inspectEnvelope) {
   return keys;
 }
 
-const HEX = (() => {
-  const t = new Array(256);
-  for (let b = 0; b < 256; b += 1) t[b] = b.toString(16).padStart(2, '0');
-  return t;
-})();
+const HEX_DIGIT = Uint8Array.from('0123456789abcdef', (c) => c.charCodeAt(0));
+const HEX_DECODER = new TextDecoder();
 
 /** Lowercase hexadecimal: the transport the manifest argument of the store
  *  ops takes. Artifact bytes never take this path. */
 export function hexOfBytes(bytes) {
-  const parts = new Array(bytes.length);
-  for (let i = 0; i < bytes.length; i += 1) parts[i] = HEX[bytes[i]];
-  return parts.join('');
+  // One byte buffer of ASCII digits, decoded once (issue 1039; see
+  // bin/engine.mjs hexOfBytes).
+  const out = new Uint8Array(bytes.length * 2);
+  for (let i = 0, at = 0; i < bytes.length; i += 1, at += 2) {
+    out[at] = HEX_DIGIT[bytes[i] >> 4];
+    out[at + 1] = HEX_DIGIT[bytes[i] & 15];
+  }
+  return HEX_DECODER.decode(out);
 }

package/lib/version.mjs

@@ -2,4 +2,4 @@
 // needs import attributes, which not every runtime this package targets
 // reads). test/dts-drift.test.js checks that it equals package.json's
 // `version`.
-export const version = '0.16.0';
+export const version = '0.17.0';

package/lws/server.mjs

@@ -584,11 +584,33 @@ export async function createLwsServer (options = {}) {
   let settings = { ...options }
   let opening = null
   let opened = null
+  let authz = null
 
-  /** The storage handle, opened once. */
+  /** The storage handle, opened once. With `authz` (an object of
+   *  `openAuthorizationServer` options), an authorization server that shares
+   *  the storage's issuer is opened first, on `mountedOn: baseIri`; the
+   *  storage then trusts that issuer with the JWK Set the engine published
+   *  (L4Factoidal/Authz/Mount.lean). */
   function storage () {
     if (opening === null) {
-      opening = openLwsStorage(settings).then((value) => { opened = value; return value })
+      opening = (async () => {
+        let storageSettings = settings
+        if (settings.authz !== undefined && settings.authz !== null) {
+          if (typeof settings.baseIri !== 'string') throw new LwsHostError('authz needs baseIri')
+          const { openAuthorizationServer } = await import('../authz/server.mjs')
+          authz = await openAuthorizationServer({
+            ...settings.authz,
+            ...(settings.engine !== undefined ? { engine: settings.engine } : {}),
+            ...(settings.auth && typeof settings.auth.audience === 'string' ? { audience: settings.auth.audience } : {}),
+            mountedOn: settings.baseIri
+          })
+          const issuers = [{ issuer: authz.issuer, jwks: authz.jwks }]
+          const prior = settings.auth && Array.isArray(settings.auth.issuers) ? settings.auth.issuers : []
+          storageSettings = { ...settings, auth: { ...(settings.auth || {}), issuers: issuers.concat(prior) } }
+        }
+        opened = await openLwsStorage(storageSettings)
+        return opened
+      })()
     }
     return opening
   }
@@ -610,7 +632,13 @@ export async function createLwsServer (options = {}) {
     }
     let answer
     try {
-      answer = await (await storage()).step(record)
+      const store = await storage()
+      // One port for both servers: the engine names the authorization
+      // server's paths (L4Factoidal.Authz.owns); every other request is the
+      // storage's. The token request body is form text.
+      answer = authz !== null && authz.owns(record.target)
+        ? await authz.step({ ...record, body: new TextDecoder('utf-8').decode(record.body) })
+        : await store.step(record)
     } catch (error) {
       // A failure of this host or of the engine, not a protocol answer.
       // The engine's own words are kept; nothing is invented.
@@ -627,6 +655,8 @@ export async function createLwsServer (options = {}) {
   return {
     server,
     storage,
+    /** The authorization server sharing the storage's issuer, or null. */
+    authz () { return authz },
     /** Set the storage root's absolute IRI, before the first request. */
     setBaseIri (value) {
       if (opening !== null) {
@@ -646,6 +676,7 @@ export async function createLwsServer (options = {}) {
       return new Promise((resolve) => {
         server.close(() => {
           if (opened !== null) opened.close()
+          if (authz !== null) authz.close()
           resolve()
         })
       })

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@factoidal/core",
-  "version": "0.16.0",
+  "version": "0.17.0",
   "description": "RDF parsing and writing (Turtle, TriG, N-Triples, N-Quads, RDF/XML, JSON-LD), SPARQL 1.1 and 1.2, RDFS and OWL reasoning, SHACL and ShEx validation, CSVW and RML mapping, RDFC-1.0, and SPARQL, LWS and Solid servers, for Node, Deno and browsers.",
   "keywords": [
     "sparql",
@@ -142,6 +142,7 @@
   "scripts": {
     "build": "bash ../../formal/lean4/Wasm/build-wasm.sh",
     "test": "node --test 'test/*.test.js' 'test/*.test.mjs'",
+    "test:jena": "node --test 'test/jena/*.test.mjs'",
     "test:speed": "node ../../tests/perf/npm_speed_gate.mjs",
     "test:costs": "node ../../tests/perf/npm_costs.mjs",
     "prepublishOnly": "npm test"