@factoidal/core 0.16.0 → 0.18.0

package/CHANGELOG.md

@@ -2,6 +2,29 @@
 
 ## Unreleased
 
+## 0.18.0 (2026-10-27)
+
+- XSD constructor functions for datatypes outside the SPARQL 1.1 §17.5 table (`xsd:byte`, `xsd:int`, `xsd:unsignedInt`, `xsd:date`, `xsd:time`, `xsd:gYear` and the other `xsd:g*` types, the duration types, `xsd:token`, …) now cast as XPath F&O 3.1 §19 says. Before, they changed only the datatype and gave ill-typed literals such as `"HaHa"^^xsd:byte`. An invalid value or a cast that F&O does not allow is now an error (the variable is unbound) (https://github.com/danbri/factoidal/issues/1041, https://github.com/danbri/factoidal-core/pull/142).
+- SPARQL: STRSTARTS, STRENDS, CONTAINS, STRBEFORE and STRAFTER compare the base direction of `rdf:dirLangString` arguments (SPARQL 1.2 §17.4.3: same language tag and same base direction). `CONTAINS("abc"@en--ltr, "a"@en)` is now an error. STRBEFORE and STRAFTER keep the base direction of the first argument: `STRBEFORE("abc"@en--ltr, "b")` is `"a"@en--ltr` (https://github.com/danbri/factoidal/issues/1042, https://github.com/danbri/factoidal-core/pull/136).
+- Store open sends the manifest to the engine as bytes, not as hexadecimal inside the JSON arguments, and decodes it once for the plan and the open (new ops `storeManifestInspectBytes`, `storeQueryPlanBytes`, `storeOpenBytes`; the hexadecimal ops stay). On the londat store (1,046 blocks, 753 kB manifest) a scoped `openStoreHandle` takes 198 ms, not 305 ms with 0.17.0 (median of 9) (https://github.com/danbri/factoidal/issues/1039, https://github.com/danbri/factoidal-core/pull/141).
+- Tests: the W3C manifest runner `l4w3c` reads `.n3` expected results when they are Turtle-compatible (else the test is unsupported, with the reason) and runs Jena's `mfx:TestQuery` as a query evaluation test; Jena ARQ manifests: 31 pass (was 13) of 308 (https://github.com/danbri/factoidal-core/pull/138). The Jena ARQ expression suite drops the 62 expected failures these fixes remove. `tools/split-read-differential.sh` runs on macOS and fails when it checks no file (https://github.com/danbri/factoidal-core/pull/133).
+- LWS client (https://github.com/danbri/factoidal/issues/1065): `read` takes `rangeStart`, `rangeEnd`, `ifNoneMatch` and `prefer` and reports `notModified` (304) and `contentRange`; `verifyNotification` takes `trustedStorages` and never asks an untrusted storage for its description (theorem `step_start_verify_trusted`); `search` and `typeIndex` take `onePage` (one page with its `next`, theorem `onPage_onePage_no_fetch`) and `endpoint` (the service endpoint, no discovery); `describe`, `readPage`, `findLinkset`, `putLinkset`, `getSubscription` and `queryFormats` (OPTIONS, `Accept-Query`) are new kinds; `subscribe` takes `expires`; `updateLinkset` takes `linksetUrl`; a session key may be Ed25519 (EdDSA) as well as P-256; the session takes `bearer` (`{token, realm?}`, a token with no realm goes to every request) and `idToken`. New wasm op `lwsClientNewKey` and `createDidKey({ algorithm })` make a P-256 or Ed25519 `did:key` agent from host random bytes (theorems `multikeyJwk?_p256`, `multikeyJwk?_ed25519`). Each `done` document has a `report` (Metadata, Item, Page, Storage, Subscription shapes read in Lean, `Client/Report.lean`) and each `fail` document a `report` with the driver error kind (`LwsClientError.report`). The lws-driver/1 adapter `deploy/lws-client-adapter/adapter.mjs` only translates JSON now; checker result in its README. Needs a wasm with the op `lwsClientNewKey`.
+- WebDAV: `factoidal webdav-serve CONFIGDIR [--port N] [--host ADDRESS]` serves LWS storages to file browsers (macOS Finder, Windows Explorer, GNOME Files, KDE Dolphin) over WebDAV, RFC 4918 classes 1 and 2 (https://github.com/danbri/factoidal/issues/1066). The root collection holds one folder per mount; a mount is one Markdown file `CONFIGDIR/NAME.md` (`- kind: local` with `- path:` an `lws-serve` storage directory, or `- kind: lws` with `- url:` a remote storage and `- auth: none | did-key | bearer`). A local mount is opened by the ops `lws-serve` uses, as the storage's owner; a remote mount is reached through the LWS client, with a self-issued `did:key` credential from a P-256 key file (made when absent). Every decision is in Lean (`L4Factoidal/WebDAV/`); the host `webdav/server.mjs` carries sockets, files and fetches. New wasm ops `webdavOpen`, `webdavLoad`, `webdavStep`, `webdavResume`, `webdavClose`. JS: `listen({ configDir })` and `openWebdav({ configDir })` from `@factoidal/core/webdav`. litmus 0.13: 208 pass, 0 fail (out of 208: basic, copymove, props, locks, http on a local and a remote mount). Not yet: dead properties and locks kept across a restart (https://github.com/danbri/factoidal/issues/1078); checks with the real file browsers (https://github.com/danbri/factoidal/issues/1079). Design record: `docs/designissues/2026-10-27-webdav.md`.
+- JSON-LD-star (RDF 1.2 triple terms, reifiers, annotations; JSON-LD-star draft, json-ld/json-ld-star `1.2-tests`): option `rdfstar` on `jsonldToRdf`, `jsonldFromRdf`, `parse` and `serialize` with `format: 'jsonld'` reads and writes `@triple`, `@reifies` and `@annotation`; without it a triple term is refused when writing JSON-LD. 170 of 171 json-ld-star tests pass (the one fail is a defective fixture, fromRdf tst07); W3C json-ld-api unchanged (791 pass, 2 local-override, of 793) (https://github.com/danbri/factoidal-core/pull/134, https://github.com/danbri/factoidal-core/pull/140).
+
+## 0.17.0 (2026-10-27)
+
+- **Breaking:** `canonicalHash(graph)` now returns the SHA-256 (64 lowercase hex digits) of the RDFC-1.0 canonical N-Quads, as the RDF Dataset Canonicalization specification uses it. Before, it returned the canonical text. Use `canonicalize()` for the text. The engine computes the digest in Lean (new op `datasetCanonicalHash`) (https://github.com/danbri/factoidal/issues/1040, https://github.com/danbri/factoidal-core/pull/127).
+- SPARQL string functions follow SPARQL 1.1 §17.4.3 argument types: STRLEN, SUBSTR, UCASE, LCASE, STRSTARTS, STRENDS, CONTAINS, ENCODE_FOR_URI, MD5 and SHA* give a type error on a number, boolean or IRI argument (MD5/SHA* also on a language-tagged literal). STRSTARTS/STRENDS/CONTAINS check argument compatibility. SUBSTR follows `fn:substring`: `SUBSTR("12345", 0, 3)` is now `"12"` (was `"123"`) (https://github.com/danbri/factoidal/issues/1036, https://github.com/danbri/factoidal-core/pull/126).
+- `factoidal pack --layout sbm11` writes an SBM11 generation: the same files as `l4sbm11-pack`, byte for byte (605 of 605 W3C census inputs). This layout also reads RDF/XML and JSON-LD. `factoidal activate`, `inspect` and `query` read SBM11 stores. `activate` checks the length, SHA-256 and Merkle root of every file before it moves `CURRENT`. `serve` does not read SBM11 stores yet. IBK5 stays the default layout. New wasm ops `storeLayout` and `sbm11Verify` (https://github.com/danbri/factoidal/issues/1009, https://github.com/danbri/factoidal-core/pull/131).
+- Store open in Node is faster: on the londat store (142,314 quads, 1,046 blocks) `openStoreHandle` takes 562 ms, not 1485 ms (median of 7 runs, Apple M1, Node 22). The engine decodes the hexadecimal manifest over bytes, reads `JSON.stringify` arguments in place, verifies artifacts with HACL* SHA-256 in `storeOpen`, and decodes the arguments once per call. What remains is the manifest parse, which grows with the number of blocks; an SBM11 store opens in about 5 ms (https://github.com/danbri/factoidal/issues/1039, https://github.com/danbri/factoidal-core/pull/135).
+- The chunked Turtle reader (the default `l4sbm11-pack` path and the source layer's split read) reads a subject such as `prefix:Atto`, `base:x` or `version:v` as a prefixed name. Before, it took the name for a `PREFIX`/`BASE`/`VERSION` directive and refused the document (QUDT-all-in-one-OWL.ttl) (https://github.com/danbri/factoidal/issues/1028, https://github.com/danbri/factoidal-core/pull/128).
+- RML: a reference to a SPARQL result binding (`formats:SPARQL_Results_CSV`) keeps the literal's datatype when the term map states none. `"5"^^xsd:integer` gives `"5"^^xsd:integer`, not `"5"` (https://github.com/danbri/factoidal/issues/1034, https://github.com/danbri/factoidal-core/pull/129).
+- Tests: 996 Apache Jena ARQ expression tests run as SPARQL on the package (`test/jena-arq-expr.test.mjs`). Each case that differs from Jena has a verdict: a Factoidal defect with an open issue (https://github.com/danbri/factoidal/issues/1041, https://github.com/danbri/factoidal/issues/1042, https://github.com/danbri/factoidal/issues/573), or a Jena extension or difference from the SPARQL specification (https://github.com/danbri/factoidal-core/pull/132). Graph names that are blank nodes: `test/graph-names-876.test.mjs` (https://github.com/danbri/factoidal/issues/876 part 2, fixed in 0.16.0 by https://github.com/danbri/factoidal-core/pull/75; https://github.com/danbri/factoidal-core/pull/130).
+- Tests and release process: no npm test skips when the wasm lacks an op; each test file asserts the ops it needs, so a wasm without an op fails. The `serve --tdb2` test, which needs Apache Jena, moved to `npm run test:jena`. Core PR CI runs `tests/dataflow/run.sh` and `npm test`. `tools/npm-release-check.mjs` checks the version in package.json, package-lock.json and lib/version.mjs, and a dated changelog heading; the publish workflow runs it. `tools/fetch-builds.sh` warns when the installed release was built from a commit whose source differs from HEAD; `--strict` fails (https://github.com/danbri/factoidal/issues/1038, https://github.com/danbri/factoidal-core/pull/124, https://github.com/danbri/factoidal-core/pull/125).
+- LWS: `factoidal lws-serve DIR --authz ASDIR` serves the storage and an authorization server on one port, the issuer the storage root without its trailing slash. The token endpoint and the JWK Set are under the storage root's `.well-known/` (`lws-token`, `lws-jwks`), so no storage resource name is taken from clients; the RFC 8414 metadata is at `/.well-known/lws-configuration` + the issuer's path. The engine decides which requests are the authorization server's (new op `authzOwns`, Lean `L4Factoidal/Authz/Mount.lean`, theorem `endpointOf?_mem`); the storage trusts the issuer with the JWK Set the engine publishes. JS: `listen({ authz: { dir, ... } })` from `@factoidal/core/lws/server`. `authzOpen` takes `mountedOn`, `tokenPath` and `jwksPath`, and answers `resources` and `jwks`.
+- LWS and the authorization server read both forms of their well-known paths: the whole path (a client that reaches the server directly) and the path with the storage root's or issuer's path removed (a reverse proxy that serves the server at a path prefix and removes the prefix, as the router of Touchstone's `targetservers` does). Up to 0.16.0 the protected resource metadata named the wrong resource behind such a proxy (`L4Factoidal.LWS.resourceOfMetadataPath`). Test: `tests/authz/mounted.mjs`. Docker target for third-party testing: `deploy/lws-target/`.
+
 ## 0.16.0 (2026-10-26)
 
 - Node settings in the evidence (https://github.com/danbri/factoidal/issues/975). The params of `node()` are the node's settings. With a wasm module that has the new op `pipelineCheckNodes` (Lean `Wasm/Ops/PipelineNodes.lean`, `Dataflow/NodeSettings.lean`), each evidence record carries `settings` (canonical: keys sorted, defaults of absent settings that narrow the contract such as `lenient: false` and `entail: 'none'`, a Dataset value as `{nquadsSha256}`) and `settingsDigest` (`sha256:` and the hex SHA-256 of the canonical text). A setting outside the proved domain of a proved stage (a lenient reader, an evaluator with `entail` other than `none` or with `output`, a key the stage's settings table does not name) makes the node's status `unproved`, with `registryStatus` and `outside` (each setting and the reason) in the record, so `run()` reports `proved: false`. A reader's `format` must name the stage's syntax (`ttl12` selects RDF 1.2 mode, reported as `derived.mode`); an evaluator's query must parse, else `node()` rejects. Design record: `docs/designissues/2026-10-25-pipeline-settings-and-provenance.md`.

package/README.md

@@ -8,14 +8,15 @@ npm install @factoidal/core
 ```
 
 ```js
-import { parse, csvwToRdf } from "@factoidal/core";
+import { parse, csvwToRdf, canonicalHash } from "@factoidal/core";
 
 const ds = await parse('<http://x/a> <http://x/n> "A" .', { format: "ntriples" });
 const rows = await ds.query("SELECT ?n WHERE { ?s <http://x/n> ?n }");
 console.log(rows[0].get("n").value);              // A
 console.log(ds.toString("turtle"));
 console.log(await ds.serialize({ format: "rdfxml" }));
-console.log(await ds.canonicalize());              // RDFC-1.0
+console.log(await ds.canonicalize());              // RDFC-1.0 canonical N-Quads
+console.log(await canonicalHash(ds));              // its SHA-256, 64 hex digits
 
 const table = await csvwToRdf("id,name\n1,a\n", "", { mode: "minimal", url: "http://x/t.csv" });
 console.log(table.size);                           // 2

package/authz/server.mjs

@@ -22,6 +22,7 @@
 //   authzFetched [handle, { url, status, body, now } | { host, addresses, now }] -> { ok }
 //   authzKeys    [handle, { nonce?, private? }] -> { ok, keys } | { ok, jwks }
 //   authzRotate  [handle, { now, random }]  -> { ok, changed, kid }
+//   authzOwns    [handle, target]          -> { ok, owns }
 //   authzClose   [handle]                   -> { ok }
 //
 // FETCHES. `resolve` names a host: this host resolves it (`dns.lookup`, every
@@ -38,7 +39,7 @@ import { loadEngine } from '../bin/engine.mjs'
 import { storageKeyText } from '../lws/persist.mjs'
 
 /** The ops this host needs from the engine. */
-export const AUTHZ_OPS = ['authzOpen', 'authzStep', 'authzFetched', 'authzKeys', 'authzRotate', 'authzClose']
+export const AUTHZ_OPS = ['authzOpen', 'authzStep', 'authzFetched', 'authzKeys', 'authzRotate', 'authzOwns', 'authzClose']
 
 /** The key file in DIR. */
 export const KEY_FILE = 'authz-keys.json'
@@ -174,12 +175,16 @@ export async function openAuthorizationServer (options = {}) {
   const engine = options.engine !== undefined ? options.engine : await loadEngine()
   const dir = typeof options.dir === 'string' ? options.dir : null
   const config = {
-    issuer: options.issuer,
     resources: Array.isArray(options.resources) ? options.resources : [],
     now: nowSeconds(),
     entropy: randomBase64(64)
   }
-  for (const name of ['alg', 'tokenLifetime', 'leeway', 'rotateAfter', 'docTtl']) {
+  // `mountedOn` (an LWS storage root that shares this issuer) or `issuer`:
+  // passed unread; the engine derives the issuer, the resource and the
+  // endpoint paths from `mountedOn` (L4Factoidal/Authz/Mount.lean).
+  if (typeof options.mountedOn === 'string') config.mountedOn = options.mountedOn
+  else config.issuer = options.issuer
+  for (const name of ['alg', 'tokenLifetime', 'leeway', 'rotateAfter', 'docTtl', 'tokenPath', 'jwksPath', 'audience']) {
     if (options[name] !== undefined) config[name] = options[name]
   }
   if (options.allowHttp === true) config.allowHttp = true
@@ -208,6 +213,15 @@ export async function openAuthorizationServer (options = {}) {
     engine,
     handle,
     issuer: opened.issuer,
+    /** The resources tokens may be issued for, as the engine set them. */
+    resources: opened.resources,
+    /** The published JWK Set when the handle was opened. */
+    jwks: opened.jwks,
+    /** Whether a request target is this server's (the engine decides:
+     *  L4Factoidal.Authz.owns). */
+    owns (target) {
+      return engine.call('authzOwns', [handle, String(target)]).owns === true
+    },
     /**
      * One request.
      * @param {{method: string, target: string, headers: Array<[string, string]>, body: string}} request

package/bin/engine.mjs

@@ -120,13 +120,8 @@ export function loadEngine (name = 'l4factoidal') {
   return promise
 }
 
-const HEX = (() => {
-  const table = new Array(256)
-  for (let byte = 0; byte < 256; byte += 1) {
-    table[byte] = byte.toString(16).padStart(2, '0')
-  }
-  return table
-})()
+const HEX_DIGIT = Uint8Array.from('0123456789abcdef', (c) => c.charCodeAt(0))
+const HEX_DECODER = new TextDecoder()
 
 /**
  * Lowercase hexadecimal of a byte array. This is a transport encoding for
@@ -137,7 +132,14 @@ const HEX = (() => {
  * @returns {string}
  */
 export function hexOfBytes (bytes) {
-  const parts = new Array(bytes.length)
-  for (let index = 0; index < bytes.length; index += 1) parts[index] = HEX[bytes[index]]
-  return parts.join('')
+  // One byte buffer of ASCII digits, decoded once: about 20 times faster
+  // than joining one string per byte for a 753 kB store manifest
+  // (https://github.com/danbri/factoidal/issues/1039).
+  const out = new Uint8Array(bytes.length * 2)
+  for (let index = 0, at = 0; index < bytes.length; index += 1, at += 2) {
+    const byte = bytes[index]
+    out[at] = HEX_DIGIT[byte >> 4]
+    out[at + 1] = HEX_DIGIT[byte & 15]
+  }
+  return HEX_DECODER.decode(out)
 }