@factoidal/core 0.19.0 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +65 -0
- package/README.md +59 -15
- package/authz/server.mjs +23 -2
- package/bin/engine.mjs +22 -1
- package/bin/factoidal.mjs +710 -38
- package/bin/pack-host.mjs +5 -0
- package/bin/serve.mjs +19 -3
- package/browser.d.ts +23 -0
- package/browser.js +26 -0
- package/index.d.ts +463 -24
- package/index.mjs +29 -0
- package/l4-assets/l4factoidal.js +1 -1
- package/l4-assets/l4factoidal.wasm +0 -0
- package/l4-assets/version.json +6 -6
- package/lib/api.mjs +480 -28
- package/lib/browser-engine.mjs +28 -1
- package/lib/cid-node.mjs +77 -0
- package/lib/cid.mjs +146 -0
- package/lib/cipher-host.mjs +126 -0
- package/lib/dataset.mjs +96 -12
- package/lib/engine-integrity.mjs +56 -0
- package/lib/lean-entry.mjs +30 -12
- package/lib/nodes.mjs +318 -17
- package/lib/sbm11-lws.mjs +4 -2
- package/lib/sbm11.mjs +29 -6
- package/lib/version.mjs +1 -1
- package/lws/README.md +9 -0
- package/lws/client.mjs +19 -2
- package/lws/persist.mjs +169 -0
- package/lws/server.mjs +82 -29
- package/package.json +2 -1
- package/solid/server/index.mjs +79 -13
- package/webdav/server.mjs +154 -25
- package/xmpp/client.mjs +332 -0
- package/xmpp/proxy.mjs +114 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,71 @@
|
|
|
2
2
|
|
|
3
3
|
## Unreleased
|
|
4
4
|
|
|
5
|
+
## 0.21.0 (2026-10-28)
|
|
6
|
+
|
|
7
|
+
### Security: WebDAV (https://github.com/danbri/factoidal/issues/1147)
|
|
8
|
+
|
|
9
|
+
- `webdav-serve` is read only: PUT, DELETE, MKCOL, COPY, MOVE, PROPPATCH, LOCK and UNLOCK answer 403 on every mount; `writable:` is ignored; OPTIONS answers `DAV: 1` with the read methods in `Allow`, so a file browser mounts the volume read only. Before, any client that reached the listener could act as a mount's LWS agent, and a `kind: local` mount opened a storage with no authorization.
|
|
10
|
+
- Every request acts as one named user. New `--guest NAME`: a request with no credentials acts as NAME (with no `--guest`: 401). A request with credentials logs in through `--users` (with no users file: 401). A mount file needs `users:`; another user gets 404 for that mount, and the root does not list it.
|
|
11
|
+
- `kind: local` mounts are refused at start (use `kind: lws` with an agent key). `auth: bearer` needs `allow-external-token: true` (a warning is printed).
|
|
12
|
+
- Lean theorems (`L4Factoidal/WebDAV/Theorems.lean`, `Route.lean`): `webdav_within_lws` (every storage operation that a request causes is a read, on a mount that lists the request's user, and leaves the server only as an LWS client operation on that mount, which the LWS server authorizes), `load_no_unauthorized_storage` (every accepted mount is `kind: lws`), `handle_no_write`, `hidden_mount_trace`. Not yet: each WebDAV user mapped to an LWS agent of its own (a mount's listed users act as the mount's agent).
|
|
13
|
+
|
|
14
|
+
### Security: LWS storage data
|
|
15
|
+
|
|
16
|
+
- LWS: a Lean theorem, `lws_data_only_through_authorized_step` (`L4Factoidal/LWS/AccessTheorems.lean`), proves that only the LWS request handler reads or changes an LWS storage's data, and only after `Authorization.decide`; a refused request changes nothing. The one exception is `lws-serve --insecure-no-auth` (`insecureNoAuth`). The engine's LWS handle tables are private to the LWS ops, and `lwsStep` is the only op that serves resource content. Not covered: file system access to the storage directory (encryption at rest, `FACTOIDAL_LWS_KEY`, addresses it), and JavaScript in the same process.
|
|
17
|
+
- `solid-serve` (and `openSolidStorage`) refuses a directory that holds an LWS storage written by `lws-serve`. Before, it served that data under Solid rules, with no authentication when no `auth` option was given, and dropped the LWS notification and access state.
|
|
18
|
+
|
|
19
|
+
### Other changes
|
|
20
|
+
|
|
21
|
+
- The engine is checked against its release stamp when it loads (https://github.com/danbri/factoidal/issues/1106, layer 1): when `l4-assets/version.json` is stamped, the SHA-256 and size of `l4factoidal.wasm` must match it, in Node, Deno and the browser, or the package does not start (error code `engine.modified`, naming the file and both digests). An unstamped copy (a source checkout) loads as before. `FACTOIDAL_ALLOW_MODIFIED_ENGINE=1` loads a changed engine with a warning. This catches a stale or partly upgraded install; it does not defend against someone who can edit the package. In the browser a stamped engine is compiled from its fetched bytes, not streamed.
|
|
22
|
+
- `rdfcStage` takes a Dataset or text again, as `canonicalize` does (0.20.0 took a Dataset only), and `canonicalHash` takes what `canonicalize` takes: a Dataset, or text with `{format, baseIRI}`. Text is a string of characters: decode bytes as UTF-8 first; a `Uint8Array` is refused (owner decision 2026-10-27).
|
|
23
|
+
- `syntaxFor('onto.owl')` is `'rdfxml'` when no media type is given, as Jena's `RDFLanguages.filenameToLang` gives (0.20.0 gave `null`). `.owl` is not registered for RDF/XML; `.xml` still gives `null`, and `formatOfDocument` (the proved `decodeStage`) keeps to the registrations (owner decision 2026-10-27).
|
|
24
|
+
- SBM11: `openSbm11` admits any chunk size of 1 to 16 MiB that agrees with a file's length and chunk count (0.20.0 required 65,536, the size every packer writes). The checked-chunk cache keeps 16 MiB per file whatever the chunk size. Every byte is still checked against the manifest's Merkle roots (owner decision 2026-10-27).
|
|
25
|
+
- Fixed: `Dataset.query(q, { output })` (json, xml, csv, tsv, table) calls registered extension functions (SPARQL 1.1 §17.6), sync and async, as `Dataset.query(q)` does (https://github.com/danbri/factoidal/issues/1090).
|
|
26
|
+
- Fixed: `Dataset.add` and `Dataset.delete` take a quad in a graph named by a blank node (RDF 1.1 Concepts §4) (https://github.com/danbri/factoidal/issues/1094).
|
|
27
|
+
- Fixed: in a SPARQL Update, `IRI()` and `URI()` resolve a relative IRI against the `BASE` of the request (SPARQL 1.1 Query §17.4.2.8) (https://github.com/danbri/factoidal/issues/1095).
|
|
28
|
+
- Changed: a `GRAPH ?g { … }` block of an INSERT or DELETE template whose `?g` is unbound or not an IRI (for example a literal) adds and removes nothing; before, it used the default graph (SPARQL 1.1 Update §3.1.3) (https://github.com/danbri/factoidal/issues/1096).
|
|
29
|
+
- SBM11 trust anchor (https://github.com/danbri/factoidal/issues/1105): `openSbm11(source, { manifestSha256 })` refuses the open when the SHA-256 of `manifest.sbm11` is not the anchor (64 hex digits), naming the manifest and both digests; the anchor also binds the manifest fields that no read checks. `store.manifestSha256` (and `manifestSha256` in the answers of the wasm ops `sbm11Open` and `sbm11Verify`) gives the digest to pin. Every open also checks the manifest's term count against the dictionary header.
|
|
30
|
+
- Controlled Identifiers v1.0 (https://github.com/danbri/factoidal/issues/1100 items 7 and 9): new functions `cidValidate(doc)` and `cidRetrieveVerificationMethod(url, relationship, {fetch?, resolve?, now?, allowHttp?, allowPrivateFetch?})` in Node, Deno and the browser; the authorization server's fetch guard applies; a fetched document needs status 200 and a CID media type; with `now`, an expired or revoked method is refused. New wasm ops `cidValidate`, `cidRetrieve`; `capabilities().cid`.
|
|
31
|
+
- New command `factoidal xmpp-http-proxy --xmpp JID --xmpp-password-file FILE --target JID/RESOURCE` (https://github.com/danbri/factoidal/issues/1107, step 2): a local HTTP listener on 127.0.0.1 (default port 8091; `--host` with any other address is refused) whose every request goes over XMPP (XEP-0332) to the one target JID, so a file browser, rclone, cadaver or curl reaches `webdav-serve --xmpp` behind NAT through `http://127.0.0.1:PORT/`. Options `--xmpp-host`, `--xmpp-port`, `--xmpp-tls`, `--xmpp-ca`, `--xmpp-resource` (default `proxy`), `--timeout SECONDS` (1 to 3,600, default 60). Requests go to `--target` only; an answer from another JID is dropped; `Authorization` passes unchanged and the proxy adds no credential; `Proxy-Authorization` and hop-by-hop headers are not forwarded; 400 for an absolute-form request target, 405 for CONNECT, 413 for a body above 16 MiB, 503 for more than 16 waiting requests, 504 on timeout, 403 when the proxy's JID is not on the server's allow list. Lean `XMPP/HttpRequester.lean`; new wasm ops `xmppClientRequest`, `xmppClientTick` and the requester mode of `xmppClientOpen` (`proxyTarget`); `xmppClientFeed` and `xmppClientClose` answer `responses`. Not yet: a `Host`/`Origin` check on the listener (https://github.com/danbri/factoidal/issues/1124), an expected failure. Test: `node tests/webdav/xmpp-proxy.mjs` (litmus 104 of 104, rclone and cadaver through the proxy).
|
|
32
|
+
- **Breaking:** listeners without authentication refuse browser reach from other sites (https://github.com/danbri/factoidal/issues/1124; row C7 of `docs/designissues/2026-10-28-feature-combination-safety.md`). (1) `lws-serve --insecure-no-auth` sends no CORS fields by default (before: it echoed any `Origin`, so any web page could read and write the storage); `--cors-origin ORIGIN` still names pages, and config `cors: true` with `insecureNoAuth` stops the start. (2) `lws-serve --insecure-no-auth`, `solid-serve --insecure-no-auth`, `webdav-serve` (with or without `--users`) and `factoidal serve` answer 421 Misdirected Request when `Host` names no name of the listener (DNS rebinding), and 403 to a request with a method other than GET, HEAD or OPTIONS whose `Origin` is another origin (a cross-site form POST of `update=DROP ALL`). Names: `localhost`, `127.0.0.1`, `[::1]`, the `--host` address, the interface addresses and the base IRI host. New repeatable options `--allow-host NAME` and `--allow-origin ORIGIN` on the four commands (config members `allowHosts`, `allowOrigins`; for `serve` and `webdav-serve` the host passes `localGuard`). Requests with no `Origin` (curl) and same-origin requests work as before. Lean `HTTP/LocalGuard.lean` (theorems `localGuard_foreign_host`, `localGuard_cross_origin_write`, `localGuard_admits_safe`, `localGuard_admits_no_origin`), `LWS.Cors.policyFor` (`insecure_no_echo`), `HTTP.Cors.decorate_no_credentials`, `StoreEndpoint.handle_guard_refused`. Still open, expected failures under the same issue: `solid-serve` echoes any `Origin` (reads) and sends `Access-Control-Allow-Credentials: true`. Test: `test/local-guard-1124.test.mjs`.
|
|
33
|
+
- One writing process per storage directory (https://github.com/danbri/factoidal/issues/1135): `lws-serve DIR`, `solid-serve DIR` and every `webdav-serve` local mount (and `openLwsStorage({dir})`, `openSolidStorage({dir})`) hold the lock file `DIR/lws.lock` (format `lws-lock-1`: process id, command, start time, host name, a random token of the process) while they run; `close()`, SIGINT, SIGTERM and the end of the process remove it. A start whose lock names a process that runs, a lock of another host, a file that is not a lock, and a second open in the same process are refused (exit 1, the message names the lock file and the holder). A lock whose process does not run is taken over with a warning. Lean `LWS/DirLock.lean` (`decide`; theorems `decide_alive_refuses`, `decide_otherHost_refuses`, `decide_sameInstance_refuses`, `decide_replace_dead`, `decide_create_absent`); new pure wasm op `lwsDirLock`. Test: `node tests/lws/dir-lock.mjs`.
|
|
34
|
+
- Encryption at rest stays off by default; every start of `lws-serve`, `solid-serve` and each `webdav-serve` local mount now says on standard error whether the storage is sealed (and its key id) or written in clear text, and how to make a key (owner, 2026-10-28: "For our lws encourage attention to encryption at rest also"). New command `factoidal lws-keygen [--json]` prints a new 32-byte storage key (64 hex digits) and its key id. New pure wasm ops `lwsAtRestNotice` and `lwsStorageKeyNew`; `openLwsStorage` and `openSolidStorage` answer `notices` and `lockFile`. Docs: section "Encryption at rest" of the hub servers page and of this README. Not yet: a key change, and sealing an existing clear storage (https://github.com/danbri/factoidal/issues/1136, expected failure in `tests/lws/encryption.mjs`).
|
|
35
|
+
- Pipelines as acyclic graphs with named ports (https://github.com/danbri/factoidal/issues/1130, stage 1b). `graph({ inputs, nodes: [{ node, from }] })`: each node reads one source per input port, a graph input (`{ input: k }`) or the output of an earlier node (`{ node: j }`); one output can feed many nodes. The engine checks the graph (new wasm op `graphCheckNodes`, Lean `graphIssues`) and `graph()` refuses a graph with an issue: an unknown graph input, a reference to the node itself or to a later node (a cycle), a wrong number of sources, or a wire that neither equals the port's wire nor converts to it (`FactoidalError` code `op.failed`, the list in `err.issues`; with an older wasm, code `op.unknown`). `run(graph, inputs[])` runs the nodes in order and answers `graph` (the last value), `values` (every node's value), one evidence record per node with `node` and `from`, and the engine's union of the named assumptions. `node('datasetMerge')`: the first stage with two ports (`left`, `right` on `Dataset`), status `proved`, contract `DatasetDen.IsMerge`; `node.ports` lists the ports of every node. New function `datasetMerge(a, b)` and wasm op `datasetMerge`: the RDF merge of two datasets (RDF 1.1 Semantics section 5.2), with the blank nodes of each input kept apart (Lean theorem `mergeDatasets_isMerge`). `capabilities()` reports `graphCheck` and `datasetMerge`. `pipe()` is unchanged; it refuses a node with more than one port. Test: `node --test test/pipeline-graph-1130.test.mjs`.
|
|
36
|
+
- `webdav-serve CONFIGDIR --xmpp JID --xmpp-password-file FILE --xmpp-allow JID[,JID]` (https://github.com/danbri/factoidal/issues/1107, step 1): the WebDAV server connects out to an XMPP server as JID and answers HTTP over XMPP (XEP-0332) requests from the allowed bare JIDs; it opens no port, so a server behind NAT is reachable. Options `--xmpp-host`, `--xmpp-port` (no DNS SRV lookup), `--xmpp-tls` (direct TLS, XEP-0368; STARTTLS is not implemented), `--xmpp-resource` (default `dav`). It prints `xmpp:JID/RESOURCE`. Lean: the client half of RFC 6120 (`XMPP/ClientStream.lean`: stream, SASL SCRAM-SHA-256 with the server signature checked, binding, `service-unavailable` for an IQ that is not HTTP); `HttpOverXmpp.step` split into `plan` and `finish` with no change of behaviour. New wasm ops `xmppClientOpen`, `xmppClientFeed`, `xmppClientAnswer`, `xmppClientClose`. The JID allow list is the access control on this path, so the start is refused when `--xmpp-allow` is missing or names a JID with a resource, when the transport is plain TCP to a host that is not loopback, or when a local mount is writable and `--dangerously-allow-writable-without-login` is not given (https://github.com/danbri/factoidal/issues/1125); `--xmpp-ca FILE` trusts a certificate for `--xmpp-tls`. Not yet: a login on the XMPP path (https://github.com/danbri/factoidal/issues/1125) and a login limit keyed by JID (https://github.com/danbri/factoidal/issues/1128), both expected failures. No file browser speaks XEP-0332; the local proxy is the next step. Test: `node tests/webdav/xmpp.mjs`.
|
|
37
|
+
- `webdav-serve --users` limits failed logins by default (https://github.com/danbri/factoidal/issues/1119). Owner, 2026-10-28: "default to protection but have a —dangerously-disable-ratelimits option". After 5 failed logins in 60 s for one user name (known or not) or from one client address (IPv6 by its /64), a request with credentials gets 429 with `Retry-After`, with no PBKDF2 derivation and no cache read, also for the right password, until the window ends. A success resets no counter; at most 10,000 counters, and a blocked one is never evicted. New options `--login-limit N` (1 to 1,000), `--login-window SECONDS` (1 to 86,400) and `--dangerously-disable-ratelimits` (no limit; a warning on standard error at start). Lean `WebDAV/Login.lean` (`Failures`, `limitsOf`, theorem `gate_no_derive_of_blocked`); `webdavOpen` takes `login.limit`, `login.window`, `login.disableRateLimits`; `ready` carries `warnings`; a `webdavStep` request carries `remoteAddress`.
|
|
38
|
+
- `webdav-serve`: a PUT with no Content-Type (or an empty one), and a LOCK that makes an empty resource, now store the media type of the target's name: the extension of the last segment, in ASCII lower case, by a fixed table (text, Markdown, CSV, HTML, JSON, JSON-LD, RDF syntaxes, SPARQL, images, PDF, `.desktop`, `.url`, `.webloc`, ...); a name with no known extension gets `application/octet-stream` as before, and a name that starts with `._` (macOS AppleDouble) gets `application/applefile` (RFC 1740). A Content-Type the client sends is kept. Before, every such resource was `application/octet-stream`, and GNOME Files (gvfs, which sends no Content-Type) then showed a saved text file as binary (https://github.com/danbri/factoidal/issues/1122). Lean `WebDAV/MediaType.lean` (theorem `mediaTypeForName_cases`, a `#guard` per rule), used by `WebDAV/Server.lean`.
|
|
39
|
+
- Deno: a `deno compile` executable of the CLI packs in process; before, `pack` and `sample-store` re-ran the executable with `run ...` (meant for the `deno` command) and stopped with `unknown command "run"` (`bin/pack-host.mjs`, `denoReexec`).
|
|
40
|
+
- Tools: `tools/build-exe.sh OUTDIR` builds the CLI as one executable per platform with `deno compile` (Linux, macOS and Windows on x86_64 and arm64), the engine wasm included; `tools/check-exe.sh EXE` checks one (help, version, sample-store and a query, serve over HTTP, the desktop WebDAV harness). Stage 1 of `docs/designissues/2026-10-28-single-file-app.md`.
|
|
41
|
+
- LWS client, three options the lws-driver adapter answered `Unsupported` (https://github.com/danbri/factoidal/issues/1116), Lean `LWS/Client/Machine.lean`: (1) new session option `requireHttps` (`createLwsClient({ requireHttps: true })`, `factoidal lws-client --require-https`; default off): the client sends no request to a URL whose scheme is not `https`, in any phase (the operation, discovery, metadata, the authorization server, the token endpoint); the operation fails with code `insecureTransport` and the URL. Theorems `step_requireHttps`, `start_requireHttps`. (2) Operation member `ifNoneMatchTags` (CLI `--if-none-match TAG[,TAG]`): `If-None-Match` with entity tags (RFC 9110 §13.1.2) on `update`, `patch`, `delete`, `updateLinkset` and `move`, alone (precondition `ifNoneMatch`, the default when only tags are given) or with `If-Match` or `If-Unmodified-Since`; a malformed tag, or `*` with another tag, is `badArguments`. (3) Operation member `policies`: `requestAccess` and `grantAccess` write a document with any number of policies (`[{actions, assignee, targets, constraint}]`, CLI `--policies JSON`); the single-policy members still give one policy. The driver adapter maps `allowInsecureHttp: false`, a non-`*` `ifNoneMatch` (also with `ifMatch`, and on `patch`) and multi-policy documents to them; it answers `Unsupported` for no option now.
|
|
42
|
+
- `webdav-serve` has its own login and TLS (https://github.com/danbri/factoidal/issues/1108). `--users FILE`: every request needs HTTP Basic (RFC 7617) for a user of FILE, else 401 with `WWW-Authenticate: Basic realm="factoidal webdav", charset="UTF-8"`. A users file line is `NAME:pbkdf2-sha256:ITERATIONS:SALT_B64:HASH_B64` (PBKDF2-HMAC-SHA256); a malformed line stops the start with the file name and line number. New command `factoidal webdav-user FILE NAME [--iterations N]` reads the password from standard input (no echo on a terminal), makes a 16-byte salt, and adds or replaces the line (default 600,000 iterations, at least 100,000; mode 0600, written by rename). PBKDF2 runs in WebCrypto: the engine answers `need` (`op: 'pbkdf2-hmac-sha256'`) and the host gives the answer of `answerCipherRequest` to `webdavResume`; `@factoidal/core/cipher-host` has the new op. A successful login is cached for 300 s in the engine's memory. Without TLS, `--users` is refused unless the address is loopback. `--cert FILE --key FILE` (PEM) serve HTTPS. Lean `WebDAV/Login.lean`; new wasm op `webdavUser`; `webdavOpen` takes `login: {users, tls, host}`. Expert review requested: https://github.com/danbri/factoidal/issues/1117.
|
|
43
|
+
- `webdav-serve`: dead properties and locks survive a restart (https://github.com/danbri/factoidal/issues/1078). The engine keeps them in `CONFIGDIR/.webdav-state.json` (JSON with `format` and `version` members; form in Lean `WebDAV/State.lean`). At start it reads the file and drops locks that have expired; a file that is not JSON, has an unknown format or version, or a malformed member stops the start with a message that names the file. After a request that changes them, the `webdavStep`/`webdavResume` response carries `write`, and the host replaces the file (new file in the same directory, mode 0600, then rename) before it sends the response. `webdavOpen` takes `now`.
|
|
44
|
+
- `query(sparql, { sparql12: true, entail })` and `Dataset.query` with both options: the engine closes the dataset under the regime and runs the SPARQL 1.2 evaluator (new wasm op `datasetQuery12Entail`). Before, the combination was refused with `option.conflict`. `output` with SPARQL 1.2 is still refused (https://github.com/danbri/factoidal/issues/1109).
|
|
45
|
+
- LWS client, three interop defects found with Touchstone's target servers (https://github.com/danbri/factoidal/issues/1115), Lean `LWS/Client/Machine.lean`: (1) new option `legacyChallenge` (`createLwsClient({ legacyChallenge: true })`, `factoidal lws-client --legacy-challenge`, session member `legacyChallenge`; default off) follows a 401 with only `Bearer as_uri="…", realm="…"` (the drafts before 2026-10-19; css, sparq-lws) when the realm has the origin of the requested URL and contains it; it skips checks 1 and 2 of lws10-core §Discovery Validation by a Client. (2) An access token now also goes to the service endpoints of its storage description that have the origin of the requested URL (lws-drupal's type index is outside the StorageRoot path); a token entry of the session has a new member `endpoints`. (3) `subscribe` with `endpoint` (CLI `--endpoint URL`) POSTs the subscription to the named notification endpoint with no storage-link discovery (lwsd). The driver adapter `deploy/lws-client-adapter/` passes both.
|
|
46
|
+
- `webdav-serve`: two `did-key` mounts that name one key file that does not exist yet get one key. Before, the engine made a key for each mount, the host wrote the first and the start stopped with `EEXIST` on the second (Lean `Wasm/Ops/WebDAV.lean`, `davKeyPathKey`; `alice.jwk` and `./alice.jwk` are one file).
|
|
47
|
+
- Docs: hub page "RDF 1.2 use cases in JSON-LD-star" (`docs/web/hub/rdf12-jsonld-usecases.md`): annotation and reification use cases, each read from JSON-LD-star and queried with SPARQL 1.2; `entail` with `sparql12` is refused (https://github.com/danbri/factoidal/issues/1109).
|
|
48
|
+
- Docs: hub page "WebDAV" (`docs/web/hub/webdav.md`): quick start, mount files, a remote LWS storage with did:key login, login and HTTPS options, Tailscale and SSH tunnels, the XMPP gap (https://github.com/danbri/factoidal/issues/1107), file browser steps; the listener has no login or TLS (https://github.com/danbri/factoidal/issues/1108).
|
|
49
|
+
- JWE (RFC 7516 Compact Serialization), decryption and encryption, Lean 4 (https://github.com/danbri/factoidal/issues/1085): new wasm ops `jweDecrypt` and `jweEncrypt`. `alg` RSA-OAEP, RSA-OAEP-256, ECDH-ES, ECDH-ES+A128KW, ECDH-ES+A256KW, A128KW, A256KW, dir; `enc` A128GCM, A256GCM, A128CBC-HS256, A256CBC-HS512. Refused before any key is read: RSA1_5, PBES2-*, every other `alg` or `enc`, `zip`, `crit`. Every failure after that is the one error "decryption failed". RSA, P-256 ECDH and X25519 run in the wasm (HACL*); every AES operation is a `need` request that the host answers with WebCrypto (`runWithCipher` from `@factoidal/core/cipher-host`). Byte fields are standard base64. Lean: `JOSE/Jwe.lean`, `JOSE/JweKey.lean` (private JWKs), `JOSE/JweTests.lean`, HMAC-SHA-512 in `Crypto/Hmac.lean`; nested JWT (RFC 7519 §5.2) in Lean as `Jwe.decryptThenVerify`. Tests: `lake exe l4jwe-probe`, `tests/jose/jwe-wasm.mjs`, `tests/jose/jwe-diff.mjs`.
|
|
50
|
+
- XML Encryption 1.1 (https://github.com/danbri/factoidal/issues/1084): new wasm ops `xmlDecrypt` and `xmlEncrypt` (`{document, keys, allowCbc, answers}`; `{document, recipient, algorithm, keyTransport, keyWrap, content, saml, random, answers}`), answering `{document, canonical}` or `{need: {op, key, iv, aad, data}}`. AES runs in the host's WebCrypto through `runWithCipher` (`@factoidal/core/cipher-host`); RSA-OAEP, P-256 and X25519 run in the wasm (HACL\*); every decision is in Lean (`XML/Enc.lean`). Allowlist: `aes128-gcm`, `aes256-gcm` (`aes128-cbc`, `aes256-cbc` only with `allowCbc`), `rsa-oaep-mgf1p`, `xmlenc11#rsa-oaep` (SHA-1/mgf1sha1 or SHA-256/mgf1sha256), `kw-aes128`, `kw-aes256`, ECDH-ES (P-256, X25519) and `xmldsig-more#x25519` with ConcatKDF SHA-256; `rsa-1_5`, `CipherReference` and remote `RetrievalMethod` are refused. Every failure after the structure check is one error, `XML decryption failed`. Keys are JWKs (`kid` is the `ds:KeyName`). Checked against Apache Santuario 4.0.4 in both directions and the W3C XML Encryption 1.1 test cases (`tests/xmlenc/`).
|
|
51
|
+
- Package functions `xmlDecrypt`, `xmlEncrypt`, `jweDecrypt` and `jweEncrypt` (all hosts: Node, Deno, browsers) wrap the four ops: each answers the engine's AES requests with WebCrypto (`runWithCipher`) and returns the result, or throws a `FactoidalError`. `jweDecrypt` gives the plaintext as a `Uint8Array`; `jweEncrypt` takes a `Uint8Array` or a string (written as UTF-8). The random octets come from `crypto.getRandomValues`. Test: `test/xmlenc-jwe.test.mjs` (round trips, and node:crypto in both directions).
|
|
52
|
+
- SAML (https://github.com/danbri/factoidal/issues/1073): Lean `SAML.validateWith` decrypts a `saml:EncryptedAssertion` subject token and a `saml:EncryptedID` subject with configured keys; the decrypted assertion must carry its own valid signature. `authz-serve --saml-decrypt-key FILE` (repeatable, a private JWK: RSA, EC P-256, OKP X25519 or oct; also `lws-serve --authz`), config member `samlKeys`, gives the server its decryption keys. RSA-OAEP, P-256 and X25519 run in the wasm (HACL*); each AES operation comes back from `authzStep` as `{cipher: {op, key, iv, aad, data}}`, the Node host answers it with WebCrypto and passes `{request, answer}` to `authzFetched`. Without keys, encrypted assertions are refused. `saml:EncryptedAttribute` stays refused.
|
|
53
|
+
|
|
54
|
+
## 0.20.0 (2026-10-27)
|
|
55
|
+
|
|
56
|
+
- **Breaking:** in a checked pipeline (`node`, `pipe`, `run`), each node now puts a value of its output wire on the wire: text after a writer, the rows after SELECT, a boolean after ASK, the report after `shaclValidator`, `[canonical, sha256]` after `rdfcStage`, the result after `rdfsExplainGraphStage`. Before, such a node left its input Dataset on the wire, so a well-typed reader → writer → reader pipeline failed at run time. `run()` rejects a runner value that is not of the node's `wireOut`. `rdfcStage` as a node takes a Dataset only (https://github.com/danbri/factoidal-core/pull/155).
|
|
57
|
+
- `node()` keeps its own copy of its settings (frozen at every depth; a Dataset is cloned). A change to the caller's objects after `node()` no longer changes what runs while the evidence keeps the old `settingsDigest`. `run()` rejects a node whose settings changed through `node.params` (https://github.com/danbri/factoidal-core/pull/155).
|
|
58
|
+
- **Breaking:** `canonicalHash(dataset)` keeps graph names: it is the SHA-256 of the text `canonicalize()` answers. 0.17.0 to 0.19.0 dropped graph names, so datasets that differ only in graph names got one hash. New `asDefaultGraph(dataset)`; `canonicalHash(asDefaultGraph(g))` gives a graph's hash whatever its name (https://github.com/danbri/factoidal-core/pull/155).
|
|
59
|
+
- SBM11 stores opened with `openSbm11` (and the wasm ops `sbm11Open` / `sbm11Query`) are checked against their Merkle roots by default: the open admits each file's `.merkle` sidecar against `manifest.sbm11`, and every byte a query decodes comes from a 65,536-byte chunk checked by its Merkle proof. A changed data file, sidecar or Merkle reference in the manifest fails the open or the query with an error that names the file (before, a changed `dict.gdv1` was answered as data). `openSbm11(source, { verify: 'none' })` keeps the earlier length-only check; `store.verify` says which mode is in use. Measured cost: within run-to-run noise on stores of 1 to 6 MB (https://github.com/danbri/factoidal-core/pull/157).
|
|
60
|
+
- Parse is faster again after the IRI check of 0.19.0 (https://github.com/danbri/factoidal/issues/1057): the check tests bytes, and a repeated IRI is not checked again. Parse of 100,000 triples is now about 3% slower than 0.18.0 (it was 10% to 19%). The IRIs that are accepted and refused do not change (https://github.com/danbri/factoidal-core/pull/158).
|
|
61
|
+
- CID 1.0 (Controlled Identifiers, W3C Recommendation 2025-05-15): new Lean module `L4Factoidal/CID/` with the section 2 data model (one named error per MUST), Multikey and JsonWebKey material, the five verification relationships, Fragment Resolution, Retrieve Verification Method and the section 3.5 processing errors. The CID context is vendored. Authorization server, CID suite (lws10-authn-ssi-cid): the subject document is checked as a whole and the method is found by Retrieve Verification Method with `authentication`; `expires`/`revoked` need a time zone; `Accept` starts with `application/cid`. VC Data Integrity (Lean): `verificationMethod` can be a method in a controlled identifier document (not only an Ed25519 did:key), and it must be in the relationship that `proofPurpose` names. No npm functions yet (https://github.com/danbri/factoidal/issues/1100, https://github.com/danbri/factoidal-core/pull/161).
|
|
62
|
+
- `parse` takes the option `rdfstar` for JSON-LD, as the 0.18.0 changelog says: `parse(text, { format: 'jsonld', rdfstar: true })` reads `@triple`, `@reifies` and `@annotation` as RDF 1.2 triple terms and reifiers, as `jsonldToRdf` does. 0.18.0 and 0.19.0 refused the option with `option.unknown`. `rdfstar` with another format, or with `lenient`, is refused with `option.conflict` (https://github.com/danbri/factoidal/issues/1091, https://github.com/danbri/factoidal-core/pull/152).
|
|
63
|
+
- Docs: hub page "JSON-LD-star and JSON-LD 1.3" (`docs/web/hub/jsonld-star.md`), with live cells on 0.19.0; the hub pages load 0.19.0 (were 0.16.0).
|
|
64
|
+
- Value matching (https://github.com/danbri/factoidal/issues/1052): new `sameValue(a, b)` is SPARQL 1.1 `=` on two terms (sections 17.3, 17.4.1.7), decided by the engine (Lean `SPARQL/SameValue.lean`). It answers `true`, `false`, or `null` for incomparable (the type error of `=`). `Dataset.has`, `match` and `delete` take `{match: 'value'}`: a literal object then matches each object literal of the same value (`"1"^^xsd:int` finds `"01"^^xsd:integer` and `"1.0"^^xsd:decimal`), through an index of the literals by value. Term matching stays the default. New wasm op `sameValue`; `datasetMatch`, `datasetMatchCount` and `datasetDeleteQuadsJson` take an optional options argument.
|
|
65
|
+
- Syntax names (https://github.com/danbri/factoidal/issues/1053): new `syntaxFor(fileName)` and `syntaxFor({fileName, mediaType})` give the syntax of a document from the W3C registrations (Lean `RDF/MediaType.lean`), or `null`. Each row of `formats()` has `mediaTypes` and `extensions` from the same table (README format table: two new columns). New wasm op `syntaxFor`.
|
|
66
|
+
- IRIs, vocabularies, lists and containers (https://github.com/danbri/factoidal/issues/1051 items 4 to 6): new `splitIri(iri)` (namespace and the longest XML NCName suffix), `isNCName(text)`, `vocabularies()` (rdf, rdfs, xsd and owl with their terms; foaf, dcterms, dc and schema namespaces), `memberIri(n)` and `memberOrdinal(iri)` (`rdf:_n`). Dataset methods `listItems(head, graph?)` (refuses a list that is not well formed, with the reason), `containerItems(node, graph?)` and `addList(items, graph?)`. Lean `RDF/IriSplit.lean`, `RDF/VocabularyTable.lean`, `RDF/Collections.lean`. New wasm ops `splitIri`, `isNCName`, `vocabularies`, `memberIri`, `memberOrdinal`, `datasetListItems`, `datasetContainerItems`, `datasetAddList`. These need a wasm built from this source.
|
|
67
|
+
- Literal values (https://github.com/danbri/factoidal/issues/1051 items 1 to 3). New functions `literalValue(term)`, `isValidLiteral(term)`, `canonicalLiteral(term)` and `literalFromValue(value, {datatype?})`, synchronous after `ready()` (API.md "Literal values"). `literalValue` gives the XSD 1.1 lexical-to-value mapping of a literal for every XSD built-in the engine has (numeric types and the derived integer types with their bounds, boolean, the string types, date/time and `g*` types, durations, binary, lists) and for `rdf:langString`, `rdf:dirLangString`, `rdf:XMLLiteral`, `rdf:JSON`: `{datatype, recognized, valid, value?, canonical?}`, with the value as JSON-safe data (a number that a JavaScript number does not hold exactly is a decimal string). `valid: false` is an ill-typed literal; `recognized: false` is a datatype the engine does not have. `literalFromValue` maps a boolean, number, bigint, string or Date to a typed literal with one engine table (safe integer `xsd:integer`, other number `xsd:double`, bigint `xsd:integer`, Date `xsd:dateTime` in UTC), or to a named datatype (`{datatype: 'int'}`) when the value is in its value space, else a `FactoidalError` with the new code `literal.value`. New wasm ops `literalValue` and `literalFromValue` (Lean `L4Factoidal/RDF/LiteralValue.lean`, `Wasm/Ops/LiteralValue.lean`). An extension function may now answer a `Date` (an `xsd:dateTime`).
|
|
68
|
+
- The canonical form of an `xsd:float` or `xsd:double` value is now the XSD 1.1 Part 2 §3.3.4.2/§3.3.5.2 form (algorithm §E.1): the fewest significant digits that map back to the value. `"0.1"^^xsd:double` has the canonical form `1.0E-1`; before, it was the exact binary expansion `1.000000000000000055511151231257827021181583404541015625E-1`. It is the `canonical` member of `literalValue` and RML's canonical form of a double (RML-Core natural mapping). SPARQL casts do not change (`xsd:double("0.1")` is still `"0.1"^^xsd:double`). Theorems `double_canonical_roundTrip`, `float_canonical_roundTrip`, `integer_canonical_roundTrip` (`XSD/CanonicalTheorems.lean`).
|
|
69
|
+
|
|
5
70
|
## 0.19.0 (2026-10-27)
|
|
6
71
|
|
|
7
72
|
- Notice (owner decision, 2026-10-27): "EXPERIMENTAL UNREVIEWED CODE - USE WITH CARE - DO NOT CREATE PUBLIC SERVICES USING THIS WITHOUT CAREFUL INDEPENDENT EXPERT REVIEW." It is at the top of both READMEs and the hub, in the package description, in every `factoidal` help text, and on standard error when `serve`, `lws-serve`, `solid-serve`, `solid-client`, `lws-client`, `authz-serve`, `authz-keys`, `xmpp-serve` or `webdav-serve` starts.
|
package/README.md
CHANGED
|
@@ -38,10 +38,14 @@ every consumer accepts one, so a pipeline does not re-parse between steps.
|
|
|
38
38
|
An unknown option key throws. With no `format`, the format is inferred and
|
|
39
39
|
reported in `dataset.format`; an explicit `format` is never overridden.
|
|
40
40
|
RDF 1.2 input uses `turtle12`, `ntriples12`, `nquads12` or `trig12`.
|
|
41
|
+
RDF 1.2 in JSON-LD (JSON-LD-star: `@triple`, `@reifies`, `@annotation`,
|
|
42
|
+
`@direction`) needs the option `rdfstar`: `jsonldToRdf(text, { rdfstar: true })`
|
|
43
|
+
and `dataset.serialize({ format: 'jsonld', rdfstar: true })`.
|
|
41
44
|
|
|
42
45
|
## What it does
|
|
43
46
|
|
|
44
47
|
- Syntaxes: N-Triples, N-Quads, Turtle, TriG, RDF/XML, JSON-LD; RDF 1.1 and 1.2
|
|
48
|
+
(RDF 1.2 in JSON-LD through the JSON-LD-star draft, option `rdfstar`)
|
|
45
49
|
- SPARQL 1.1 and 1.2 query and update; GeoSPARQL functions, text search,
|
|
46
50
|
JavaScript extension functions, SERVICE
|
|
47
51
|
- Reasoning: RDFS, OWL 2 RL closure, restricted checked OWL consistency,
|
|
@@ -56,8 +60,12 @@ RDF 1.2 input uses `turtle12`, `ntriples12`, `nquads12` or `trig12`.
|
|
|
56
60
|
self-issued CID credentials and SAML 2.0 assertions)
|
|
57
61
|
- Security: JWS/JWT (ES256, RS256, EdDSA), XML Signature verification in the
|
|
58
62
|
SAML Core profile (Exclusive XML Canonicalization 1.0, rsa-sha256), with
|
|
59
|
-
HACL\* primitives. XML Encryption and JWE
|
|
60
|
-
|
|
63
|
+
HACL\* primitives. XML Encryption 1.1 and JWE (RFC 7516), decryption and
|
|
64
|
+
encryption (`xmlDecrypt`, `xmlEncrypt`, `jweDecrypt`, `jweEncrypt`):
|
|
65
|
+
RSA-OAEP, P-256 and X25519 run in the wasm (HACL\*), AES runs in the host's
|
|
66
|
+
WebCrypto. The authorization server decrypts encrypted SAML assertions
|
|
67
|
+
with `--saml-decrypt-key`. Native AES on CPUs other than x86-64:
|
|
68
|
+
https://github.com/danbri/factoidal/issues/677
|
|
61
69
|
- A persisted store with a delta log for updates (early stage)
|
|
62
70
|
- Terms: a JavaScript extension function can answer with an exact term
|
|
63
71
|
(`{value: "007", datatype: XSD + "integer"}`, an RDF/JS term, a triple
|
|
@@ -118,19 +126,19 @@ and marks those steps in the evidence. Details: API.md, "Stages as nodes" and
|
|
|
118
126
|
Formats (generated from `formats()`):
|
|
119
127
|
|
|
120
128
|
<!-- formats-table:start (generated by test/formats-table.test.mjs) -->
|
|
121
|
-
| format | read RDF 1.1 | read RDF 1.2 | write RDF 1.1 | write RDF 1.2 | engine |
|
|
122
|
-
|
|
123
|
-
| nquads | yes | yes | yes | yes | core |
|
|
124
|
-
| ntriples | yes | yes | yes | yes | core |
|
|
125
|
-
| turtle | yes | yes | yes | yes | core |
|
|
126
|
-
| trig | yes | yes | yes | yes | core |
|
|
127
|
-
| rdfxml | yes | no | yes | no | core |
|
|
128
|
-
| jsonld | yes | no | yes | no | full |
|
|
129
|
-
| rdfc | no | no | yes | no | core |
|
|
130
|
-
| sparql-results-json | no | no | yes | yes | core |
|
|
131
|
-
| sparql-results-xml | no | no | yes | yes | core |
|
|
132
|
-
| sparql-results-csv | no | no | yes | yes | core |
|
|
133
|
-
| sparql-results-tsv | no | no | yes | yes | core |
|
|
129
|
+
| format | read RDF 1.1 | read RDF 1.2 | write RDF 1.1 | write RDF 1.2 | engine | media type | extension |
|
|
130
|
+
|---|---|---|---|---|---|---|---|
|
|
131
|
+
| nquads | yes | yes | yes | yes | core | `application/n-quads` | `.nq` |
|
|
132
|
+
| ntriples | yes | yes | yes | yes | core | `application/n-triples` | `.nt` |
|
|
133
|
+
| turtle | yes | yes | yes | yes | core | `text/turtle` | `.ttl` |
|
|
134
|
+
| trig | yes | yes | yes | yes | core | `application/trig` | `.trig` |
|
|
135
|
+
| rdfxml | yes | no | yes | no | core | `application/rdf+xml` | `.rdf` |
|
|
136
|
+
| jsonld | yes | no | yes | no | full | `application/ld+json` | `.jsonld` |
|
|
137
|
+
| rdfc | no | no | yes | no | core | | |
|
|
138
|
+
| sparql-results-json | no | no | yes | yes | core | | |
|
|
139
|
+
| sparql-results-xml | no | no | yes | yes | core | | |
|
|
140
|
+
| sparql-results-csv | no | no | yes | yes | core | | |
|
|
141
|
+
| sparql-results-tsv | no | no | yes | yes | core | | |
|
|
134
142
|
<!-- formats-table:end -->
|
|
135
143
|
|
|
136
144
|
Missing writers (HDT, N3, RDFa): https://github.com/danbri/factoidal/issues/883.
|
|
@@ -160,6 +168,7 @@ Codes (generated from `engineTables().errorCodes`):
|
|
|
160
168
|
| `term.invalid-iri` | <https://www.w3.org/TR/rdf12-concepts/#section-IRIs> | A term that must be an IRI is not one. |
|
|
161
169
|
| `term.invalid-langtag` | <https://www.w3.org/TR/rdf12-concepts/#section-Graph-Literal> | A language tag is not well-formed (BCP 47). |
|
|
162
170
|
| `term.invalid` | <https://www.w3.org/TR/sparql11-results-json/#select-encode-terms> | A term or quad given as JSON is not an RDF term or quad. |
|
|
171
|
+
| `literal.value` | <https://www.w3.org/TR/xmlschema11-2/#value-space> | The value is not in the value space of the named datatype, or the datatype is not one the engine has. |
|
|
163
172
|
| `query.syntax` | <https://www.w3.org/TR/sparql11-query/#grammar> | The SPARQL query does not match the SPARQL grammar. |
|
|
164
173
|
| `query.failed` | <https://www.w3.org/TR/sparql11-query/#sparqlDefinition> | The SPARQL query parsed but its evaluation was refused. |
|
|
165
174
|
| `update.syntax` | <https://www.w3.org/TR/sparql11-update/#grammar> | The SPARQL update does not match the SPARQL Update grammar. |
|
|
@@ -206,6 +215,38 @@ $ npx factoidal solid-serve DIR # Solid Protocol
|
|
|
206
215
|
$ npx factoidal authz-serve DIR --resource IRI --saml-idp idp.json # OAuth 2.0 authorization server
|
|
207
216
|
```
|
|
208
217
|
|
|
218
|
+
### Storage directories of `lws-serve`, `solid-serve` and `webdav-serve`
|
|
219
|
+
|
|
220
|
+
One process writes a storage directory
|
|
221
|
+
(https://github.com/danbri/factoidal/issues/1135). The server holds
|
|
222
|
+
`DIR/lws.lock` while it runs; a second server on the same directory is
|
|
223
|
+
refused, with a message that names the lock file and the process that holds
|
|
224
|
+
it. A lock left by a process that does not run (after a crash) is taken over
|
|
225
|
+
with a warning.
|
|
226
|
+
|
|
227
|
+
#### Encryption at rest
|
|
228
|
+
|
|
229
|
+
Off by default. Without a key the storage is written to disk in clear text,
|
|
230
|
+
and the server says so at start. To turn it on:
|
|
231
|
+
|
|
232
|
+
```console
|
|
233
|
+
$ export FACTOIDAL_LWS_KEY="$(npx factoidal lws-keygen)" # keep a copy of the key
|
|
234
|
+
$ npx factoidal lws-serve DIR --issuer IRI
|
|
235
|
+
lws-serve: encryption at rest is ON: the storage in DIR is sealed with ChaCha20-Poly1305 under key id ...
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
- Sealed (ChaCha20-Poly1305, HACL*): the snapshot `DIR/lws-state.json`; with
|
|
239
|
+
`--storage files`, every body file, the journal and `lws-files.json`.
|
|
240
|
+
- Not protected: file names, file sizes and write times; the lock file; the
|
|
241
|
+
storage in the memory of the running server; the key in its environment.
|
|
242
|
+
- A lost key is lost data. A wrong key, a changed file, a clear storage with
|
|
243
|
+
a key and a sealed storage without the key each stop the start.
|
|
244
|
+
- Start on an empty directory: a key change, and sealing an existing clear
|
|
245
|
+
storage, are not possible yet
|
|
246
|
+
(https://github.com/danbri/factoidal/issues/1136).
|
|
247
|
+
|
|
248
|
+
Details: https://danbri.github.io/factoidal/web/hub/servers/#encryption-at-rest
|
|
249
|
+
|
|
209
250
|
`npx factoidal --help` lists every command. Replacing Apache Jena Fuseki:
|
|
210
251
|
[docs/fuseki-replacement.md](https://github.com/danbri/factoidal-core/blob/main/npm/factoidal/docs/fuseki-replacement.md).
|
|
211
252
|
|
|
@@ -239,6 +280,8 @@ and the [assurance record](https://github.com/danbri/factoidal-core/blob/main/do
|
|
|
239
280
|
| JSON-LD 1.1 toRdf, expand, flatten | 910 / 0 / 910 |
|
|
240
281
|
| JSON-LD 1.1 compact, fromRdf | 298 / 0 / 300 (2 not run) |
|
|
241
282
|
| JSON-LD 1.1 Framing | 29 / 63 / 92 |
|
|
283
|
+
| JSON-LD-star draft (RDF 1.2), option `rdfstar` | 168 / 1 / 171 (2 local override; 170 of 171 with them) [J] |
|
|
284
|
+
| RDF 1.2 eval tests written as JSON-LD-star and read back | 140 / 0 / 140 [J] |
|
|
242
285
|
| RDFC-1.0 | 86 / 0 / 86 |
|
|
243
286
|
| OWL 2 RL | 126 / 0 / 126 |
|
|
244
287
|
| OWL 2 DL type-inconsistency | 116 / 11 / 128 (1 other) |
|
|
@@ -262,6 +305,7 @@ and the [assurance record](https://github.com/danbri/factoidal-core/blob/main/do
|
|
|
262
305
|
[L] 2026-10-22, https://github.com/danbri/factoidal-core/blob/2b9d2e39af53be31ed92463b58d3e132e65c4fe4/docs/designissues/2026-10-21-lws-touchstone-run.md (with server options, 220 of 221 pass in at least one configuration)
|
|
263
306
|
[S] 2026-10-21, the current W3C manifest (29 active tests), https://github.com/danbri/factoidal-core/blob/2b9d2e39af53be31ed92463b58d3e132e65c4fe4/docs/claude-rules/current-state.md
|
|
264
307
|
[C] 2026-10-21, validation test308, https://github.com/danbri/factoidal/issues/700
|
|
308
|
+
[J] 2026-10-27, tests of https://github.com/json-ld/json-ld-star/pull/61; the fail is a defective fixture (fromRdf tst07), https://github.com/danbri/factoidal-core/blob/2a656d7bfb4af329d77669afb5e62eb00625bd15/docs/designissues/2026-10-26-jsonld-star.md
|
|
265
309
|
|
|
266
310
|
## Limits
|
|
267
311
|
|
package/authz/server.mjs
CHANGED
|
@@ -19,7 +19,9 @@
|
|
|
19
19
|
// authzStep [handle, requestJson] -> { ok, response, changed }
|
|
20
20
|
// | { ok, resolve: { host }, changed }
|
|
21
21
|
// | { ok, fetch: { url, accept, address }, changed }
|
|
22
|
-
//
|
|
22
|
+
// | { ok, cipher: { op, key, iv, aad, data }, changed }
|
|
23
|
+
// authzFetched [handle, { url, status, body, now } | { host, addresses, now }
|
|
24
|
+
// | { request, answer }] -> { ok }
|
|
23
25
|
// authzKeys [handle, { nonce?, private? }] -> { ok, keys } | { ok, jwks }
|
|
24
26
|
// authzRotate [handle, { now, random }] -> { ok, changed, kid }
|
|
25
27
|
// authzOwns [handle, target] -> { ok, owns }
|
|
@@ -32,11 +34,17 @@
|
|
|
32
34
|
// address to connect to: this host connects to exactly that address (the name
|
|
33
35
|
// is not resolved again), sends no cookie or credential, follows no redirect,
|
|
34
36
|
// reads at most MAX_DOCUMENT_BYTES, and passes the status and the text back.
|
|
37
|
+
//
|
|
38
|
+
// CIPHER. `cipher` names one AES operation (an encrypted SAML subject token,
|
|
39
|
+
// issue 1073): this host runs exactly that WebCrypto call
|
|
40
|
+
// (`answerCipherRequest`, ../lib/cipher-host.mjs) and passes the request and
|
|
41
|
+
// the answer back. It reads no token and chooses no key or algorithm.
|
|
35
42
|
|
|
36
43
|
import { closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, writeSync } from 'node:fs'
|
|
37
44
|
import { join, resolve } from 'node:path'
|
|
38
45
|
import { loadEngine } from '../bin/engine.mjs'
|
|
39
46
|
import { storageKeyText } from '../lws/persist.mjs'
|
|
47
|
+
import { answerCipherRequest, MAX_CIPHER_REQUESTS } from '../lib/cipher-host.mjs'
|
|
40
48
|
|
|
41
49
|
/** The ops this host needs from the engine. */
|
|
42
50
|
export const AUTHZ_OPS = ['authzOpen', 'authzStep', 'authzFetched', 'authzKeys', 'authzRotate', 'authzOwns', 'authzClose']
|
|
@@ -171,6 +179,8 @@ async function fetchFrom (url, accept, address) {
|
|
|
171
179
|
* default: echo any valid request Origin (issue 923)
|
|
172
180
|
* @param {{entityId: string, jwk: object}[]} [options.samlIdps] the SAML 2.0 identity providers
|
|
173
181
|
* whose signed assertions are accepted, each with its RSA public JWK (issue 926)
|
|
182
|
+
* @param {object[]} [options.samlKeys] private JWKs (RSA, EC P-256, OKP X25519, oct) that decrypt
|
|
183
|
+
* saml:EncryptedAssertion and saml:EncryptedID in a SAML subject token (issue 1073)
|
|
174
184
|
* @param {object} [options.engine] an engine to reuse
|
|
175
185
|
*/
|
|
176
186
|
export async function openAuthorizationServer (options = {}) {
|
|
@@ -197,6 +207,9 @@ export async function openAuthorizationServer (options = {}) {
|
|
|
197
207
|
// SAML 2.0 identity providers (issue 926): [{ entityId, jwk }], passed
|
|
198
208
|
// unread; the engine checks each (Wasm/Ops/Authz.lean samlIdpsOf?).
|
|
199
209
|
if (options.samlIdps !== undefined) config.samlIdps = options.samlIdps
|
|
210
|
+
// SAML decryption keys (issue 1073): private JWKs, passed unread; the
|
|
211
|
+
// engine checks each (Wasm/Ops/Authz.lean samlKeysOf?).
|
|
212
|
+
if (options.samlKeys !== undefined) config.samlKeys = options.samlKeys
|
|
200
213
|
const key = storageKeyText(options.storageKey)
|
|
201
214
|
if (key !== undefined) config.storageKey = key
|
|
202
215
|
if (dir !== null) {
|
|
@@ -234,6 +247,7 @@ export async function openAuthorizationServer (options = {}) {
|
|
|
234
247
|
*/
|
|
235
248
|
async step (request) {
|
|
236
249
|
let needs = 0
|
|
250
|
+
let ciphers = 0
|
|
237
251
|
for (;;) {
|
|
238
252
|
const now = nowSeconds()
|
|
239
253
|
const record = { ...request, now, random: randomBase64(112) }
|
|
@@ -243,6 +257,13 @@ export async function openAuthorizationServer (options = {}) {
|
|
|
243
257
|
const r = envelope.response
|
|
244
258
|
return { status: r.status, headers: Array.isArray(r.headers) ? r.headers : [], body: typeof r.body === 'string' ? r.body : '' }
|
|
245
259
|
}
|
|
260
|
+
if (envelope.cipher !== undefined && envelope.cipher !== null) {
|
|
261
|
+
ciphers += 1
|
|
262
|
+
if (ciphers > MAX_CIPHER_REQUESTS) throw new AuthzHostError(`the engine asked for more than ${MAX_CIPHER_REQUESTS} cipher operations`)
|
|
263
|
+
const answer = await answerCipherRequest(envelope.cipher)
|
|
264
|
+
engine.call('authzFetched', [handle, JSON.stringify({ request: envelope.cipher, answer })])
|
|
265
|
+
continue
|
|
266
|
+
}
|
|
246
267
|
needs += 1
|
|
247
268
|
if (needs > MAX_NEEDS) throw new AuthzHostError('the engine asked for more than one request may fetch')
|
|
248
269
|
if (envelope.resolve !== undefined && envelope.resolve !== null) {
|
|
@@ -254,7 +275,7 @@ export async function openAuthorizationServer (options = {}) {
|
|
|
254
275
|
const got = await fetchFrom(String(f.url), String(f.accept), String(f.address))
|
|
255
276
|
engine.call('authzFetched', [handle, JSON.stringify({ url: f.url, status: got.status, body: got.body, now: nowSeconds() })])
|
|
256
277
|
} else {
|
|
257
|
-
throw new AuthzHostError('authzStep answered with no response, resolve or
|
|
278
|
+
throw new AuthzHostError('authzStep answered with no response, resolve, fetch or cipher')
|
|
258
279
|
}
|
|
259
280
|
}
|
|
260
281
|
},
|
package/bin/engine.mjs
CHANGED
|
@@ -21,6 +21,7 @@
|
|
|
21
21
|
// digest, decodes a block, or chooses an artifact.
|
|
22
22
|
|
|
23
23
|
import { readWhole } from '../store-host/index.mjs'
|
|
24
|
+
import { checkEngineStamp } from '../lib/engine-integrity.mjs'
|
|
24
25
|
import { fileUrlToPath, joinPath } from '../store-host/paths.mjs'
|
|
25
26
|
|
|
26
27
|
/** Read one environment variable, or null where reading it is refused. */
|
|
@@ -114,7 +115,27 @@ export function loadEngine (name = 'l4factoidal') {
|
|
|
114
115
|
? loaderPath
|
|
115
116
|
: 'file://' + encodeURI(loaderPath).replace(/#/g, '%23')
|
|
116
117
|
const module = await import(url)
|
|
117
|
-
|
|
118
|
+
// The release stamp check (lib/engine-integrity.mjs, issue 1106): the
|
|
119
|
+
// full engine's bytes against l4-assets/version.json, before they are
|
|
120
|
+
// instantiated. The stamp describes l4factoidal.wasm only.
|
|
121
|
+
if (name !== 'l4factoidal') return module.loadL4()
|
|
122
|
+
const dir = loaderPath.slice(0, loaderPath.length - `${name}.js`.length)
|
|
123
|
+
const wasmPath = dir + `${name}.wasm`
|
|
124
|
+
let stamp = null
|
|
125
|
+
try {
|
|
126
|
+
stamp = JSON.parse(new TextDecoder().decode(readWhole(dir + 'version.json')))
|
|
127
|
+
} catch (_error) {
|
|
128
|
+
stamp = null // no version.json: an unstamped copy
|
|
129
|
+
}
|
|
130
|
+
const bytes = readWhole(wasmPath)
|
|
131
|
+
await checkEngineStamp(bytes, stamp, {
|
|
132
|
+
where: wasmPath,
|
|
133
|
+
allowModified: environment('FACTOIDAL_ALLOW_MODIFIED_ENGINE') !== null,
|
|
134
|
+
warn: (msg) => { try { console.warn(msg) } catch (_e) {} }
|
|
135
|
+
})
|
|
136
|
+
return module.loadL4({
|
|
137
|
+
instantiateWasm: (imports) => WebAssembly.instantiate(bytes, imports)
|
|
138
|
+
})
|
|
118
139
|
})()
|
|
119
140
|
enginePromises.set(name, promise)
|
|
120
141
|
return promise
|