@factoidal/core 0.7.1 → 0.11.0

package/docs/fuseki-replacement.md

@@ -0,0 +1,108 @@
+# Using `factoidal serve` in place of Apache Jena Fuseki
+
+Status: 2026-10-18, `@factoidal/core` 0.10.0. Every row below was tested
+with curl against the installed package, and is gated by
+`test/fuseki-compat.test.mjs` (assertions for what works, expected failures
+naming an issue for what does not).
+
+## Command-line mapping
+
+| Fuseki | factoidal |
+|---|---|
+| `fuseki-server --file data.ttl /ds` | `factoidal serve --file data.ttl /ds` |
+| `fuseki-server --update --file data.ttl /ds` | the same; updates are always on |
+| `fuseki-server --loc DB /ds` (TDB2, read only) | `factoidal serve --tdb2 DB /ds`: the TDB2 database is read in place and is read only (see Storage) |
+| `fuseki-server --loc DB /ds` (TDB2, writable) | no TDB2 writer. Export with `tdb2.tdbdump`, then `factoidal pack data.nq STORE/gen-1 --layout ibk5`, `factoidal activate STORE gen-1`, `factoidal serve STORE --query-path /ds/sparql --gsp-path /ds/data` |
+| `--port 3030` | `--port 3030` (the default is 3030 in both) |
+| `--localhost` | the default; `--host 0.0.0.0` to listen on all addresses |
+| `--mem /ds` | `--file` with an empty file |
+| `--config config.ttl` | none: https://github.com/danbri/factoidal/issues/832 |
+
+`--file` packs the file into a new collection in a temporary directory
+(layout `ibk5`, so every graph accepts writes) and serves it. Updates go to
+that directory and are lost when the process ends, as with
+`fuseki-server --file`. For data that must persist, use the `STORE` form:
+updates are appended to the collection's delta log and synced before the
+response is sent.
+
+## Endpoints
+
+With `--file DATA /ds`, or `--query-path /ds/sparql --gsp-path /ds/data`:
+
+| Path | Fuseki | factoidal serve |
+|---|---|---|
+| `/ds/sparql` | query and update | query and update; GET with no query string returns the Service Description |
+| `/ds/query` | query | query and update (the same endpoint) |
+| `/ds/update` | update | query and update (the same endpoint) |
+| `/ds/data` | Graph Store read and write | Graph Store read and write (`?default`, `?graph=IRI`, or a path under `/ds/data`) |
+| `/ds/get` | Graph Store read | Graph Store read; PUT, POST and DELETE answer 405 with `Allow: GET, HEAD, OPTIONS` |
+| `/ds/data` with no graph parameter | the whole dataset | GET: N-Quads (the default) or TriG by `Accept`; a triple syntax in `Accept` selects the default graph. PUT of N-Quads or TriG replaces the dataset. POST of N-Quads or TriG adds to it (blank nodes renamed apart). POST of a triple syntax creates a new graph (Graph Store Protocol section 5.5). PUT of a triple syntax answers 415; DELETE answers 405 with `Allow`. https://github.com/danbri/factoidal/issues/829 |
+| `/$/ping`, `/$/datasets`, `/$/datasets/ds`, `/$/stats`, `/$/stats/ds`, `/$/server` | administration API | read-only, with Fuseki's JSON member names (`ds.name`, `ds.state`, `ds.services`, `Requests`, `RequestsGood`, `RequestsBad`, `startDateTime`, `uptime`); `built` is not reported. POST or DELETE of a dataset answers 405; other `/$/` operations answer 404. https://github.com/danbri/factoidal/issues/831 |
+
+Without `--query-path`, the endpoint is `/sparql` and also `/query` and
+`/update`; the Graph Store is `/data`.
+
+Conformance: W3C SPARQL 1.1 Protocol, Graph Store HTTP Protocol (direct and
+indirect identification) and Service Description tests against the running
+server over TCP (`lake exe l4protocol-live`): 50 pass, 0 fail (out of 50),
+re-run 2026-10-13 after the endpoint aliases were added
+(`docs/designissues/2026-10-10-store-serve.md` has the command).
+
+## Content types (tested)
+
+- Query results: `application/sparql-results+json`,
+  `application/sparql-results+xml`, `text/csv`, `text/tab-separated-values`.
+- CONSTRUCT and DESCRIBE: `text/turtle`, `application/n-triples` and
+  `application/rdf+xml`. A graph with no RDF/XML form (a predicate IRI that
+  is no XML QName) answers the next acceptable syntax, or 406 when RDF/XML
+  is the only one acceptable.
+- Query requests: GET `?query=`, POST `application/x-www-form-urlencoded`,
+  POST `application/sparql-query`. Updates: POST
+  `application/sparql-update` and form `update=`.
+- Graph Store GET: `text/turtle`, `application/n-triples`,
+  `application/rdf+xml`; on the whole dataset also `application/n-quads` and
+  `application/trig`. JSON-LD answers 406:
+  https://github.com/danbri/factoidal/issues/830.
+
+## Storage
+
+Fuseki keeps a dataset in memory (`--mem`, `--file`) or in TDB2 (`--loc`).
+factoidal serves a Shardborough collection: immutable generations plus a
+delta log. A triples generation (`--layout ibk3`) accepts writes to the
+default graph only; a named-graph write answers 501. Use `ibk4` or `ibk5`
+for named graphs. The npm command cannot yet fold the delta log into a new
+generation (`factoidal compact` and `factoidal update` exit 3:
+https://github.com/danbri/factoidal/issues/641); the native
+`l4block-shard-compact` of the source tree does it. Updates through the
+server work.
+TDB2 databases: `factoidal serve --tdb2 DB /ds` reads a Jena TDB2 database in
+place with the Lean decoder (`L4Factoidal/Tdb2/Read.lean`; the same quads as
+`tdb2.tdbdump`, record `docs/designissues/2026-10-13-tdb2-format.md`) and
+serves SPARQL query and Graph Store GET at `/ds/sparql`, `/ds/query`,
+`/ds/get` and `/ds/data`. `serveStore(DB, { tdb2: true, queryPath, gspPath })`
+does the same from Node. The database is read only: every update and every
+Graph Store write answers 501 with the message "this dataset is a read-only
+Jena TDB2 database read in place", and changes nothing. The current
+`Data-NNNN` generation is served; a database with a non-empty `journal.jrnl`
+(Jena has not replayed it) is refused. The server holds every quad in memory
+while it runs: it does not answer a pattern from the TDB2 indexes on disk
+(https://github.com/danbri/factoidal/issues/775 is the same gap for
+Shardborough). Not decoded: RDF 1.2 triple terms, prefixed datatype names.
+The native CLI reads TDB2 as well (`l4factoidal tdb2-dump DB`,
+`l4factoidal query DB --format tdb2 ...`). Test: `test/fuseki-compat.test.mjs`
+(needs a Jena install in `SOTA_DL`; skipped with that reason otherwise).
+Note that TDB2 itself changes the lexical form of numeric, boolean and date
+literals (`"0"^^xsd:boolean` reads back as `"false"`), so a dump is not always
+the data that was loaded.
+
+## Not supported
+
+- Several datasets in one process, assembler configuration, jena-text
+  indexes, inference models, `unionDefaultGraph`:
+  https://github.com/danbri/factoidal/issues/832.
+- Authentication, access control, HTTPS:
+  https://github.com/danbri/factoidal/issues/833. Bind to 127.0.0.1 (the
+  default) or put a reverse proxy in front.
+- The administration operations that change the server (create and delete
+  a dataset, backup, compaction, tasks, metrics) and the UI:
+  https://github.com/danbri/factoidal/issues/831.

package/fn.d.ts

@@ -16,10 +16,6 @@ import type {
   XFormsNodeValidity,
   SchematronFinding,
   ToanExpr,
-  MatrixCell,
-  MatrixResult,
-  ScaledValue,
-  SigmoidParams,
 } from './index';
 
 export type {
@@ -28,10 +24,6 @@ export type {
   XFormsNodeValidity,
   SchematronFinding,
   ToanExpr,
-  MatrixCell,
-  MatrixResult,
-  ScaledValue,
-  SigmoidParams,
 } from './index';
 
 /**
@@ -68,10 +60,10 @@ export class FnDataset implements Iterable<Quad> {
 export const EMPTY: FnDataset;
 
 /** Snapshot a (possibly mutable) RDF/JS Dataset into an FnDataset. */
-export function fromDataset(dataset: Dataset): FnDataset;
+export function fromDataset(dataset: Dataset | (Iterable<Quad> & { size: number })): FnDataset;
 
-/** Materialize an FnDataset into a fresh, independently-mutable Dataset. */
-export function toDataset(ds: FnDataset): Dataset;
+/** Materialize an FnDataset into a fresh, mutable RDF/JS DatasetCore (from @factoidal/core/rdfjs). */
+export function toDataset(ds: FnDataset): Iterable<Quad> & { size: number; add(q: Quad): unknown; delete(q: Quad): unknown; has(q: Quad): boolean };
 
 /**
  * A mutable accumulator for incrementally-arriving quads — the
@@ -118,8 +110,7 @@ export function mapQuads(ds: FnDataset, f: (quad: Quad) => Quad): FnDataset;
 
 /**
  * Run a SPARQL 1.1 query. SELECT -> Bindings[]; ASK -> boolean;
- * CONSTRUCT -> FnDataset (needs the npm-entry engine bundle — see
- * capabilities()).
+ * CONSTRUCT and DESCRIBE -> FnDataset.
  */
 export function query(
   ds: FnDataset,
@@ -131,7 +122,7 @@ export function query(
  * Materialize an entailment closure over ds's default graph as a new
  * FnDataset (a self-CONSTRUCT under the given regime — reuses
  * query()'s entailment support; named-graph triples are not
- * restated). Needs the npm-entry engine bundle.
+ * restated).
  */
 export function entail(ds: FnDataset, regime: EntailRegime): Promise<FnDataset>;
 
@@ -143,7 +134,7 @@ export interface ValidateResult {
 }
 
 /**
- * SHACL Core validation. Needs the npm-entry engine bundle. Neither
+ * SHACL Core validation. Neither
  * argument is mutated or consumed.
  * @param ds the data graph
  * @param shapes the shapes graph
@@ -152,10 +143,9 @@ export function validate(ds: FnDataset, shapes: FnDataset): Promise<ValidateResu
 
 /**
  * ShEx (Shape Expressions) validation of one focus node against one
- * shape. Needs the npm-entry engine bundle. `null` means "deferred" --
- * outside this engine's decidable ShEx fragment, never a guessed
- * answer.
- * @param schema ShExJ (JSON Schema form), as text
+ * shape. The Lean validator decides every schema it reads: the answer
+ * is a boolean.
+ * @param schema ShExJ (first non-whitespace character '{') or ShExC, as text
  * @param focus an IRI, "_:label", or an RDF/JS NamedNode/BlankNode term
  * @param shape a shape label (same shapes as focus); omit/null for the
  *   schema's own `start`
@@ -169,8 +159,7 @@ export function shex(
 
 /**
  * Evaluate an RML mapping graph against one logical source's raw data,
- * materializing the generated triples as a new FnDataset. Needs the
- * npm-entry engine bundle. Every triples map in `mapping` reads the
+ * materializing the generated triples as a new FnDataset. Every triples map in `mapping` reads the
  * SAME `source` -- joins across two different logical sources are out
  * of scope for this entry point.
  * @param mapping the RML mapping graph
@@ -183,8 +172,7 @@ export function fromMapping(
 ): Promise<FnDataset>;
 
 /**
- * CSVW csv2rdf conversion, materialized as a new FnDataset. Needs the
- * npm-entry engine bundle. Both arguments are raw text (CSVW's
+ * CSVW csv2rdf conversion, materialized as a new FnDataset. Both arguments are raw text (CSVW's
  * metadata format is JSON, not RDF); '' or omitted metadata infers
  * the schema from the CSV's own header row. Every table in a
  * multi-table `tables` group reads the SAME `csv` text.
@@ -204,7 +192,7 @@ export function fromCsvw(
 /**
  * RIF Core forward-chaining saturation, materialized as a new
  * FnDataset (input triples + derived triples, default graph only).
- * Needs the npm-entry engine bundle.
+ *
  * @param ds the premise graph
  * @param rules a RIF Core XML rule document
  */
@@ -216,7 +204,7 @@ export function rif(ds: FnDataset, rules: string): Promise<FnDataset>;
  * rdfs2/3/5/7/9/11 only, with the machine-checked decides-iff
  * (docs/theorem-registry.md). "corerdfs" is this project's API name
  * for the fragment the literature calls ρdf (subPropertyOf/
- * subClassOf/type/domain/range). Needs the npm-entry engine bundle.
+ * subClassOf/type/domain/range).
  */
 export function coreRdfsClosure(
   ds: FnDataset,
@@ -231,8 +219,7 @@ export function rhoDfClosure(
 
 /**
  * Decidable core-RDFS fragment check (`is_rho_df_frag`): does the
- * certified coreRdfsClosure guarantee apply to `ds`? Needs the
- * npm-entry engine bundle.
+ * certified coreRdfsClosure guarantee apply to `ds`?
  */
 export function coreRdfsCheck(
   ds: FnDataset,
@@ -251,19 +238,18 @@ export function rhoDfFragmentCheck(
  * owl:sameAs, owl:inverseOf, Symmetric/Transitive/Functional/
  * InverseFunctionalProperty, equivalentClass/Property. Every OWL row
  * carries a proved licensing + truth lemma; no chain-level
- * completeness claim (theorem registry). Needs the npm-entry engine
- * bundle.
+ * completeness claim (theorem registry).
  */
 export function rdfsPlusClosure(
   ds: FnDataset,
   options?: { format?: string }
-): Promise<{ ok: boolean; dataset: FnDataset; rounds: number }>;
+): Promise<{ ok: boolean; dataset: FnDataset }>;
 
 /**
  * OWL tableau materialisation (formal/fstar/Tableau.fst's
  * `tableau_materialise`): add `i rdf:type <ClassExpression>` for every
  * individual the model-construction reasoner proves is a member of an
- * OWL class expression. Needs the npm-entry engine bundle. Default
+ * OWL class expression. Default
  * graph only. `dataset` is input + tableau-derived triples;
  * `addedCount` is how many triples the tableau added.
  */
@@ -274,8 +260,7 @@ export function tableauMaterialise(
 /**
  * OWL DL inconsistency verdict (bin/owl-runner's DL pipeline: OWL-RL
  * closure -> tableau materialise -> OWL-RL closure -> is_inconsistent).
- * `rlAlone` is the plain OWL-RL verdict on the same input. Needs the
- * npm-entry engine bundle. Default graph only.
+ * `rlAlone` is the plain OWL-RL verdict on the same input. Default graph only.
  */
 export function tableauDlInconsistent(
   ds: FnDataset
@@ -283,8 +268,7 @@ export function tableauDlInconsistent(
 
 /**
  * OWL DL consistency verdict via the verified clash-detecting tableau
- * (Tableau.Refute.tableau_consistent over the OWL-RL closure). Needs the
- * npm-entry engine bundle. Default graph only. `ontology` is an
+ * (Tableau.Refute.tableau_consistent over the OWL-RL closure). Default graph only. `ontology` is an
  * FnDataset or raw RDF text (Turtle by default). `consistent` is
  * `false`, `true`, or `null` (budget-out, `reason` names the fuel cap).
  */
@@ -295,8 +279,7 @@ export function owlIsConsistent(
 
 /**
  * OWL entailment check: does `premise` entail `conclusion`? Verified
- * two-path dispatch (`via: "closure"` / `via: "refutation"`). Needs the
- * npm-entry engine bundle. Default graph only. Each argument is an
+ * two-path dispatch (`via: "closure"` / `via: "refutation"`). Default graph only. Each argument is an
  * FnDataset or raw RDF text. `entailed` is `true`, `false`, or `null`
  * (refutation budget-out, `reason` names the cap).
  */
@@ -306,12 +289,12 @@ export function owlEntails(
   options?: { format?: string; fuel?: number | string }
 ): Promise<{ entailed: boolean | null; via: 'closure' | 'refutation'; reason?: string }>;
 
-/** RDFC-1.0 canonical N-Quads text. Needs the npm-entry engine bundle. */
+/** RDFC-1.0 canonical N-Quads text. */
 export function canonicalize(ds: FnDataset): Promise<string>;
 
 /**
  * sha256 hex digest of canonicalize(ds); memoized by FnDataset
- * identity. Needs the npm-entry engine bundle (canonicalize's cost).
+ * identity.
  */
 export function hash(ds: FnDataset): Promise<string>;
 
@@ -323,7 +306,7 @@ export function hash(ds: FnDataset): Promise<string>;
 export function equals(a: FnDataset, b: FnDataset): Promise<boolean>;
 
 /** Enumerate named graphs (default graph excluded), first-seen order. */
-export function graphs(ds: FnDataset): Array<[iri: string, graph: FnDataset]>;
+export function graphs(ds: FnDataset): Array<[name: NamedNode | BlankNode, graph: FnDataset]>;
 
 /** A minimal mutable box holding one dataflow input value. */
 export interface Cell<T> {
@@ -418,25 +401,6 @@ export function toanSubst(
   variable: string,
   value: ToanExpr
 ): Promise<string>;
-export function matrixDeterminant(
-  matrix: MatrixCell[][]
-): Promise<MatrixResult>;
-export function matrixScalarProduct(
-  a: MatrixCell[],
-  b: MatrixCell[]
-): Promise<MatrixResult>;
-export function matrixVectorProduct(
-  a: MatrixCell[],
-  b: MatrixCell[]
-): Promise<MatrixResult>;
-export function matrixOuterProduct(
-  a: MatrixCell[],
-  b: MatrixCell[]
-): Promise<MatrixResult>;
-export function sigmoidPoints(
-  params: SigmoidParams
-): Promise<Array<{ x: ScaledValue; y: ScaledValue }>>;
-export function sigmoidFormulaMathml(): Promise<string>;
 
 // ---------------------------------------------------------------------
 // VC Data Integrity crypto (eddsa-rdfc-2022, HACL* wasm backend).
@@ -467,44 +431,9 @@ export function vcEddsaVerifyFromCanonical(
   proofValue: string
 ): Promise<boolean>;
 
-// ---------------------------------------------------------------------
-// In-memory COTTAS/Parquet bytes store. Unlike every other constructor
-// here, openCottas() does NOT return an FnDataset: rows decode lazily
-// as query() touches them (see fn.js's doc comment and
-// docs/designissues/2026-07-06-inmemory-bytes-store.md). Needs the
-// npm-entry engine bundle.
-// ---------------------------------------------------------------------
-
-/** A read-only, lazily-decoding COTTAS store handle (see openCottas). */
-export interface CottasStore {
-  /** The opaque handle string the underlying engine registry uses. */
-  readonly handle: string;
-  /**
-   * SPARQL over the store. SELECT -> Bindings[]; ASK -> boolean;
-   * CONSTRUCT -> FnDataset (materialized once). No `entail` option, no
-   * write overlay (read-only) — see fn.js's openCottas doc comment for
-   * the full divergence list from query(). Throws after close().
-   */
-  query(sparql: string): Promise<Bindings[] | boolean | FnDataset>;
-  /** Release the store. Idempotent. */
-  close(): Promise<void>;
-}
-
-/** Open a COTTAS/Parquet artifact's raw bytes as a read-only store. */
-export function openCottas(
-  bytes: Uint8Array | ArrayBuffer | string
-): Promise<CottasStore>;
-
-/**
- * Serialize an FnDataset to COTTAS/Parquet bytes via the native writer
- * (round-trips back into openCottas()). Needs the npm-entry bundle.
- */
-export function toCottas(ds: FnDataset): Promise<Uint8Array>;
-
 /**
- * Run a SPARQL 1.1 query against a read-only HDT artifact's raw bytes
- * (factoidal_cli.ml's `--data-hdt` backend). No npm-entry engine
- * bundle needed -- CLI-only. Default graph only, SELECT/ASK only.
+ * Run a SPARQL 1.1 query against a read-only HDT file's raw bytes (op
+ * queryHdt). Default graph only, SELECT/ASK only.
  */
 export function queryHdt(
   hdtBytes: Uint8Array | ArrayBuffer | string,