@lde/pipeline 0.33.4 → 0.34.1

package/README.md

@@ -298,7 +298,7 @@ After all stages for a dataset have run, the pipeline calls `validator.report(da
 Writes generated quads to a destination. A `Writer` is transactional: each pipeline run opens one run on it (`openRun(context)`), writes every dataset through the resulting `RunWriter`, and ends with exactly one `commit()` (on success) or `abort(error)` (on failure). The run lifecycle is the home of destination-level concerns such as atomic swaps, deletion sweeps and cross-pod locks; the pipeline drives `openRun → write* → commit/abort` uniformly and never branches on the writer’s update mode. After a dataset’s stages complete, its `flush(dataset, outcome)` says whether the dataset succeeded, so a writer can gate destructive finalization (e.g. an In-place stale-document sweep) on `'success'`.
 
 - `SparqlUpdateWriter` — writes to a SPARQL endpoint via UPDATE queries; writes are visible as they land, so `commit`/`abort` are no-ops
-- `FileWriter` — writes to local files; each file is streamed to a sibling temp file and atomically renamed on flush, `commit` finalizes any files still open, and `abort` discards temp output
+- `FileWriter` — writes to local files; each file is streamed to a sibling temp file and atomically renamed on flush, `commit` finalizes any files still open, and `abort` discards temp output. In `n-quads` format it skolemises blank nodes to dataset-scoped IRIs, because such files are typically `cat`-indexed into one served graph where document-scoped blank-node labels from different datasets would otherwise fuse (see `blankNodes`/`failOnBlankNodes` and ldelements/lde#474); Turtle/N-Triples output is left unchanged
 
 The `RunContext` handed to `openRun` carries the run’s identity (`runId`, `startedAt`), the full selection (`selectedSources()`, complete by commit time – including datasets skipped as unchanged) and the pipeline’s provenance store, when configured.
 
@@ -390,51 +390,50 @@ Otherwise it imports (if needed), runs the stages, and writes an updated record.
 
 ### Plugins
 
-Plugins hook into the pipeline lifecycle via the `PipelinePlugin` interface. Register them in the `plugins` array when constructing a `Pipeline`.
+Plugins hook into the pipeline lifecycle via the `PipelinePlugin` interface. Register them in the `plugins` array when constructing a `Pipeline`. A plugin can implement either or both of two hooks, each a `QuadTransform`:
+
+- **`beforeStageWrite`** runs over each stage’s output as it is written – scoped to one stage’s quads for one dataset.
+- **`beforeDatasetWrite`** runs once over a dataset’s combined cross-stage output, just before it is written – scoped to the whole dataset, so it can reconcile quads that different stages produced. The output is streamed through a single queue rather than materialized, so memory stays bounded by whatever the transform itself buffers; it honors the writer’s run lifecycle (`flush`/`reset`/`commit`/`abort`), and a failure is isolated to the dataset being written.
+
+Both hooks are domain-agnostic – dedup, roll-ups, provenance stamping, or namespace rewriting can use them. This package ships `provenancePlugin()`, `namespaceNormalizationPlugin()`, and `schemaOrgNormalizationPlugin()` (all `beforeStageWrite` plugins). Plugins that must understand a particular RDF shape live with that vocabulary: for example, [@lde/pipeline-void](../pipeline-void) ships `schemaOrgPartitionMergePlugin()`, a `beforeDatasetWrite` plugin that merges the VoID partition _nodes_ that Schema.org `http`/`https` variants produce.
+
+#### `provenancePlugin()`
+
+Appends [PROV-O](https://www.w3.org/TR/prov-o/) provenance quads (`prov:Entity`, `prov:Activity`, `prov:startedAtTime`, `prov:endedAtTime`) to every stage’s output. The `prov:Activity` is a stable IRI keyed on `(dataset, stage)`, not a blank node, so activities stay distinct – and a re-run stays idempotent – when per-dataset outputs are merged into one graph (blank-node labels are not unique across separately serialised documents and would fuse unrelated activities).
 
 #### `namespaceNormalizationPlugin(options)`
 
-Generic plugin that rewrites namespace prefixes in `void:class` and `void:property` quad objects. Accepts `from` and `to` options specifying the source and target namespace URI prefixes. `void:vocabulary` quads are left unchanged so consumers can see which namespace the source dataset actually uses.
+Rewrites every IRI in the `from` namespace to the `to` namespace, in subject, predicate and object position alike. A blanket, vocabulary-agnostic rewrite for standardizing a namespace across a dataset’s own quads – for example when mapping instance data to an application profile.
 
 ```typescript
-import { namespaceNormalizationPlugin } from ‘@lde/pipeline’;
+import { namespaceNormalizationPlugin } from '@lde/pipeline';
 
 new Pipeline({
   // ...
   plugins: [
     namespaceNormalizationPlugin({
-      from: ‘http://example.org/’,
-      to: ‘https://example.org/’,
+      from: 'http://example.org/',
+      to: 'https://example.org/',
     }),
   ],
 });
 ```
 
-#### `provenancePlugin()`
-
-Appends [PROV-O](https://www.w3.org/TR/prov-o/) provenance quads (`prov:Entity`, `prov:Activity`, `prov:startedAtTime`, `prov:endedAtTime`) to every stage’s output. The `prov:Activity` is a stable IRI keyed on `(dataset, stage)`, not a blank node, so activities stay distinct – and a re-run stays idempotent – when per-dataset outputs are merged into one graph (blank-node labels are not unique across separately serialised documents and would fuse unrelated activities).
-
 #### `schemaOrgNormalizationPlugin(options?)`
 
-Normalizes Schema.org namespace prefixes in `void:class` and `void:property` quad objects. By default, rewrites `http://schema.org/` to `https://schema.org/`. Pass `{ reverse: true }` to normalize in the opposite direction (`https://` to `http://`). `void:vocabulary` quads are left unchanged so consumers can see which namespace the source dataset actually uses.
-
-This is a convenience wrapper around `namespaceNormalizationPlugin`.
+Convenience wrapper around `namespaceNormalizationPlugin` for the common case: rewrites `http://schema.org/` to `https://schema.org/` (pass `{ reverse: true }` for the opposite direction).
 
 ```typescript
-import { schemaOrgNormalizationPlugin, provenancePlugin } from ‘@lde/pipeline’;
+import { schemaOrgNormalizationPlugin, provenancePlugin } from '@lde/pipeline';
 
 new Pipeline({
   // ...
   plugins: [schemaOrgNormalizationPlugin(), provenancePlugin()],
 });
-
-// Or reverse: normalize https to http
-new Pipeline({
-  // ...
-  plugins: [schemaOrgNormalizationPlugin({reverse: true})],
-});
 ```
 
+This is a plain rewrite and knows nothing about VoID. If your pipeline emits VoID partitions and a dataset mixes both schema.org namespaces, rewriting the `void:class` objects alone still leaves two `void:classPartition` nodes for one class – use [`schemaOrgPartitionMergePlugin`](../pipeline-void) from `@lde/pipeline-void`, which also merges those nodes.
+
 ## Usage
 
 ```typescript

package/dist/guard/blankNodes.d.ts

@@ -16,6 +16,13 @@ import type { QuadTransform } from '../stage.js';
  * NO blank nodes. Mint stable (skolem) IRIs instead — see `skolemIri` in
  * `@lde/dataset`. These helpers make that invariant testable and enforceable.
  *
+ * `FileWriter` skolemises blank nodes automatically for `n-quads` output as a
+ * safety net, so a producer that slips a blank node through cannot fuse it into
+ * another dataset there. These helpers still matter for producers that must hold
+ * the invariant regardless of writer – e.g. output bound for a `SparqlUpdateWriter`
+ * or a `cat`-indexed Turtle sink, which do not skolemise – and for turning a slip
+ * into a loud failure rather than a silently rewritten IRI.
+ *
  * See ldelements/lde#474 and netwerk-digitaal-erfgoed/dataset-knowledge-graph#352.
  */
 /**

package/dist/guard/blankNodes.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"blankNodes.d.ts","sourceRoot":"","sources":["../../src/guard/blankNodes.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAC;AACzC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAEjD;;;;;;;;;;;;;;;;;GAiBG;AAEH;;;GAGG;AACH,wBAAgB,UAAU,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,MAAM,EAAE,CAU1D;AAED;;;GAGG;AACH,wBAAgB,kBAAkB,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,IAAI,CAU9D;AAED;;;;;GAKG;AACH,wBAAgB,gBAAgB,CAAC,OAAO,KAAK,aAAa,CAAC,OAAO,CAAC,CAelE"}
+{"version":3,"file":"blankNodes.d.ts","sourceRoot":"","sources":["../../src/guard/blankNodes.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAC;AACzC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAEjD;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AAEH;;;GAGG;AACH,wBAAgB,UAAU,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,MAAM,EAAE,CAU1D;AAED;;;GAGG;AACH,wBAAgB,kBAAkB,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,IAAI,CAU9D;AAED;;;;;GAKG;AACH,wBAAgB,gBAAgB,CAAC,OAAO,KAAK,aAAa,CAAC,OAAO,CAAC,CAelE"}

package/dist/guard/blankNodes.js

@@ -14,6 +14,13 @@
  * NO blank nodes. Mint stable (skolem) IRIs instead — see `skolemIri` in
  * `@lde/dataset`. These helpers make that invariant testable and enforceable.
  *
+ * `FileWriter` skolemises blank nodes automatically for `n-quads` output as a
+ * safety net, so a producer that slips a blank node through cannot fuse it into
+ * another dataset there. These helpers still matter for producers that must hold
+ * the invariant regardless of writer – e.g. output bound for a `SparqlUpdateWriter`
+ * or a `cat`-indexed Turtle sink, which do not skolemise – and for turning a slip
+ * into a loud failure rather than a silently rewritten IRI.
+ *
  * See ldelements/lde#474 and netwerk-digitaal-erfgoed/dataset-knowledge-graph#352.
  */
 /**

package/dist/index.d.ts

@@ -12,7 +12,7 @@ export * from './sparql/index.js';
 export * from './distribution/index.js';
 export * from './provenance/index.js';
 export * from './writer/index.js';
-export * from './plugin/namespaceNormalization.js';
 export * from './plugin/provenance.js';
+export * from './plugin/namespaceNormalization.js';
 export * from './plugin/schemaOrgNormalization.js';
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,gBAAgB,CAAC;AAC/B,cAAc,qBAAqB,CAAC;AACpC,cAAc,iBAAiB,CAAC;AAChC,cAAc,YAAY,CAAC;AAC3B,cAAc,eAAe,CAAC;AAC9B,cAAc,uBAAuB,CAAC;AACtC,cAAc,eAAe,CAAC;AAC9B,cAAc,YAAY,CAAC;AAC3B,cAAc,0BAA0B,CAAC;AACzC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,oCAAoC,CAAC;AACnD,cAAc,wBAAwB,CAAC;AACvC,cAAc,oCAAoC,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,gBAAgB,CAAC;AAC/B,cAAc,qBAAqB,CAAC;AACpC,cAAc,iBAAiB,CAAC;AAChC,cAAc,YAAY,CAAC;AAC3B,cAAc,eAAe,CAAC;AAC9B,cAAc,uBAAuB,CAAC;AACtC,cAAc,eAAe,CAAC;AAC9B,cAAc,YAAY,CAAC;AAC3B,cAAc,0BAA0B,CAAC;AACzC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,wBAAwB,CAAC;AACvC,cAAc,oCAAoC,CAAC;AACnD,cAAc,oCAAoC,CAAC"}

package/dist/index.js

@@ -12,6 +12,6 @@ export * from './sparql/index.js';
 export * from './distribution/index.js';
 export * from './provenance/index.js';
 export * from './writer/index.js';
-export * from './plugin/namespaceNormalization.js';
 export * from './plugin/provenance.js';
+export * from './plugin/namespaceNormalization.js';
 export * from './plugin/schemaOrgNormalization.js';

package/dist/pipeline.d.ts

@@ -20,16 +20,40 @@ export interface BeforeStageWriteContext {
     dataset: Dataset;
     stage: string;
 }
+/**
+ * Context handed to a {@link PipelinePlugin.beforeDatasetWrite} transform: the
+ * `dataset` whose complete, cross-stage output is being written. Unlike
+ * {@link BeforeStageWriteContext} there is no `stage` — the transform sees one
+ * dataset's quads from every stage as a single stream.
+ */
+export interface BeforeDatasetWriteContext {
+    dataset: Dataset;
+}
 /** Plugin that hooks into pipeline lifecycle events. */
 export interface PipelinePlugin {
     name: string;
     /**
-     * Transform the merged, post-stage quad stream before writing (extension
-     * point 2: pipeline-wide, post-merge). The home of cross-cutting concerns
-     * – provenance, namespace normalisation – that apply regardless of which
-     * reader produced a quad.
+     * Transform a single stage's merged output before writing (extension point
+     * 2: per stage, post-merge). Runs once per stage per dataset, carrying the
+     * stage identity so it can mint stable per-`(dataset, stage)` IRIs. The home
+     * of stage-scoped concerns — provenance stamping, namespace rewriting of a
+     * stage's own quads.
      */
     beforeStageWrite?: QuadTransform<BeforeStageWriteContext>;
+    /**
+     * Transform a whole dataset's output before writing (extension point 3: per
+     * dataset, cross-stage). Runs once over the concatenation of every stage's
+     * output for one dataset, after all stages complete and before the write is
+     * flushed. The home of concerns that need to reconcile quads *across* stages
+     * — deduplication, cross-stage roll-ups, or merging partition nodes that
+     * different stages contribute to.
+     *
+     * The transform is a streaming {@link QuadTransform}: the pipeline pipes each
+     * stage's output through it via a bounded queue and never materializes the
+     * dataset, so a pass-through or streaming transform stays O(1). A transform
+     * that genuinely needs cross-stage state buffers only its own working set.
+     */
+    beforeDatasetWrite?: QuadTransform<BeforeDatasetWriteContext>;
 }
 export interface PipelineOptions {
     datasetSelector: DatasetSelector;
@@ -83,6 +107,7 @@ export declare class Pipeline {
     private readonly stages;
     private readonly writer;
     private readonly beforeStageWrite?;
+    private readonly beforeDatasetWrite?;
     private readonly distributionResolver;
     private readonly chaining?;
     private readonly reporter?;

package/dist/pipeline.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"pipeline.d.ts","sourceRoot":"","sources":["../src/pipeline.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,OAAO,EAAgB,MAAM,cAAc,CAAC;AAGrD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AACrD,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAChD,OAAO,KAAK,EAKV,MAAM,EACP,MAAM,oBAAoB,CAAC;AAE5B,OAAO,EACL,KAAK,oBAAoB,EAI1B,MAAM,4BAA4B,CAAC;AAKpC,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAC;AAY7D,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,0BAA0B,CAAC;AACpE,OAAO,KAAK,EAEV,gBAAgB,EACjB,MAAM,uBAAuB,CAAC;AAG/B,OAAO,EAEL,KAAK,aAAa,EACnB,MAAM,2BAA2B,CAAC;AAEnC;;;;;;;GAOG;AACH,MAAM,WAAW,uBAAuB;IACtC,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,EAAE,MAAM,CAAC;CACf;AAED,wDAAwD;AACxD,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb;;;;;OAKG;IACH,gBAAgB,CAAC,EAAE,aAAa,CAAC,uBAAuB,CAAC,CAAC;CAC3D;AAED,MAAM,WAAW,eAAe;IAC9B,eAAe,EAAE,eAAe,CAAC;IACjC,MAAM,EAAE,KAAK,EAAE,CAAC;IAChB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAAC;IAC3B,OAAO,CAAC,EAAE,cAAc,EAAE,CAAC;IAC3B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,oBAAoB,CAAC,EAAE,oBAAoB,CAAC;IAC5C,QAAQ,CAAC,EAAE;QACT,mBAAmB,EAAE,mBAAmB,CAAC;QACzC,SAAS,EAAE,MAAM,CAAC;KACnB,CAAC;IACF;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,gBAAgB,GAAG,SAAS,gBAAgB,EAAE,CAAC;IAC1D;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,eAAe,CAAC;IAClC;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;;;;;;OASG;IACH,OAAO,CAAC,EAAE,MAAM,aAAa,CAAC;CAC/B;AAqID,qBAAa,QAAQ;IACnB,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAS;IAC9B,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAkB;IAClD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAU;IACjC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;IAChC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAC,CAAyC;IAC3E,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAuB;IAC5D,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA8B;IACxD,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAmB;IAC7C,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAsB;IACrD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAkB;IACnD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAS;gBAE9B,OAAO,EAAE,eAAe;IA+C9B,GAAG,IAAI,OAAO,CAAC,IAAI,CAAC;YAyCZ,cAAc;IAqN5B,+EAA+E;YACjE,aAAa;YAmBb,gBAAgB;IAW9B,OAAO,CAAE,aAAa;IAOtB;;;;;OAKG;IACH,OAAO,CAAC,WAAW;IAMnB;;;;;;;;OAQG;IACH,OAAO,CAAC,0BAA0B;IA4BlC;;;;;OAKG;YACW,SAAS;IAuCvB;;;OAGG;YACW,QAAQ;IA0CtB,2EAA2E;YAC7D,eAAe;YAqBf,QAAQ;YAuEP,SAAS;CAczB"}
+{"version":3,"file":"pipeline.d.ts","sourceRoot":"","sources":["../src/pipeline.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,OAAO,EAAgB,MAAM,cAAc,CAAC;AAGrD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AACrD,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAEhD,OAAO,KAAK,EAKV,MAAM,EACP,MAAM,oBAAoB,CAAC;AAE5B,OAAO,EACL,KAAK,oBAAoB,EAI1B,MAAM,4BAA4B,CAAC;AAKpC,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAC;AAY7D,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,0BAA0B,CAAC;AACpE,OAAO,KAAK,EAEV,gBAAgB,EACjB,MAAM,uBAAuB,CAAC;AAG/B,OAAO,EAEL,KAAK,aAAa,EACnB,MAAM,2BAA2B,CAAC;AAEnC;;;;;;;GAOG;AACH,MAAM,WAAW,uBAAuB;IACtC,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,EAAE,MAAM,CAAC;CACf;AAED;;;;;GAKG;AACH,MAAM,WAAW,yBAAyB;IACxC,OAAO,EAAE,OAAO,CAAC;CAClB;AAED,wDAAwD;AACxD,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb;;;;;;OAMG;IACH,gBAAgB,CAAC,EAAE,aAAa,CAAC,uBAAuB,CAAC,CAAC;IAC1D;;;;;;;;;;;;OAYG;IACH,kBAAkB,CAAC,EAAE,aAAa,CAAC,yBAAyB,CAAC,CAAC;CAC/D;AAED,MAAM,WAAW,eAAe;IAC9B,eAAe,EAAE,eAAe,CAAC;IACjC,MAAM,EAAE,KAAK,EAAE,CAAC;IAChB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAAC;IAC3B,OAAO,CAAC,EAAE,cAAc,EAAE,CAAC;IAC3B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,oBAAoB,CAAC,EAAE,oBAAoB,CAAC;IAC5C,QAAQ,CAAC,EAAE;QACT,mBAAmB,EAAE,mBAAmB,CAAC;QACzC,SAAS,EAAE,MAAM,CAAC;KACnB,CAAC;IACF;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,gBAAgB,GAAG,SAAS,gBAAgB,EAAE,CAAC;IAC1D;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,eAAe,CAAC;IAClC;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;;;;;;OASG;IACH,OAAO,CAAC,EAAE,MAAM,aAAa,CAAC;CAC/B;AA0OD,qBAAa,QAAQ;IACnB,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAS;IAC9B,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAkB;IAClD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAU;IACjC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;IAChC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAC,CAAyC;IAC3E,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAC,CAA2C;IAC/E,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAuB;IAC5D,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA8B;IACxD,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAmB;IAC7C,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAsB;IACrD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAkB;IACnD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAS;gBAE9B,OAAO,EAAE,eAAe;IAyD9B,GAAG,IAAI,OAAO,CAAC,IAAI,CAAC;YA8CZ,cAAc;IAgO5B,+EAA+E;YACjE,aAAa;YAmBb,gBAAgB;IAW9B,OAAO,CAAE,aAAa;IAOtB;;;;;OAKG;IACH,OAAO,CAAC,WAAW;IAMnB;;;;;;;;OAQG;IACH,OAAO,CAAC,0BAA0B;IA4BlC;;;;;OAKG;YACW,SAAS;IAuCvB;;;OAGG;YACW,QAAQ;IA0CtB,2EAA2E;YAC7D,eAAe;YAqBf,QAAQ;YAuEP,SAAS;CAczB"}

package/dist/pipeline.js

@@ -1,6 +1,7 @@
 import { createReadStream } from 'node:fs';
 import { randomUUID } from 'node:crypto';
 import { StreamParser } from 'n3';
+import { AsyncQueue } from './asyncQueue.js';
 import { FileWriter } from './writer/fileWriter.js';
 import { NoDistributionAvailable, ResolvedDistribution, } from './distribution/resolver.js';
 import { SparqlDistributionResolver } from './distribution/index.js';
@@ -121,12 +122,102 @@ class TransformWriter {
         await this.inner.write(dataset, this.transform(quads, { dataset, stage: this.stage }));
     }
 }
+/** Discards a dataset's queued output; used to unblock the transform on reset. */
+class DatasetReset extends Error {
+}
+/**
+ * A {@link RunWriter} decorator that applies a {@link PipelinePlugin.beforeDatasetWrite}
+ * transform once over a dataset's complete, cross-stage output.
+ *
+ * Every stage's {@link write} for a dataset feeds a single {@link AsyncQueue};
+ * the transform consumes that queue as one stream and its output goes to the
+ * inner writer. The queue is created on the first write and closed on
+ * {@link flush}, so stage output streams through the transform (bounded by the
+ * queue's backpressure) rather than being materialized. Datasets are processed
+ * one at a time, so a single in-flight queue suffices.
+ */
+class DatasetTransformWriter {
+    inner;
+    transform;
+    current;
+    constructor(inner, transform) {
+        this.inner = inner;
+        this.transform = transform;
+    }
+    async write(dataset, quads) {
+        const { queue } = this.open(dataset);
+        for await (const quad of quads) {
+            await queue.push(quad);
+        }
+    }
+    async flush(dataset, outcome) {
+        let writeError;
+        if (this.current?.iri === dataset.iri.toString()) {
+            const { queue, innerWrite } = this.current;
+            this.current = undefined;
+            queue.close();
+            // Capture rather than propagate immediately, so the inner writer still
+            // gets its flush (as 'failed') to finalize/clean up this dataset.
+            try {
+                await innerWrite;
+            }
+            catch (error) {
+                writeError = error;
+            }
+        }
+        await this.inner.flush?.(dataset, writeError ? 'failed' : outcome);
+        if (writeError)
+            throw writeError;
+    }
+    async reset(dataset) {
+        await this.discardCurrent(dataset, new DatasetReset());
+        await this.inner.reset?.(dataset);
+    }
+    async commit() {
+        await this.inner.commit();
+    }
+    async abort(error) {
+        await this.discardCurrent(undefined, error);
+        await this.inner.abort(error);
+    }
+    open(dataset) {
+        const iri = dataset.iri.toString();
+        if (this.current?.iri === iri)
+            return this.current;
+        const queue = new AsyncQueue();
+        const innerWrite = this.inner.write(dataset, this.transform(queue, { dataset }));
+        // If the inner write (or the transform) fails, abort the queue so a
+        // producer blocked on push() is released and surfaces the error, instead
+        // of hanging forever waiting for a consumer that has stopped pulling. The
+        // rejection is re-observed (and re-thrown) by flush()/discardCurrent via
+        // `innerWrite`, so attaching this handler does not swallow it.
+        void innerWrite.catch((error) => queue.abort(error));
+        this.current = { iri, queue, innerWrite };
+        return this.current;
+    }
+    /**
+     * Abandon the in-flight dataset (the given one, or any when `dataset` is
+     * undefined): abort the queue so the transform and inner write terminate,
+     * then await them, swallowing the abort so it does not mask the caller's flow.
+     */
+    async discardCurrent(dataset, error) {
+        if (!this.current)
+            return;
+        if (dataset && this.current.iri !== dataset.iri.toString())
+            return;
+        const { queue, innerWrite } = this.current;
+        this.current = undefined;
+        queue.abort(error);
+        await innerWrite.catch(() => undefined);
+    }
+}
 export class Pipeline {
     name;
     datasetSelector;
     stages;
     writer;
     beforeStageWrite;
+    beforeDatasetWrite;
     distributionResolver;
     chaining;
     reporter;
@@ -156,6 +247,12 @@ export class Pipeline {
         this.beforeStageWrite = transforms?.length
             ? (quads, context) => transforms.reduce((q, fn) => fn(q, context), quads)
             : undefined;
+        const datasetTransforms = options.plugins
+            ?.map((p) => p.beforeDatasetWrite)
+            .filter((t) => t !== undefined);
+        this.beforeDatasetWrite = datasetTransforms?.length
+            ? (quads, context) => datasetTransforms.reduce((q, fn) => fn(q, context), quads)
+            : undefined;
         this.distributionResolver =
             options.distributionResolver ?? new SparqlDistributionResolver();
         this.chaining = options.chaining;
@@ -186,7 +283,12 @@ export class Pipeline {
             selectedSources: () => selectedSources,
             provenance: this.provenanceStore,
         };
-        const runWriter = await this.writer.openRun(context);
+        const openedWriter = await this.writer.openRun(context);
+        // Wrap the run writer so a whole-dataset transform sees every stage's
+        // output for a dataset as one stream before it is written.
+        const runWriter = this.beforeDatasetWrite
+            ? new DatasetTransformWriter(openedWriter, this.beforeDatasetWrite)
+            : openedWriter;
         try {
             for await (const dataset of datasets) {
                 selectedSources.push(dataset.iri.toString());
@@ -345,7 +447,16 @@ export class Pipeline {
             await this.distributionResolver.cleanup?.();
             unsubscribe?.();
         }
-        await runWriter.flush?.(dataset, stageFailed ? 'failed' : 'success');
+        try {
+            await runWriter.flush?.(dataset, stageFailed ? 'failed' : 'success');
+        }
+        catch (error) {
+            // A per-dataset writer failure must not abort the whole run: isolate it
+            // like a stage failure so the remaining datasets still process. The
+            // dataset is recorded 'failed' below and retried on the next run.
+            this.reporter?.stageFailed?.('write', error instanceof Error ? error : new Error(String(error)));
+            stageFailed = true;
+        }
         await this.reportValidators(dataset);
         // A dataset whose stages threw produced incomplete output; record it as
         // ‘failed’ rather than freezing a broken result under a ‘success’ record.

package/dist/plugin/namespaceNormalization.d.ts

@@ -1,29 +1,31 @@
 import type { QuadTransform } from '../stage.js';
-import type { PipelinePlugin } from '../pipeline.js';
-import type { Dataset } from '@lde/dataset';
+import type { BeforeStageWriteContext, PipelinePlugin } from '../pipeline.js';
 export interface NamespaceNormalizationOptions {
     /** Namespace URI prefix to match (e.g. `http://schema.org/`). */
     from: string;
-    /** Namespace URI prefix to replace with (e.g. `https://schema.org/`). */
+    /** Namespace URI prefix to replace it with (e.g. `https://schema.org/`). */
     to: string;
 }
 /**
- * Creates a QuadTransform that rewrites namespace prefixes in `void:class` and
- * `void:property` quad objects from {@link NamespaceNormalizationOptions.from}
- * to {@link NamespaceNormalizationOptions.to}.
+ * Creates a {@link QuadTransform} that rewrites every IRI in the
+ * {@link NamespaceNormalizationOptions.from} namespace to
+ * {@link NamespaceNormalizationOptions.to} — in subject, predicate and object
+ * position alike.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * This is a blanket, vocabulary-agnostic rewrite: it normalizes the namespace
+ * wherever it appears, with no knowledge of VoID or any other RDF shape. Use it
+ * to standardize a namespace across a dataset's own quads (for example
+ * `http://schema.org/` → `https://schema.org/` when mapping instance data to an
+ * application profile). Merging VoID partition _nodes_ that two namespace
+ * variants produced is a separate, shape-aware concern handled by
+ * `@lde/pipeline-void`.
  */
-export declare function namespaceNormalizationTransform(options: NamespaceNormalizationOptions): QuadTransform<{
-    dataset: Dataset;
-}>;
+export declare function namespaceNormalizationTransform(options: NamespaceNormalizationOptions): QuadTransform<BeforeStageWriteContext>;
 /**
- * Pipeline plugin that normalizes namespace prefixes in `void:class` and
- * `void:property` quad objects.
- *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * A generic {@link PipelinePlugin} that normalizes a namespace prefix across a
+ * stage's output via {@link PipelinePlugin.beforeStageWrite}. Rewrites every
+ * matching IRI, in any term position; see
+ * {@link namespaceNormalizationTransform}.
  */
 export declare function namespaceNormalizationPlugin(options: NamespaceNormalizationOptions): PipelinePlugin;
 //# sourceMappingURL=namespaceNormalization.d.ts.map

package/dist/plugin/namespaceNormalization.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"namespaceNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/namespaceNormalization.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AACrD,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAO5C,MAAM,WAAW,6BAA6B;IAC5C,iEAAiE;IACjE,IAAI,EAAE,MAAM,CAAC;IACb,yEAAyE;IACzE,EAAE,EAAE,MAAM,CAAC;CACZ;AAED;;;;;;;GAOG;AACH,wBAAgB,+BAA+B,CAC7C,OAAO,EAAE,6BAA6B,GACrC,aAAa,CAAC;IAAE,OAAO,EAAE,OAAO,CAAA;CAAE,CAAC,CAErC;AAED;;;;;;GAMG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,EAAE,6BAA6B,GACrC,cAAc,CAKhB"}
+{"version":3,"file":"namespaceNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/namespaceNormalization.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,uBAAuB,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAM9E,MAAM,WAAW,6BAA6B;IAC5C,iEAAiE;IACjE,IAAI,EAAE,MAAM,CAAC;IACb,4EAA4E;IAC5E,EAAE,EAAE,MAAM,CAAC;CACZ;AAED;;;;;;;;;;;;;GAaG;AACH,wBAAgB,+BAA+B,CAC7C,OAAO,EAAE,6BAA6B,GACrC,aAAa,CAAC,uBAAuB,CAAC,CAKxC;AAED;;;;;GAKG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,EAAE,6BAA6B,GACrC,cAAc,CAKhB"}

package/dist/plugin/namespaceNormalization.js

@@ -1,23 +1,31 @@
+import { canonicalizeIri } from '../namespaceAlias.js';
 import { DataFactory } from 'n3';
-import { _void } from '@tpluscode/rdf-ns-builders';
 const { namedNode, quad } = DataFactory;
 /**
- * Creates a QuadTransform that rewrites namespace prefixes in `void:class` and
- * `void:property` quad objects from {@link NamespaceNormalizationOptions.from}
- * to {@link NamespaceNormalizationOptions.to}.
+ * Creates a {@link QuadTransform} that rewrites every IRI in the
+ * {@link NamespaceNormalizationOptions.from} namespace to
+ * {@link NamespaceNormalizationOptions.to} — in subject, predicate and object
+ * position alike.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * This is a blanket, vocabulary-agnostic rewrite: it normalizes the namespace
+ * wherever it appears, with no knowledge of VoID or any other RDF shape. Use it
+ * to standardize a namespace across a dataset's own quads (for example
+ * `http://schema.org/` → `https://schema.org/` when mapping instance data to an
+ * application profile). Merging VoID partition _nodes_ that two namespace
+ * variants produced is a separate, shape-aware concern handled by
+ * `@lde/pipeline-void`.
  */
 export function namespaceNormalizationTransform(options) {
-    return (quads) => normalizeNamespace(quads, options);
+    const namespaceAliases = [
+        { canonical: options.to, alias: options.from },
+    ];
+    return (quads) => normalize(quads, namespaceAliases);
 }
 /**
- * Pipeline plugin that normalizes namespace prefixes in `void:class` and
- * `void:property` quad objects.
- *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * A generic {@link PipelinePlugin} that normalizes a namespace prefix across a
+ * stage's output via {@link PipelinePlugin.beforeStageWrite}. Rewrites every
+ * matching IRI, in any term position; see
+ * {@link namespaceNormalizationTransform}.
  */
 export function namespaceNormalizationPlugin(options) {
     return {
@@ -25,15 +33,25 @@ export function namespaceNormalizationPlugin(options) {
         beforeStageWrite: namespaceNormalizationTransform(options),
     };
 }
-async function* normalizeNamespace(quads, { from, to }) {
+function normalizeTerm(term, namespaceAliases) {
+    if (term.termType !== 'NamedNode') {
+        return term;
+    }
+    const canonical = canonicalizeIri(term.value, namespaceAliases);
+    return (canonical === term.value ? term : namedNode(canonical));
+}
+async function* normalize(quads, namespaceAliases) {
     for await (const q of quads) {
-        if ((q.predicate.equals(_void.class) || q.predicate.equals(_void.property)) &&
-            q.object.termType === 'NamedNode' &&
-            q.object.value.startsWith(from)) {
-            yield quad(q.subject, q.predicate, namedNode(to + q.object.value.slice(from.length)), q.graph);
+        const subject = normalizeTerm(q.subject, namespaceAliases);
+        const predicate = normalizeTerm(q.predicate, namespaceAliases);
+        const object = normalizeTerm(q.object, namespaceAliases);
+        if (subject === q.subject &&
+            predicate === q.predicate &&
+            object === q.object) {
+            yield q;
         }
         else {
-            yield q;
+            yield quad(subject, predicate, object, q.graph);
         }
     }
 }

package/dist/plugin/schemaOrgNormalization.d.ts

@@ -1,23 +1,24 @@
 import type { QuadTransform } from '../stage.js';
-import type { PipelinePlugin } from '../pipeline.js';
-import type { Dataset } from '@lde/dataset';
+import type { BeforeStageWriteContext, PipelinePlugin } from '../pipeline.js';
 export interface SchemaOrgNormalizationOptions {
     /** When true, normalizes `https://schema.org/` to `http://schema.org/` instead. */
     reverse?: boolean;
 }
-/** QuadTransform that normalizes `http://schema.org/` to `https://schema.org/` in `void:class` and `void:property` objects. */
-export declare const schemaOrgNormalizationTransform: QuadTransform<{
-    dataset: Dataset;
-}>;
 /**
- * Pipeline plugin that normalizes Schema.org namespace prefixes in `void:class`
- * and `void:property` quad objects.
- *
- * By default, rewrites `http://schema.org/` to `https://schema.org/`. Pass
- * `{ reverse: true }` to normalize in the opposite direction.
+ * A {@link QuadTransform} that normalizes `http://schema.org/` to
+ * `https://schema.org/` in every term position. See
+ * {@link namespaceNormalizationTransform}.
+ */
+export declare const schemaOrgNormalizationTransform: QuadTransform<BeforeStageWriteContext>;
+/**
+ * A generic {@link PipelinePlugin} that normalizes the Schema.org namespace
+ * across a stage's output.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * By default rewrites `http://schema.org/` to `https://schema.org/`; pass
+ * `{ reverse: true }` to normalize the other way. It is a blanket rewrite over
+ * every matching IRI and knows nothing about VoID — to merge the VoID partition
+ * nodes that mixed `http`/`https` variants produce, use
+ * `schemaOrgPartitionMergePlugin` from `@lde/pipeline-void` instead.
  */
 export declare function schemaOrgNormalizationPlugin(options?: SchemaOrgNormalizationOptions): PipelinePlugin;
 //# sourceMappingURL=schemaOrgNormalization.d.ts.map

package/dist/plugin/schemaOrgNormalization.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"schemaOrgNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/schemaOrgNormalization.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AACrD,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAS5C,MAAM,WAAW,6BAA6B;IAC5C,mFAAmF;IACnF,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,+HAA+H;AAC/H,eAAO,MAAM,+BAA+B,EAAE,aAAa,CAAC;IAC1D,OAAO,EAAE,OAAO,CAAC;CAClB,CAGC,CAAC;AAEH;;;;;;;;;GASG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,CAAC,EAAE,6BAA6B,GACtC,cAAc,CAOhB"}
+{"version":3,"file":"schemaOrgNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/schemaOrgNormalization.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,uBAAuB,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAS9E,MAAM,WAAW,6BAA6B;IAC5C,mFAAmF;IACnF,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED;;;;GAIG;AACH,eAAO,MAAM,+BAA+B,EAAE,aAAa,CAAC,uBAAuB,CAI/E,CAAC;AAEL;;;;;;;;;GASG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,CAAC,EAAE,6BAA6B,GACtC,cAAc,CAOhB"}

package/dist/plugin/schemaOrgNormalization.js

@@ -1,20 +1,24 @@
 import { namespaceNormalizationPlugin, namespaceNormalizationTransform, } from './namespaceNormalization.js';
 const HTTP_SCHEMA_ORG = 'http://schema.org/';
 const HTTPS_SCHEMA_ORG = 'https://schema.org/';
-/** QuadTransform that normalizes `http://schema.org/` to `https://schema.org/` in `void:class` and `void:property` objects. */
+/**
+ * A {@link QuadTransform} that normalizes `http://schema.org/` to
+ * `https://schema.org/` in every term position. See
+ * {@link namespaceNormalizationTransform}.
+ */
 export const schemaOrgNormalizationTransform = namespaceNormalizationTransform({
     from: HTTP_SCHEMA_ORG,
     to: HTTPS_SCHEMA_ORG,
 });
 /**
- * Pipeline plugin that normalizes Schema.org namespace prefixes in `void:class`
- * and `void:property` quad objects.
- *
- * By default, rewrites `http://schema.org/` to `https://schema.org/`. Pass
- * `{ reverse: true }` to normalize in the opposite direction.
+ * A generic {@link PipelinePlugin} that normalizes the Schema.org namespace
+ * across a stage's output.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * By default rewrites `http://schema.org/` to `https://schema.org/`; pass
+ * `{ reverse: true }` to normalize the other way. It is a blanket rewrite over
+ * every matching IRI and knows nothing about VoID — to merge the VoID partition
+ * nodes that mixed `http`/`https` variants produce, use
+ * `schemaOrgPartitionMergePlugin` from `@lde/pipeline-void` instead.
  */
 export function schemaOrgNormalizationPlugin(options) {
     const from = options?.reverse ? HTTPS_SCHEMA_ORG : HTTP_SCHEMA_ORG;

package/dist/writer/fileWriter.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"fileWriter.d.ts","sourceRoot":"","sources":["../../src/writer/fileWriter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAOvC,OAAO,EAAE,cAAc,EAAE,UAAU,EAAE,SAAS,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAE5E,MAAM,WAAW,iBAAiB;IAChC;;OAEG;IACH,SAAS,EAAE,MAAM,CAAC;IAClB;;;OAGG;IACH,MAAM,CAAC,EAAE,QAAQ,GAAG,WAAW,GAAG,SAAS,CAAC;IAC5C;;;OAGG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;OAGG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAClC;;;;;;;;OAQG;IACH,QAAQ,CAAC,EAAE,CAAC,OAAO,EAAE,OAAO,KAAK,GAAG,CAAC;CACtC;AAgBD;;;;;GAKG;AACH,MAAM,WAAW,aAAc,SAAQ,SAAS;IAC9C,KAAK,CAAC,OAAO,EAAE,OAAO,EAAE,OAAO,CAAC,EAAE,cAAc,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IACjE,KAAK,CAAC,OAAO,EAAE,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CACxC;AAED;;;;;;;;;;GAUG;AACH,qBAAa,UAAW,YAAW,MAAM;IACvC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IACnC,QAAQ,CAAC,MAAM,EAAE,QAAQ,GAAG,WAAW,GAAG,SAAS,CAAC;IACpD,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAS;IAC9C,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAyB;IACnD,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA4B;gBAE1C,OAAO,EAAE,iBAAiB;IAQhC,OAAO,CAAC,QAAQ,CAAC,EAAE,UAAU,GAAG,OAAO,CAAC,aAAa,CAAC;YAwB9C,UAAU;YAqCV,SAAS;YAOT,SAAS;YAgCT,WAAW;IAczB;;;;OAIG;YACW,cAAc;IAQ5B,aAAa,CAAC,OAAO,EAAE,OAAO,GAAG,MAAM;IAIvC,WAAW,CAAC,OAAO,EAAE,OAAO,GAAG,MAAM;IAQrC,OAAO,CAAC,WAAW;YAIL,eAAe;IA6B7B,OAAO,CAAC,YAAY;CAUrB"}
+{"version":3,"file":"fileWriter.d.ts","sourceRoot":"","sources":["../../src/writer/fileWriter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAOvC,OAAO,EAAE,cAAc,EAAE,UAAU,EAAE,SAAS,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAG5E,MAAM,WAAW,iBAAiB;IAChC;;OAEG;IACH,SAAS,EAAE,MAAM,CAAC;IAClB;;;OAGG;IACH,MAAM,CAAC,EAAE,QAAQ,GAAG,WAAW,GAAG,SAAS,CAAC;IAC5C;;;OAGG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;OAGG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAClC;;;;;;;;OAQG;IACH,QAAQ,CAAC,EAAE,CAAC,OAAO,EAAE,OAAO,KAAK,GAAG,CAAC;CACtC;AAgBD;;;;;GAKG;AACH,MAAM,WAAW,aAAc,SAAQ,SAAS;IAC9C,KAAK,CAAC,OAAO,EAAE,OAAO,EAAE,OAAO,CAAC,EAAE,cAAc,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IACjE,KAAK,CAAC,OAAO,EAAE,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CACxC;AAED;;;;;;;;;;GAUG;AACH,qBAAa,UAAW,YAAW,MAAM;IACvC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IACnC,QAAQ,CAAC,MAAM,EAAE,QAAQ,GAAG,WAAW,GAAG,SAAS,CAAC;IACpD,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAS;IAC9C,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAyB;IACnD,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA4B;gBAE1C,OAAO,EAAE,iBAAiB;IAQhC,OAAO,CAAC,QAAQ,CAAC,EAAE,UAAU,GAAG,OAAO,CAAC,aAAa,CAAC;YAwB9C,UAAU;YAsDV,SAAS;YAOT,SAAS;YAgCT,WAAW;IAczB;;;;OAIG;YACW,cAAc;IAQ5B,aAAa,CAAC,OAAO,EAAE,OAAO,GAAG,MAAM;IAIvC,WAAW,CAAC,OAAO,EAAE,OAAO,GAAG,MAAM;IAQrC,OAAO,CAAC,WAAW;YAIL,eAAe;IA6B7B,OAAO,CAAC,YAAY;CAUrB"}

package/dist/writer/fileWriter.js

@@ -3,6 +3,7 @@ import { mkdir, rename, rm } from 'node:fs/promises';
 import { join, dirname } from 'node:path';
 import filenamifyUrl from 'filenamify-url';
 import { DataFactory, Writer as N3Writer } from 'n3';
+import { skolemizeBlankNodes } from './skolemizeBlankNodes.js';
 const formatMap = {
     turtle: 'Turtle',
     'n-triples': 'N-Triples',
@@ -71,6 +72,22 @@ export class FileWriter {
         const addQuad = (quad) => n3Writer.addQuad(graphNode
             ? DataFactory.quad(quad.subject, quad.predicate, quad.object, graphNode)
             : quad);
+        // n-quads files are cat-ed into one served graph, where document-scoped
+        // blank-node labels from different datasets collapse into one node
+        // (ldelements/lde#478, dataset-knowledge-graph#352/#420). Skolemise blank
+        // nodes to dataset-scoped IRIs so they stay distinct. Hashing the write to
+        // scope the IRIs needs it buffered, so only n-quads pays that cost – Turtle
+        // and N-Triples (single-document, offline) stream through unchanged.
+        if (this.format === 'n-quads') {
+            const batch = [first.value];
+            for await (const quad of { [Symbol.asyncIterator]: () => iterator }) {
+                batch.push(quad);
+            }
+            for (const quad of skolemizeBlankNodes(batch, dataset.iri.toString())) {
+                addQuad(quad);
+            }
+            return;
+        }
         addQuad(first.value);
         for await (const quad of { [Symbol.asyncIterator]: () => iterator }) {
             addQuad(quad);

package/dist/writer/skolemizeBlankNodes.d.ts

@@ -0,0 +1,27 @@
+import type { Quad } from '@rdfjs/types';
+/**
+ * Rewrite every blank node in `quads` to a deterministic, dataset-scoped IRI,
+ * leaving the quads otherwise unchanged.
+ *
+ * A served store such as the Dataset Knowledge Graph assembles one graph per
+ * dataset by concatenating every per-dataset n-quads file into a single index
+ * (`qlever index` over the `cat` of all files). Document-scoped blank-node
+ * labels are not preserved across that concatenation: two datasets' `_:b0`
+ * collapse into one node, silently fusing unrelated measurements, activities
+ * and linksets across datasets (see ldelements/lde#478 and
+ * netwerk-digitaal-erfgoed/dataset-knowledge-graph#352 / #420). IRIs are never
+ * relabelled, so skolemising each blank node keeps distinct nodes distinct.
+ *
+ * Each blank node becomes `<dataset>/.well-known/skolem#<batch>-<label>`. The
+ * dataset IRI rules out fusion across datasets; `<batch>`, a content hash of
+ * this write's quads, rules out fusion across the separate writes that land in
+ * one dataset's file – their labels both restart at `b0`, but writes carrying
+ * different content hash differently. Two byte-identical writes collapse onto
+ * the same IRIs, which is correct: they carry the same nodes.
+ *
+ * @param quads - The quads to rewrite, possibly containing blank nodes.
+ * @param datasetIri - The dataset the quads belong to; scopes every minted IRI.
+ * @returns The same quads with every blank node replaced by a skolem IRI.
+ */
+export declare function skolemizeBlankNodes(quads: Quad[], datasetIri: string): Quad[];
+//# sourceMappingURL=skolemizeBlankNodes.d.ts.map

package/dist/writer/skolemizeBlankNodes.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"skolemizeBlankNodes.d.ts","sourceRoot":"","sources":["../../src/writer/skolemizeBlankNodes.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,IAAI,EAKL,MAAM,cAAc,CAAC;AAMtB;;;;;;;;;;;;;;;;;;;;;;;GAuBG;AACH,wBAAgB,mBAAmB,CAAC,KAAK,EAAE,IAAI,EAAE,EAAE,UAAU,EAAE,MAAM,GAAG,IAAI,EAAE,CAgB7E"}

package/dist/writer/skolemizeBlankNodes.js

@@ -0,0 +1,53 @@
+import { DataFactory } from 'n3';
+import { hashSuffix, skolemIri } from '@lde/dataset';
+const { namedNode, quad: makeQuad } = DataFactory;
+/**
+ * Rewrite every blank node in `quads` to a deterministic, dataset-scoped IRI,
+ * leaving the quads otherwise unchanged.
+ *
+ * A served store such as the Dataset Knowledge Graph assembles one graph per
+ * dataset by concatenating every per-dataset n-quads file into a single index
+ * (`qlever index` over the `cat` of all files). Document-scoped blank-node
+ * labels are not preserved across that concatenation: two datasets' `_:b0`
+ * collapse into one node, silently fusing unrelated measurements, activities
+ * and linksets across datasets (see ldelements/lde#478 and
+ * netwerk-digitaal-erfgoed/dataset-knowledge-graph#352 / #420). IRIs are never
+ * relabelled, so skolemising each blank node keeps distinct nodes distinct.
+ *
+ * Each blank node becomes `<dataset>/.well-known/skolem#<batch>-<label>`. The
+ * dataset IRI rules out fusion across datasets; `<batch>`, a content hash of
+ * this write's quads, rules out fusion across the separate writes that land in
+ * one dataset's file – their labels both restart at `b0`, but writes carrying
+ * different content hash differently. Two byte-identical writes collapse onto
+ * the same IRIs, which is correct: they carry the same nodes.
+ *
+ * @param quads - The quads to rewrite, possibly containing blank nodes.
+ * @param datasetIri - The dataset the quads belong to; scopes every minted IRI.
+ * @returns The same quads with every blank node replaced by a skolem IRI.
+ */
+export function skolemizeBlankNodes(quads, datasetIri) {
+    const base = `${datasetIri}/.well-known/skolem#${hashSuffix(fingerprint(quads))}`;
+    const skolemize = (term) => term.termType === 'BlankNode'
+        ? namedNode(skolemIri(base, term.value))
+        : term;
+    // A skolemised blank node is a NamedNode, valid in every term position, so the
+    // casts hold: skolemize only rewrites blank nodes and leaves other terms as-is.
+    return quads.map((quad) => makeQuad(skolemize(quad.subject), quad.predicate, skolemize(quad.object), skolemize(quad.graph)));
+}
+/** A deterministic string identifying a write's quads, for a per-write IRI segment. */
+function fingerprint(quads) {
+    return quads
+        .map((quad) => `${termKey(quad.subject)} ${quad.predicate.value} ${termKey(quad.object)}`)
+        .sort()
+        .join('\n');
+}
+function termKey(term) {
+    switch (term.termType) {
+        case 'BlankNode':
+            return `_:${term.value}`;
+        case 'Literal':
+            return `"${term.value}"^^<${term.datatype.value}>@${term.language}`;
+        default:
+            return `<${term.value}>`;
+    }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lde/pipeline",
-  "version": "0.33.4",
+  "version": "0.34.1",
   "repository": {
     "url": "git+https://github.com/ldelements/lde.git",
     "directory": "packages/pipeline"
@@ -39,7 +39,7 @@
     "fetch-sparql-endpoint": "^7.1.1",
     "filenamify-url": "^4.0.0",
     "is-network-error": "^1.3.2",
-    "n3": "^2.1.0",
+    "n3": "^2.1.1",
     "p-retry": "^8.0.0",
     "rdf-string": "^2.0.1",
     "tslib": "^2.3.0"