@lde/pipeline 0.33.3 → 0.34.0

package/README.md

@@ -390,51 +390,50 @@ Otherwise it imports (if needed), runs the stages, and writes an updated record.
 
 ### Plugins
 
-Plugins hook into the pipeline lifecycle via the `PipelinePlugin` interface. Register them in the `plugins` array when constructing a `Pipeline`.
+Plugins hook into the pipeline lifecycle via the `PipelinePlugin` interface. Register them in the `plugins` array when constructing a `Pipeline`. A plugin can implement either or both of two hooks, each a `QuadTransform`:
+
+- **`beforeStageWrite`** runs over each stage’s output as it is written – scoped to one stage’s quads for one dataset.
+- **`beforeDatasetWrite`** runs once over a dataset’s combined cross-stage output, just before it is written – scoped to the whole dataset, so it can reconcile quads that different stages produced. The output is streamed through a single queue rather than materialized, so memory stays bounded by whatever the transform itself buffers; it honors the writer’s run lifecycle (`flush`/`reset`/`commit`/`abort`), and a failure is isolated to the dataset being written.
+
+Both hooks are domain-agnostic – dedup, roll-ups, provenance stamping, or namespace rewriting can use them. This package ships `provenancePlugin()`, `namespaceNormalizationPlugin()`, and `schemaOrgNormalizationPlugin()` (all `beforeStageWrite` plugins). Plugins that must understand a particular RDF shape live with that vocabulary: for example, [@lde/pipeline-void](../pipeline-void) ships `schemaOrgPartitionMergePlugin()`, a `beforeDatasetWrite` plugin that merges the VoID partition _nodes_ that Schema.org `http`/`https` variants produce.
+
+#### `provenancePlugin()`
+
+Appends [PROV-O](https://www.w3.org/TR/prov-o/) provenance quads (`prov:Entity`, `prov:Activity`, `prov:startedAtTime`, `prov:endedAtTime`) to every stage’s output. The `prov:Activity` is a stable IRI keyed on `(dataset, stage)`, not a blank node, so activities stay distinct – and a re-run stays idempotent – when per-dataset outputs are merged into one graph (blank-node labels are not unique across separately serialised documents and would fuse unrelated activities).
 
 #### `namespaceNormalizationPlugin(options)`
 
-Generic plugin that rewrites namespace prefixes in `void:class` and `void:property` quad objects. Accepts `from` and `to` options specifying the source and target namespace URI prefixes. `void:vocabulary` quads are left unchanged so consumers can see which namespace the source dataset actually uses.
+Rewrites every IRI in the `from` namespace to the `to` namespace, in subject, predicate and object position alike. A blanket, vocabulary-agnostic rewrite for standardizing a namespace across a dataset’s own quads – for example when mapping instance data to an application profile.
 
 ```typescript
-import { namespaceNormalizationPlugin } from ‘@lde/pipeline’;
+import { namespaceNormalizationPlugin } from '@lde/pipeline';
 
 new Pipeline({
   // ...
   plugins: [
     namespaceNormalizationPlugin({
-      from: ‘http://example.org/’,
-      to: ‘https://example.org/’,
+      from: 'http://example.org/',
+      to: 'https://example.org/',
     }),
   ],
 });
 ```
 
-#### `provenancePlugin()`
-
-Appends [PROV-O](https://www.w3.org/TR/prov-o/) provenance quads (`prov:Entity`, `prov:Activity`, `prov:startedAtTime`, `prov:endedAtTime`) to every stage’s output. The `prov:Activity` is a stable IRI keyed on `(dataset, stage)`, not a blank node, so activities stay distinct – and a re-run stays idempotent – when per-dataset outputs are merged into one graph (blank-node labels are not unique across separately serialised documents and would fuse unrelated activities).
-
 #### `schemaOrgNormalizationPlugin(options?)`
 
-Normalizes Schema.org namespace prefixes in `void:class` and `void:property` quad objects. By default, rewrites `http://schema.org/` to `https://schema.org/`. Pass `{ reverse: true }` to normalize in the opposite direction (`https://` to `http://`). `void:vocabulary` quads are left unchanged so consumers can see which namespace the source dataset actually uses.
-
-This is a convenience wrapper around `namespaceNormalizationPlugin`.
+Convenience wrapper around `namespaceNormalizationPlugin` for the common case: rewrites `http://schema.org/` to `https://schema.org/` (pass `{ reverse: true }` for the opposite direction).
 
 ```typescript
-import { schemaOrgNormalizationPlugin, provenancePlugin } from ‘@lde/pipeline’;
+import { schemaOrgNormalizationPlugin, provenancePlugin } from '@lde/pipeline';
 
 new Pipeline({
   // ...
   plugins: [schemaOrgNormalizationPlugin(), provenancePlugin()],
 });
-
-// Or reverse: normalize https to http
-new Pipeline({
-  // ...
-  plugins: [schemaOrgNormalizationPlugin({reverse: true})],
-});
 ```
 
+This is a plain rewrite and knows nothing about VoID. If your pipeline emits VoID partitions and a dataset mixes both schema.org namespaces, rewriting the `void:class` objects alone still leaves two `void:classPartition` nodes for one class – use [`schemaOrgPartitionMergePlugin`](../pipeline-void) from `@lde/pipeline-void`, which also merges those nodes.
+
 ## Usage
 
 ```typescript

package/dist/index.d.ts

@@ -1,4 +1,5 @@
 export * from './validator.js';
+export * from './namespaceAlias.js';
 export * from './asyncQueue.js';
 export * from './batch.js';
 export * from './pipeline.js';
@@ -11,7 +12,7 @@ export * from './sparql/index.js';
 export * from './distribution/index.js';
 export * from './provenance/index.js';
 export * from './writer/index.js';
-export * from './plugin/namespaceNormalization.js';
 export * from './plugin/provenance.js';
+export * from './plugin/namespaceNormalization.js';
 export * from './plugin/schemaOrgNormalization.js';
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,gBAAgB,CAAC;AAC/B,cAAc,iBAAiB,CAAC;AAChC,cAAc,YAAY,CAAC;AAC3B,cAAc,eAAe,CAAC;AAC9B,cAAc,uBAAuB,CAAC;AACtC,cAAc,eAAe,CAAC;AAC9B,cAAc,YAAY,CAAC;AAC3B,cAAc,0BAA0B,CAAC;AACzC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,oCAAoC,CAAC;AACnD,cAAc,wBAAwB,CAAC;AACvC,cAAc,oCAAoC,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,gBAAgB,CAAC;AAC/B,cAAc,qBAAqB,CAAC;AACpC,cAAc,iBAAiB,CAAC;AAChC,cAAc,YAAY,CAAC;AAC3B,cAAc,eAAe,CAAC;AAC9B,cAAc,uBAAuB,CAAC;AACtC,cAAc,eAAe,CAAC;AAC9B,cAAc,YAAY,CAAC;AAC3B,cAAc,0BAA0B,CAAC;AACzC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AACtC,cAAc,mBAAmB,CAAC;AAClC,cAAc,wBAAwB,CAAC;AACvC,cAAc,oCAAoC,CAAC;AACnD,cAAc,oCAAoC,CAAC"}

package/dist/index.js

@@ -1,4 +1,5 @@
 export * from './validator.js';
+export * from './namespaceAlias.js';
 export * from './asyncQueue.js';
 export * from './batch.js';
 export * from './pipeline.js';
@@ -11,6 +12,6 @@ export * from './sparql/index.js';
 export * from './distribution/index.js';
 export * from './provenance/index.js';
 export * from './writer/index.js';
-export * from './plugin/namespaceNormalization.js';
 export * from './plugin/provenance.js';
+export * from './plugin/namespaceNormalization.js';
 export * from './plugin/schemaOrgNormalization.js';

package/dist/namespaceAlias.d.ts

@@ -0,0 +1,28 @@
+/**
+ * Declares that two namespaces should be treated as equivalent, working
+ * around vocabularies that publish under both HTTP and HTTPS variants of the
+ * same IRI (notably schema.org).
+ */
+export interface NamespaceAlias {
+    /**
+     * The namespace IRIs should be normalized to (e.g. `https://schema.org/`).
+     */
+    canonical: string;
+    /**
+     * The equivalent namespace that may appear in source data (e.g.
+     * `http://schema.org/`).
+     */
+    alias: string;
+}
+/**
+ * Rewrite an IRI in an alias namespace to its canonical form. IRIs outside
+ * every alias namespace are returned unchanged.
+ */
+export declare function canonicalizeIri(iri: string, namespaceAliases: readonly NamespaceAlias[]): string;
+/**
+ * All equivalent forms of an IRI under the given aliases: the IRI itself
+ * plus, when it falls in a canonical or alias namespace, the counterpart in
+ * the other namespace.
+ */
+export declare function aliasVariants(iri: string, namespaceAliases: readonly NamespaceAlias[]): string[];
+//# sourceMappingURL=namespaceAlias.d.ts.map

package/dist/namespaceAlias.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"namespaceAlias.d.ts","sourceRoot":"","sources":["../src/namespaceAlias.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AACH,MAAM,WAAW,cAAc;IAC7B;;OAEG;IACH,SAAS,EAAE,MAAM,CAAC;IAClB;;;OAGG;IACH,KAAK,EAAE,MAAM,CAAC;CACf;AAED;;;GAGG;AACH,wBAAgB,eAAe,CAC7B,GAAG,EAAE,MAAM,EACX,gBAAgB,EAAE,SAAS,cAAc,EAAE,GAC1C,MAAM,CAOR;AAED;;;;GAIG;AACH,wBAAgB,aAAa,CAC3B,GAAG,EAAE,MAAM,EACX,gBAAgB,EAAE,SAAS,cAAc,EAAE,GAC1C,MAAM,EAAE,CAUV"}

package/dist/namespaceAlias.js

@@ -0,0 +1,28 @@
+/**
+ * Rewrite an IRI in an alias namespace to its canonical form. IRIs outside
+ * every alias namespace are returned unchanged.
+ */
+export function canonicalizeIri(iri, namespaceAliases) {
+    for (const { canonical, alias } of namespaceAliases) {
+        if (iri.startsWith(alias)) {
+            return canonical + iri.slice(alias.length);
+        }
+    }
+    return iri;
+}
+/**
+ * All equivalent forms of an IRI under the given aliases: the IRI itself
+ * plus, when it falls in a canonical or alias namespace, the counterpart in
+ * the other namespace.
+ */
+export function aliasVariants(iri, namespaceAliases) {
+    for (const { canonical, alias } of namespaceAliases) {
+        if (iri.startsWith(canonical)) {
+            return [iri, alias + iri.slice(canonical.length)];
+        }
+        if (iri.startsWith(alias)) {
+            return [iri, canonical + iri.slice(alias.length)];
+        }
+    }
+    return [iri];
+}

package/dist/pipeline.d.ts

@@ -20,16 +20,40 @@ export interface BeforeStageWriteContext {
     dataset: Dataset;
     stage: string;
 }
+/**
+ * Context handed to a {@link PipelinePlugin.beforeDatasetWrite} transform: the
+ * `dataset` whose complete, cross-stage output is being written. Unlike
+ * {@link BeforeStageWriteContext} there is no `stage` — the transform sees one
+ * dataset's quads from every stage as a single stream.
+ */
+export interface BeforeDatasetWriteContext {
+    dataset: Dataset;
+}
 /** Plugin that hooks into pipeline lifecycle events. */
 export interface PipelinePlugin {
     name: string;
     /**
-     * Transform the merged, post-stage quad stream before writing (extension
-     * point 2: pipeline-wide, post-merge). The home of cross-cutting concerns
-     * – provenance, namespace normalisation – that apply regardless of which
-     * reader produced a quad.
+     * Transform a single stage's merged output before writing (extension point
+     * 2: per stage, post-merge). Runs once per stage per dataset, carrying the
+     * stage identity so it can mint stable per-`(dataset, stage)` IRIs. The home
+     * of stage-scoped concerns — provenance stamping, namespace rewriting of a
+     * stage's own quads.
      */
     beforeStageWrite?: QuadTransform<BeforeStageWriteContext>;
+    /**
+     * Transform a whole dataset's output before writing (extension point 3: per
+     * dataset, cross-stage). Runs once over the concatenation of every stage's
+     * output for one dataset, after all stages complete and before the write is
+     * flushed. The home of concerns that need to reconcile quads *across* stages
+     * — deduplication, cross-stage roll-ups, or merging partition nodes that
+     * different stages contribute to.
+     *
+     * The transform is a streaming {@link QuadTransform}: the pipeline pipes each
+     * stage's output through it via a bounded queue and never materializes the
+     * dataset, so a pass-through or streaming transform stays O(1). A transform
+     * that genuinely needs cross-stage state buffers only its own working set.
+     */
+    beforeDatasetWrite?: QuadTransform<BeforeDatasetWriteContext>;
 }
 export interface PipelineOptions {
     datasetSelector: DatasetSelector;
@@ -83,6 +107,7 @@ export declare class Pipeline {
     private readonly stages;
     private readonly writer;
     private readonly beforeStageWrite?;
+    private readonly beforeDatasetWrite?;
     private readonly distributionResolver;
     private readonly chaining?;
     private readonly reporter?;

package/dist/pipeline.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"pipeline.d.ts","sourceRoot":"","sources":["../src/pipeline.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,OAAO,EAAgB,MAAM,cAAc,CAAC;AAGrD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AACrD,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAChD,OAAO,KAAK,EAKV,MAAM,EACP,MAAM,oBAAoB,CAAC;AAE5B,OAAO,EACL,KAAK,oBAAoB,EAI1B,MAAM,4BAA4B,CAAC;AAKpC,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAC;AAY7D,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,0BAA0B,CAAC;AACpE,OAAO,KAAK,EAEV,gBAAgB,EACjB,MAAM,uBAAuB,CAAC;AAG/B,OAAO,EAEL,KAAK,aAAa,EACnB,MAAM,2BAA2B,CAAC;AAEnC;;;;;;;GAOG;AACH,MAAM,WAAW,uBAAuB;IACtC,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,EAAE,MAAM,CAAC;CACf;AAED,wDAAwD;AACxD,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb;;;;;OAKG;IACH,gBAAgB,CAAC,EAAE,aAAa,CAAC,uBAAuB,CAAC,CAAC;CAC3D;AAED,MAAM,WAAW,eAAe;IAC9B,eAAe,EAAE,eAAe,CAAC;IACjC,MAAM,EAAE,KAAK,EAAE,CAAC;IAChB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAAC;IAC3B,OAAO,CAAC,EAAE,cAAc,EAAE,CAAC;IAC3B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,oBAAoB,CAAC,EAAE,oBAAoB,CAAC;IAC5C,QAAQ,CAAC,EAAE;QACT,mBAAmB,EAAE,mBAAmB,CAAC;QACzC,SAAS,EAAE,MAAM,CAAC;KACnB,CAAC;IACF;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,gBAAgB,GAAG,SAAS,gBAAgB,EAAE,CAAC;IAC1D;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,eAAe,CAAC;IAClC;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;;;;;;OASG;IACH,OAAO,CAAC,EAAE,MAAM,aAAa,CAAC;CAC/B;AAqID,qBAAa,QAAQ;IACnB,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAS;IAC9B,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAkB;IAClD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAU;IACjC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;IAChC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAC,CAAyC;IAC3E,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAuB;IAC5D,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA8B;IACxD,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAmB;IAC7C,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAsB;IACrD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAkB;IACnD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAS;gBAE9B,OAAO,EAAE,eAAe;IA+C9B,GAAG,IAAI,OAAO,CAAC,IAAI,CAAC;YAyCZ,cAAc;IAqN5B,+EAA+E;YACjE,aAAa;YAmBb,gBAAgB;IAW9B,OAAO,CAAE,aAAa;IAOtB;;;;;OAKG;IACH,OAAO,CAAC,WAAW;IAMnB;;;;;;;;OAQG;IACH,OAAO,CAAC,0BAA0B;IA4BlC;;;;;OAKG;YACW,SAAS;IAuCvB;;;OAGG;YACW,QAAQ;IA0CtB,2EAA2E;YAC7D,eAAe;YAqBf,QAAQ;YAuEP,SAAS;CAczB"}
+{"version":3,"file":"pipeline.d.ts","sourceRoot":"","sources":["../src/pipeline.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,OAAO,EAAgB,MAAM,cAAc,CAAC;AAGrD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AACrD,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAEhD,OAAO,KAAK,EAKV,MAAM,EACP,MAAM,oBAAoB,CAAC;AAE5B,OAAO,EACL,KAAK,oBAAoB,EAI1B,MAAM,4BAA4B,CAAC;AAKpC,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAC;AAY7D,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,0BAA0B,CAAC;AACpE,OAAO,KAAK,EAEV,gBAAgB,EACjB,MAAM,uBAAuB,CAAC;AAG/B,OAAO,EAEL,KAAK,aAAa,EACnB,MAAM,2BAA2B,CAAC;AAEnC;;;;;;;GAOG;AACH,MAAM,WAAW,uBAAuB;IACtC,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,EAAE,MAAM,CAAC;CACf;AAED;;;;;GAKG;AACH,MAAM,WAAW,yBAAyB;IACxC,OAAO,EAAE,OAAO,CAAC;CAClB;AAED,wDAAwD;AACxD,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb;;;;;;OAMG;IACH,gBAAgB,CAAC,EAAE,aAAa,CAAC,uBAAuB,CAAC,CAAC;IAC1D;;;;;;;;;;;;OAYG;IACH,kBAAkB,CAAC,EAAE,aAAa,CAAC,yBAAyB,CAAC,CAAC;CAC/D;AAED,MAAM,WAAW,eAAe;IAC9B,eAAe,EAAE,eAAe,CAAC;IACjC,MAAM,EAAE,KAAK,EAAE,CAAC;IAChB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAAC;IAC3B,OAAO,CAAC,EAAE,cAAc,EAAE,CAAC;IAC3B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,oBAAoB,CAAC,EAAE,oBAAoB,CAAC;IAC5C,QAAQ,CAAC,EAAE;QACT,mBAAmB,EAAE,mBAAmB,CAAC;QACzC,SAAS,EAAE,MAAM,CAAC;KACnB,CAAC;IACF;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,gBAAgB,GAAG,SAAS,gBAAgB,EAAE,CAAC;IAC1D;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,eAAe,CAAC;IAClC;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;;;;;;OASG;IACH,OAAO,CAAC,EAAE,MAAM,aAAa,CAAC;CAC/B;AA0OD,qBAAa,QAAQ;IACnB,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAS;IAC9B,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAkB;IAClD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAU;IACjC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;IAChC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAC,CAAyC;IAC3E,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAC,CAA2C;IAC/E,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAuB;IAC5D,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA8B;IACxD,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAmB;IAC7C,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAsB;IACrD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAkB;IACnD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAS;gBAE9B,OAAO,EAAE,eAAe;IAyD9B,GAAG,IAAI,OAAO,CAAC,IAAI,CAAC;YA8CZ,cAAc;IAgO5B,+EAA+E;YACjE,aAAa;YAmBb,gBAAgB;IAW9B,OAAO,CAAE,aAAa;IAOtB;;;;;OAKG;IACH,OAAO,CAAC,WAAW;IAMnB;;;;;;;;OAQG;IACH,OAAO,CAAC,0BAA0B;IA4BlC;;;;;OAKG;YACW,SAAS;IAuCvB;;;OAGG;YACW,QAAQ;IA0CtB,2EAA2E;YAC7D,eAAe;YAqBf,QAAQ;YAuEP,SAAS;CAczB"}

package/dist/pipeline.js

@@ -1,6 +1,7 @@
 import { createReadStream } from 'node:fs';
 import { randomUUID } from 'node:crypto';
 import { StreamParser } from 'n3';
+import { AsyncQueue } from './asyncQueue.js';
 import { FileWriter } from './writer/fileWriter.js';
 import { NoDistributionAvailable, ResolvedDistribution, } from './distribution/resolver.js';
 import { SparqlDistributionResolver } from './distribution/index.js';
@@ -121,12 +122,102 @@ class TransformWriter {
         await this.inner.write(dataset, this.transform(quads, { dataset, stage: this.stage }));
     }
 }
+/** Discards a dataset's queued output; used to unblock the transform on reset. */
+class DatasetReset extends Error {
+}
+/**
+ * A {@link RunWriter} decorator that applies a {@link PipelinePlugin.beforeDatasetWrite}
+ * transform once over a dataset's complete, cross-stage output.
+ *
+ * Every stage's {@link write} for a dataset feeds a single {@link AsyncQueue};
+ * the transform consumes that queue as one stream and its output goes to the
+ * inner writer. The queue is created on the first write and closed on
+ * {@link flush}, so stage output streams through the transform (bounded by the
+ * queue's backpressure) rather than being materialized. Datasets are processed
+ * one at a time, so a single in-flight queue suffices.
+ */
+class DatasetTransformWriter {
+    inner;
+    transform;
+    current;
+    constructor(inner, transform) {
+        this.inner = inner;
+        this.transform = transform;
+    }
+    async write(dataset, quads) {
+        const { queue } = this.open(dataset);
+        for await (const quad of quads) {
+            await queue.push(quad);
+        }
+    }
+    async flush(dataset, outcome) {
+        let writeError;
+        if (this.current?.iri === dataset.iri.toString()) {
+            const { queue, innerWrite } = this.current;
+            this.current = undefined;
+            queue.close();
+            // Capture rather than propagate immediately, so the inner writer still
+            // gets its flush (as 'failed') to finalize/clean up this dataset.
+            try {
+                await innerWrite;
+            }
+            catch (error) {
+                writeError = error;
+            }
+        }
+        await this.inner.flush?.(dataset, writeError ? 'failed' : outcome);
+        if (writeError)
+            throw writeError;
+    }
+    async reset(dataset) {
+        await this.discardCurrent(dataset, new DatasetReset());
+        await this.inner.reset?.(dataset);
+    }
+    async commit() {
+        await this.inner.commit();
+    }
+    async abort(error) {
+        await this.discardCurrent(undefined, error);
+        await this.inner.abort(error);
+    }
+    open(dataset) {
+        const iri = dataset.iri.toString();
+        if (this.current?.iri === iri)
+            return this.current;
+        const queue = new AsyncQueue();
+        const innerWrite = this.inner.write(dataset, this.transform(queue, { dataset }));
+        // If the inner write (or the transform) fails, abort the queue so a
+        // producer blocked on push() is released and surfaces the error, instead
+        // of hanging forever waiting for a consumer that has stopped pulling. The
+        // rejection is re-observed (and re-thrown) by flush()/discardCurrent via
+        // `innerWrite`, so attaching this handler does not swallow it.
+        void innerWrite.catch((error) => queue.abort(error));
+        this.current = { iri, queue, innerWrite };
+        return this.current;
+    }
+    /**
+     * Abandon the in-flight dataset (the given one, or any when `dataset` is
+     * undefined): abort the queue so the transform and inner write terminate,
+     * then await them, swallowing the abort so it does not mask the caller's flow.
+     */
+    async discardCurrent(dataset, error) {
+        if (!this.current)
+            return;
+        if (dataset && this.current.iri !== dataset.iri.toString())
+            return;
+        const { queue, innerWrite } = this.current;
+        this.current = undefined;
+        queue.abort(error);
+        await innerWrite.catch(() => undefined);
+    }
+}
 export class Pipeline {
     name;
     datasetSelector;
     stages;
     writer;
     beforeStageWrite;
+    beforeDatasetWrite;
     distributionResolver;
     chaining;
     reporter;
@@ -156,6 +247,12 @@ export class Pipeline {
         this.beforeStageWrite = transforms?.length
             ? (quads, context) => transforms.reduce((q, fn) => fn(q, context), quads)
             : undefined;
+        const datasetTransforms = options.plugins
+            ?.map((p) => p.beforeDatasetWrite)
+            .filter((t) => t !== undefined);
+        this.beforeDatasetWrite = datasetTransforms?.length
+            ? (quads, context) => datasetTransforms.reduce((q, fn) => fn(q, context), quads)
+            : undefined;
         this.distributionResolver =
             options.distributionResolver ?? new SparqlDistributionResolver();
         this.chaining = options.chaining;
@@ -186,7 +283,12 @@ export class Pipeline {
             selectedSources: () => selectedSources,
             provenance: this.provenanceStore,
         };
-        const runWriter = await this.writer.openRun(context);
+        const openedWriter = await this.writer.openRun(context);
+        // Wrap the run writer so a whole-dataset transform sees every stage's
+        // output for a dataset as one stream before it is written.
+        const runWriter = this.beforeDatasetWrite
+            ? new DatasetTransformWriter(openedWriter, this.beforeDatasetWrite)
+            : openedWriter;
         try {
             for await (const dataset of datasets) {
                 selectedSources.push(dataset.iri.toString());
@@ -345,7 +447,16 @@ export class Pipeline {
             await this.distributionResolver.cleanup?.();
             unsubscribe?.();
         }
-        await runWriter.flush?.(dataset, stageFailed ? 'failed' : 'success');
+        try {
+            await runWriter.flush?.(dataset, stageFailed ? 'failed' : 'success');
+        }
+        catch (error) {
+            // A per-dataset writer failure must not abort the whole run: isolate it
+            // like a stage failure so the remaining datasets still process. The
+            // dataset is recorded 'failed' below and retried on the next run.
+            this.reporter?.stageFailed?.('write', error instanceof Error ? error : new Error(String(error)));
+            stageFailed = true;
+        }
         await this.reportValidators(dataset);
         // A dataset whose stages threw produced incomplete output; record it as
         // ‘failed’ rather than freezing a broken result under a ‘success’ record.

package/dist/plugin/namespaceNormalization.d.ts

@@ -1,29 +1,31 @@
 import type { QuadTransform } from '../stage.js';
-import type { PipelinePlugin } from '../pipeline.js';
-import type { Dataset } from '@lde/dataset';
+import type { BeforeStageWriteContext, PipelinePlugin } from '../pipeline.js';
 export interface NamespaceNormalizationOptions {
     /** Namespace URI prefix to match (e.g. `http://schema.org/`). */
     from: string;
-    /** Namespace URI prefix to replace with (e.g. `https://schema.org/`). */
+    /** Namespace URI prefix to replace it with (e.g. `https://schema.org/`). */
     to: string;
 }
 /**
- * Creates a QuadTransform that rewrites namespace prefixes in `void:class` and
- * `void:property` quad objects from {@link NamespaceNormalizationOptions.from}
- * to {@link NamespaceNormalizationOptions.to}.
+ * Creates a {@link QuadTransform} that rewrites every IRI in the
+ * {@link NamespaceNormalizationOptions.from} namespace to
+ * {@link NamespaceNormalizationOptions.to} — in subject, predicate and object
+ * position alike.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * This is a blanket, vocabulary-agnostic rewrite: it normalizes the namespace
+ * wherever it appears, with no knowledge of VoID or any other RDF shape. Use it
+ * to standardize a namespace across a dataset's own quads (for example
+ * `http://schema.org/` → `https://schema.org/` when mapping instance data to an
+ * application profile). Merging VoID partition _nodes_ that two namespace
+ * variants produced is a separate, shape-aware concern handled by
+ * `@lde/pipeline-void`.
  */
-export declare function namespaceNormalizationTransform(options: NamespaceNormalizationOptions): QuadTransform<{
-    dataset: Dataset;
-}>;
+export declare function namespaceNormalizationTransform(options: NamespaceNormalizationOptions): QuadTransform<BeforeStageWriteContext>;
 /**
- * Pipeline plugin that normalizes namespace prefixes in `void:class` and
- * `void:property` quad objects.
- *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * A generic {@link PipelinePlugin} that normalizes a namespace prefix across a
+ * stage's output via {@link PipelinePlugin.beforeStageWrite}. Rewrites every
+ * matching IRI, in any term position; see
+ * {@link namespaceNormalizationTransform}.
  */
 export declare function namespaceNormalizationPlugin(options: NamespaceNormalizationOptions): PipelinePlugin;
 //# sourceMappingURL=namespaceNormalization.d.ts.map

package/dist/plugin/namespaceNormalization.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"namespaceNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/namespaceNormalization.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AACrD,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAO5C,MAAM,WAAW,6BAA6B;IAC5C,iEAAiE;IACjE,IAAI,EAAE,MAAM,CAAC;IACb,yEAAyE;IACzE,EAAE,EAAE,MAAM,CAAC;CACZ;AAED;;;;;;;GAOG;AACH,wBAAgB,+BAA+B,CAC7C,OAAO,EAAE,6BAA6B,GACrC,aAAa,CAAC;IAAE,OAAO,EAAE,OAAO,CAAA;CAAE,CAAC,CAErC;AAED;;;;;;GAMG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,EAAE,6BAA6B,GACrC,cAAc,CAKhB"}
+{"version":3,"file":"namespaceNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/namespaceNormalization.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,uBAAuB,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAM9E,MAAM,WAAW,6BAA6B;IAC5C,iEAAiE;IACjE,IAAI,EAAE,MAAM,CAAC;IACb,4EAA4E;IAC5E,EAAE,EAAE,MAAM,CAAC;CACZ;AAED;;;;;;;;;;;;;GAaG;AACH,wBAAgB,+BAA+B,CAC7C,OAAO,EAAE,6BAA6B,GACrC,aAAa,CAAC,uBAAuB,CAAC,CAKxC;AAED;;;;;GAKG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,EAAE,6BAA6B,GACrC,cAAc,CAKhB"}

package/dist/plugin/namespaceNormalization.js

@@ -1,23 +1,31 @@
+import { canonicalizeIri } from '../namespaceAlias.js';
 import { DataFactory } from 'n3';
-import { _void } from '@tpluscode/rdf-ns-builders';
 const { namedNode, quad } = DataFactory;
 /**
- * Creates a QuadTransform that rewrites namespace prefixes in `void:class` and
- * `void:property` quad objects from {@link NamespaceNormalizationOptions.from}
- * to {@link NamespaceNormalizationOptions.to}.
+ * Creates a {@link QuadTransform} that rewrites every IRI in the
+ * {@link NamespaceNormalizationOptions.from} namespace to
+ * {@link NamespaceNormalizationOptions.to} — in subject, predicate and object
+ * position alike.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * This is a blanket, vocabulary-agnostic rewrite: it normalizes the namespace
+ * wherever it appears, with no knowledge of VoID or any other RDF shape. Use it
+ * to standardize a namespace across a dataset's own quads (for example
+ * `http://schema.org/` → `https://schema.org/` when mapping instance data to an
+ * application profile). Merging VoID partition _nodes_ that two namespace
+ * variants produced is a separate, shape-aware concern handled by
+ * `@lde/pipeline-void`.
  */
 export function namespaceNormalizationTransform(options) {
-    return (quads) => normalizeNamespace(quads, options);
+    const namespaceAliases = [
+        { canonical: options.to, alias: options.from },
+    ];
+    return (quads) => normalize(quads, namespaceAliases);
 }
 /**
- * Pipeline plugin that normalizes namespace prefixes in `void:class` and
- * `void:property` quad objects.
- *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * A generic {@link PipelinePlugin} that normalizes a namespace prefix across a
+ * stage's output via {@link PipelinePlugin.beforeStageWrite}. Rewrites every
+ * matching IRI, in any term position; see
+ * {@link namespaceNormalizationTransform}.
  */
 export function namespaceNormalizationPlugin(options) {
     return {
@@ -25,15 +33,25 @@ export function namespaceNormalizationPlugin(options) {
         beforeStageWrite: namespaceNormalizationTransform(options),
     };
 }
-async function* normalizeNamespace(quads, { from, to }) {
+function normalizeTerm(term, namespaceAliases) {
+    if (term.termType !== 'NamedNode') {
+        return term;
+    }
+    const canonical = canonicalizeIri(term.value, namespaceAliases);
+    return (canonical === term.value ? term : namedNode(canonical));
+}
+async function* normalize(quads, namespaceAliases) {
     for await (const q of quads) {
-        if ((q.predicate.equals(_void.class) || q.predicate.equals(_void.property)) &&
-            q.object.termType === 'NamedNode' &&
-            q.object.value.startsWith(from)) {
-            yield quad(q.subject, q.predicate, namedNode(to + q.object.value.slice(from.length)), q.graph);
+        const subject = normalizeTerm(q.subject, namespaceAliases);
+        const predicate = normalizeTerm(q.predicate, namespaceAliases);
+        const object = normalizeTerm(q.object, namespaceAliases);
+        if (subject === q.subject &&
+            predicate === q.predicate &&
+            object === q.object) {
+            yield q;
         }
         else {
-            yield q;
+            yield quad(subject, predicate, object, q.graph);
         }
     }
 }

package/dist/plugin/schemaOrgNormalization.d.ts

@@ -1,23 +1,24 @@
 import type { QuadTransform } from '../stage.js';
-import type { PipelinePlugin } from '../pipeline.js';
-import type { Dataset } from '@lde/dataset';
+import type { BeforeStageWriteContext, PipelinePlugin } from '../pipeline.js';
 export interface SchemaOrgNormalizationOptions {
     /** When true, normalizes `https://schema.org/` to `http://schema.org/` instead. */
     reverse?: boolean;
 }
-/** QuadTransform that normalizes `http://schema.org/` to `https://schema.org/` in `void:class` and `void:property` objects. */
-export declare const schemaOrgNormalizationTransform: QuadTransform<{
-    dataset: Dataset;
-}>;
 /**
- * Pipeline plugin that normalizes Schema.org namespace prefixes in `void:class`
- * and `void:property` quad objects.
- *
- * By default, rewrites `http://schema.org/` to `https://schema.org/`. Pass
- * `{ reverse: true }` to normalize in the opposite direction.
+ * A {@link QuadTransform} that normalizes `http://schema.org/` to
+ * `https://schema.org/` in every term position. See
+ * {@link namespaceNormalizationTransform}.
+ */
+export declare const schemaOrgNormalizationTransform: QuadTransform<BeforeStageWriteContext>;
+/**
+ * A generic {@link PipelinePlugin} that normalizes the Schema.org namespace
+ * across a stage's output.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * By default rewrites `http://schema.org/` to `https://schema.org/`; pass
+ * `{ reverse: true }` to normalize the other way. It is a blanket rewrite over
+ * every matching IRI and knows nothing about VoID — to merge the VoID partition
+ * nodes that mixed `http`/`https` variants produce, use
+ * `schemaOrgPartitionMergePlugin` from `@lde/pipeline-void` instead.
  */
 export declare function schemaOrgNormalizationPlugin(options?: SchemaOrgNormalizationOptions): PipelinePlugin;
 //# sourceMappingURL=schemaOrgNormalization.d.ts.map

package/dist/plugin/schemaOrgNormalization.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"schemaOrgNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/schemaOrgNormalization.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AACrD,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAS5C,MAAM,WAAW,6BAA6B;IAC5C,mFAAmF;IACnF,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,+HAA+H;AAC/H,eAAO,MAAM,+BAA+B,EAAE,aAAa,CAAC;IAC1D,OAAO,EAAE,OAAO,CAAC;CAClB,CAGC,CAAC;AAEH;;;;;;;;;GASG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,CAAC,EAAE,6BAA6B,GACtC,cAAc,CAOhB"}
+{"version":3,"file":"schemaOrgNormalization.d.ts","sourceRoot":"","sources":["../../src/plugin/schemaOrgNormalization.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,KAAK,EAAE,uBAAuB,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAS9E,MAAM,WAAW,6BAA6B;IAC5C,mFAAmF;IACnF,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED;;;;GAIG;AACH,eAAO,MAAM,+BAA+B,EAAE,aAAa,CAAC,uBAAuB,CAI/E,CAAC;AAEL;;;;;;;;;GASG;AACH,wBAAgB,4BAA4B,CAC1C,OAAO,CAAC,EAAE,6BAA6B,GACtC,cAAc,CAOhB"}

package/dist/plugin/schemaOrgNormalization.js

@@ -1,20 +1,24 @@
 import { namespaceNormalizationPlugin, namespaceNormalizationTransform, } from './namespaceNormalization.js';
 const HTTP_SCHEMA_ORG = 'http://schema.org/';
 const HTTPS_SCHEMA_ORG = 'https://schema.org/';
-/** QuadTransform that normalizes `http://schema.org/` to `https://schema.org/` in `void:class` and `void:property` objects. */
+/**
+ * A {@link QuadTransform} that normalizes `http://schema.org/` to
+ * `https://schema.org/` in every term position. See
+ * {@link namespaceNormalizationTransform}.
+ */
 export const schemaOrgNormalizationTransform = namespaceNormalizationTransform({
     from: HTTP_SCHEMA_ORG,
     to: HTTPS_SCHEMA_ORG,
 });
 /**
- * Pipeline plugin that normalizes Schema.org namespace prefixes in `void:class`
- * and `void:property` quad objects.
- *
- * By default, rewrites `http://schema.org/` to `https://schema.org/`. Pass
- * `{ reverse: true }` to normalize in the opposite direction.
+ * A generic {@link PipelinePlugin} that normalizes the Schema.org namespace
+ * across a stage's output.
  *
- * `void:vocabulary` quads are left unchanged so consumers can see which
- * namespace the source dataset actually uses.
+ * By default rewrites `http://schema.org/` to `https://schema.org/`; pass
+ * `{ reverse: true }` to normalize the other way. It is a blanket rewrite over
+ * every matching IRI and knows nothing about VoID — to merge the VoID partition
+ * nodes that mixed `http`/`https` variants produce, use
+ * `schemaOrgPartitionMergePlugin` from `@lde/pipeline-void` instead.
  */
 export function schemaOrgNormalizationPlugin(options) {
     const from = options?.reverse ? HTTPS_SCHEMA_ORG : HTTP_SCHEMA_ORG;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lde/pipeline",
-  "version": "0.33.3",
+  "version": "0.34.0",
   "repository": {
     "url": "git+https://github.com/ldelements/lde.git",
     "directory": "packages/pipeline"