@lde/pipeline-void 0.31.3 → 0.31.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,14 +10,15 @@ Returns all VoID stages in their recommended execution order. The ordering is op
10
10
 
11
11
  Accepts an optional `VoidStagesOptions` object:
12
12
 
13
- | Option | Default | Description |
14
- | ---------------- | ------- | --------------------------------------------------------------------------------------------------------------- |
15
- | `batchSize` | 10 | Maximum class bindings per reader call (per-class stages only) |
16
- | `maxConcurrency` | 10 | Maximum concurrent in-flight reader batches (per-class stages only) |
17
- | `perClass` | — | Override per-class iteration for all five per-class stages |
18
- | `uriSpaces` | — | When provided, includes the object URI space stage |
19
- | `vocabularies` | — | Additional vocabulary namespace URIs to detect beyond the built-in defaults |
20
- | `transforms` | — | Transforms to attach to bundled stages, keyed by `VOID_STAGE_NAMES` (see [Stage transforms](#stage-transforms)) |
13
+ | Option | Default | Description |
14
+ | ------------------ | ------- | --------------------------------------------------------------------------------------------------------------- |
15
+ | `batchSize` | 10 | Maximum class bindings per reader call (per-class stages only) |
16
+ | `maxConcurrency` | 10 | Maximum concurrent in-flight reader batches (per-class stages only) |
17
+ | `perClass` | — | Override per-class iteration for all five per-class stages |
18
+ | `uriSpaces` | — | When provided, includes the object URI space stage |
19
+ | `vocabularies` | — | Additional vocabulary namespace URIs to detect beyond the built-in defaults |
20
+ | `transforms` | — | Transforms to attach to bundled stages, keyed by `VOID_STAGE_NAMES` (see [Stage transforms](#stage-transforms)) |
21
+ | `namespaceAliases` | — | Namespace pairs to treat as equivalent when partitioning (see [Namespace aliases](#namespace-aliases)) |
21
22
 
22
23
  Per-request timeouts are configured at the `Pipeline` level via `PipelineOptions.timeout`, not per VoID stage.
23
24
 
@@ -72,6 +73,29 @@ Global and domain-specific factories accept `VoidStageOptions` (`transform`) and
72
73
  | `detectVocabularies()` | [`entity-properties.rq`](queries/entity-properties.rq) — Entity properties with automatic `void:vocabulary` detection. Accepts `DetectVocabulariesOptions` with an optional `vocabularies` array to extend the built-in defaults. |
73
74
  | `uriSpaces(uriSpaceMap)` | [`object-uri-space.rq`](queries/object-uri-space.rq) — Object URI namespace linksets, aggregated against a provided URI space map |
74
75
 
76
+ ## Namespace aliases
77
+
78
+ Some vocabularies publish under both HTTP and HTTPS variants of the same namespace (notably schema.org), and datasets mix them. Without normalization, each variant gets its own `void:classPartition`/`void:propertyPartition`, so consumers see two partitions for what is semantically one class.
79
+
80
+ Pass `namespaceAliases` to merge the variants into one partition per canonical class/property:
81
+
82
+ ```typescript
83
+ const stages = await voidStages({
84
+ namespaceAliases: [
85
+ { canonical: 'https://schema.org/', alias: 'http://schema.org/' },
86
+ ],
87
+ });
88
+ ```
89
+
90
+ Merging happens **after** aggregation, in a single partition-merge transform (`mergeNamespaceVariants`) attached to the class and property partition stages: it re-mints each partition IRI from its canonical key components, collapses the duplicates, and sums `void:entities` / `void:triples`. The analysis queries stay plain — except `class-properties-objects.rq`, which normalizes at query time because its `void:distinctObjects` is a distinct-count over object values that overlap across variants and so cannot be recovered by summing. See [ADR 7](../../docs/decisions/0007-merge-namespace-alias-partitions.md).
91
+
92
+ Summing pre-aggregated counts is exact only under two assumptions, both of which hold for the schema.org datasets that motivate this (the variants appear as disjoint subsets):
93
+
94
+ - **Subject/class disjointness** — no resource is typed under both namespace variants of a class (guards the class-partition and triple-count sums).
95
+ - **Predicate-namespace disjointness** — no subject uses both variants of the same property (guards the property-partition entity sum).
96
+
97
+ A resource typed under both variants over-counts its class entities. With no aliases configured the transform is a no-op. The top-level property partitions from `entity-properties.rq` and `void:vocabulary` detection keep the source namespaces, so consumers can still see which namespace the dataset actually uses.
98
+
75
99
  ## Stage transforms
76
100
 
77
101
  A VoID stage decorates its reader’s output with a `QuadTransform<ReaderContext>` attached as data (see [@lde/pipeline](../pipeline)’s extension model and [ADR 2](../../docs/decisions/0002-unify-pipeline-extension-on-quad-transforms.md)). It runs once per reader call and may fire its own SPARQL queries against the `distribution` in scope — so write it to accept being called more than once: a global stage calls it once over the complete output, a per-class stage with batching enabled once per batch (one class at `batchSize: 1`).
package/dist/index.d.ts CHANGED
@@ -1,6 +1,8 @@
1
1
  export { Stage, NotSupported } from '@lde/pipeline';
2
- export type { AttachedReader, ReaderContext, QuadTransform, } from '@lde/pipeline';
2
+ export type { AttachedReader, NamespaceAlias, ReaderContext, QuadTransform, } from '@lde/pipeline';
3
3
  export * from './stage.js';
4
+ export * from './namespaceAliases.js';
5
+ export * from './partitionMerge.js';
4
6
  export * from './vocabularyTransform.js';
5
7
  export * from './uriSpaceTransform.js';
6
8
  //# sourceMappingURL=index.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AACpD,YAAY,EACV,cAAc,EACd,aAAa,EACb,aAAa,GACd,MAAM,eAAe,CAAC;AACvB,cAAc,YAAY,CAAC;AAC3B,cAAc,0BAA0B,CAAC;AACzC,cAAc,wBAAwB,CAAC"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AACpD,YAAY,EACV,cAAc,EACd,cAAc,EACd,aAAa,EACb,aAAa,GACd,MAAM,eAAe,CAAC;AACvB,cAAc,YAAY,CAAC;AAC3B,cAAc,uBAAuB,CAAC;AACtC,cAAc,qBAAqB,CAAC;AACpC,cAAc,0BAA0B,CAAC;AACzC,cAAc,wBAAwB,CAAC"}
package/dist/index.js CHANGED
@@ -1,4 +1,6 @@
1
1
  export { Stage, NotSupported } from '@lde/pipeline';
2
2
  export * from './stage.js';
3
+ export * from './namespaceAliases.js';
4
+ export * from './partitionMerge.js';
3
5
  export * from './vocabularyTransform.js';
4
6
  export * from './uriSpaceTransform.js';
@@ -0,0 +1,23 @@
1
+ import { type NamespaceAlias, type Reader, type VariableBindings } from '@lde/pipeline';
2
+ /**
3
+ * Replace every {@link NORMALIZATION_MARKER} in a query template with the
4
+ * normalization expression for its raw variable.
5
+ */
6
+ export declare function substituteNormalizationMarkers(query: string, namespaceAliases: readonly NamespaceAlias[]): string;
7
+ /**
8
+ * Canonicalize and deduplicate the `?class` bindings a class selector
9
+ * yields, so every namespace-alias variant of a class becomes one item and
10
+ * its variants are queried together in one batch (split across batches,
11
+ * each batch would emit its own partial counts for the same partition IRI).
12
+ */
13
+ export declare function canonicalizeClassBindings(rows: AsyncIterable<VariableBindings>, namespaceAliases: readonly NamespaceAlias[]): AsyncIterable<VariableBindings>;
14
+ /**
15
+ * Decorate a {@link Reader} so each canonical `?class` binding is expanded to
16
+ * one `?class` binding per namespace-alias variant. The per-class VoID queries
17
+ * match `?s a ?class`, so this makes them pick up instances typed under either
18
+ * namespace; the partition-merge transform then collapses the variants. The
19
+ * expansion happens within one executor call, so the variants of a class are
20
+ * co-located in a single batch — a prerequisite for the per-stage merge.
21
+ */
22
+ export declare function withAliasVariantBindings(inner: Reader, namespaceAliases: readonly NamespaceAlias[]): Reader;
23
+ //# sourceMappingURL=namespaceAliases.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"namespaceAliases.d.ts","sourceRoot":"","sources":["../src/namespaceAliases.ts"],"names":[],"mappings":"AAAA,OAAO,EAGL,KAAK,cAAc,EACnB,KAAK,MAAM,EACX,KAAK,gBAAgB,EACtB,MAAM,eAAe,CAAC;AAcvB;;;GAGG;AACH,wBAAgB,8BAA8B,CAC5C,KAAK,EAAE,MAAM,EACb,gBAAgB,EAAE,SAAS,cAAc,EAAE,GAC1C,MAAM,CAIR;AAED;;;;;GAKG;AACH,wBAAuB,yBAAyB,CAC9C,IAAI,EAAE,aAAa,CAAC,gBAAgB,CAAC,EACrC,gBAAgB,EAAE,SAAS,cAAc,EAAE,GAC1C,aAAa,CAAC,gBAAgB,CAAC,CAQjC;AAED;;;;;;;GAOG;AACH,wBAAgB,wBAAwB,CACtC,KAAK,EAAE,MAAM,EACb,gBAAgB,EAAE,SAAS,cAAc,EAAE,GAC1C,MAAM,CAWR"}
@@ -0,0 +1,77 @@
1
+ import { aliasVariants, canonicalizeIri, } from '@lde/pipeline';
2
+ import { assertSafeIri } from '@lde/dataset';
3
+ import { DataFactory } from 'n3';
4
+ const { namedNode } = DataFactory;
5
+ /**
6
+ * Marker in a VoID query template that expands to a SPARQL expression
7
+ * normalizing the named raw variable to its canonical namespace, e.g.
8
+ * `BIND(#normalized:rawClass# AS ?class)`. With no aliases configured the
9
+ * marker expands to the raw variable itself.
10
+ */
11
+ const NORMALIZATION_MARKER = /#normalized:([A-Za-z][A-Za-z0-9]*)#/g;
12
+ /**
13
+ * Replace every {@link NORMALIZATION_MARKER} in a query template with the
14
+ * normalization expression for its raw variable.
15
+ */
16
+ export function substituteNormalizationMarkers(query, namespaceAliases) {
17
+ return query.replace(NORMALIZATION_MARKER, (_match, rawVariable) => normalizedExpression(rawVariable, namespaceAliases));
18
+ }
19
+ /**
20
+ * Canonicalize and deduplicate the `?class` bindings a class selector
21
+ * yields, so every namespace-alias variant of a class becomes one item and
22
+ * its variants are queried together in one batch (split across batches,
23
+ * each batch would emit its own partial counts for the same partition IRI).
24
+ */
25
+ export async function* canonicalizeClassBindings(rows, namespaceAliases) {
26
+ const seen = new Set();
27
+ for await (const row of rows) {
28
+ const canonical = canonicalizeIri(row.class.value, namespaceAliases);
29
+ if (seen.has(canonical))
30
+ continue;
31
+ seen.add(canonical);
32
+ yield { class: namedNode(canonical) };
33
+ }
34
+ }
35
+ /**
36
+ * Decorate a {@link Reader} so each canonical `?class` binding is expanded to
37
+ * one `?class` binding per namespace-alias variant. The per-class VoID queries
38
+ * match `?s a ?class`, so this makes them pick up instances typed under either
39
+ * namespace; the partition-merge transform then collapses the variants. The
40
+ * expansion happens within one executor call, so the variants of a class are
41
+ * co-located in a single batch — a prerequisite for the per-stage merge.
42
+ */
43
+ export function withAliasVariantBindings(inner, namespaceAliases) {
44
+ return {
45
+ read(dataset, distribution, options) {
46
+ const bindings = options?.bindings?.flatMap((row) => aliasVariants(row.class.value, namespaceAliases).map((iri) => ({
47
+ class: namedNode(iri),
48
+ })));
49
+ return inner.read(dataset, distribution, { ...options, bindings });
50
+ },
51
+ };
52
+ }
53
+ /**
54
+ * SPARQL expression rewriting `?rawVariable` from any alias namespace to its
55
+ * canonical namespace: nested `IF(STRSTARTS(...), IRI(CONCAT(...)), ...)`
56
+ * per alias, or the bare variable when no aliases are configured.
57
+ */
58
+ function normalizedExpression(rawVariable, namespaceAliases) {
59
+ let expression = `?${rawVariable}`;
60
+ // Build from the inside out so the first alias is the outermost check.
61
+ for (const { canonical, alias } of [...namespaceAliases].reverse()) {
62
+ assertSafeSparqlString(canonical);
63
+ assertSafeSparqlString(alias);
64
+ expression = `IF(STRSTARTS(STR(?${rawVariable}), "${alias}"), IRI(CONCAT("${canonical}", STRAFTER(STR(?${rawVariable}), "${alias}"))), ${expression})`;
65
+ }
66
+ return expression;
67
+ }
68
+ /**
69
+ * Namespaces are interpolated into double-quoted SPARQL string literals, so
70
+ * beyond {@link assertSafeIri} they must not contain quotes or backslashes.
71
+ */
72
+ function assertSafeSparqlString(namespace) {
73
+ assertSafeIri(namespace);
74
+ if (namespace.includes('"') || namespace.includes('\\')) {
75
+ throw new Error(`Namespace contains unsafe characters and cannot be interpolated into SPARQL: ${namespace}`);
76
+ }
77
+ }
@@ -0,0 +1,37 @@
1
+ import { type NamespaceAlias, type QuadTransform, type ReaderContext } from '@lde/pipeline';
2
+ /**
3
+ * A {@link QuadTransform} that merges the `void:classPartition` /
4
+ * `void:propertyPartition` subtrees of namespace-alias variants (e.g.
5
+ * `http://schema.org/CreativeWork` and `https://schema.org/CreativeWork`) into
6
+ * a single partition per canonical class/property.
7
+ *
8
+ * VoID partitions are keyed by an opaque `MD5(class[, property[, …]])` IRI, so
9
+ * two namespace variants yield two partition nodes that both, after
10
+ * canonicalization, describe the same class. This transform re-mints every
11
+ * partition IRI from its **canonical** key components (replicating the queries’
12
+ * SPARQL `MD5(CONCAT(STR(…)))`), collapses the duplicates, and **sums** their
13
+ * numeric measures (`void:entities`, `void:triples`).
14
+ *
15
+ * ## Correctness assumptions
16
+ *
17
+ * Summing pre-aggregated counts is exact only under these assumptions; the
18
+ * queries whose measures cannot be safely summed keep their normalization at
19
+ * query time instead (notably `class-properties-objects.rq`’s
20
+ * `void:distinctObjects`, which this transform never sees):
21
+ *
22
+ * - **Subject/class disjointness** — no resource is typed under two namespace
23
+ * variants of the same class. Guards the `void:entities` sum on class
24
+ * partitions and every `void:triples` sum (a doubly-typed resource’s triples
25
+ * would otherwise count under both variants).
26
+ * - **Predicate-namespace disjointness** — no subject uses two namespace
27
+ * variants of the same property (e.g. both `http://schema.org/name` and
28
+ * `https://schema.org/name`). Guards the `void:entities` sum on property
29
+ * partitions.
30
+ *
31
+ * With no aliases configured the transform is a no-op.
32
+ *
33
+ * @see substituteNormalizationMarkers for the query-time normalization used
34
+ * where summing is not safe.
35
+ */
36
+ export declare function mergeNamespaceVariants(namespaceAliases: readonly NamespaceAlias[]): QuadTransform<ReaderContext>;
37
+ //# sourceMappingURL=partitionMerge.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"partitionMerge.d.ts","sourceRoot":"","sources":["../src/partitionMerge.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,cAAc,EACnB,KAAK,aAAa,EAClB,KAAK,aAAa,EACnB,MAAM,eAAe,CAAC;AAyCvB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAiCG;AACH,wBAAgB,sBAAsB,CACpC,gBAAgB,EAAE,SAAS,cAAc,EAAE,GAC1C,aAAa,CAAC,aAAa,CAAC,CAY9B"}
@@ -0,0 +1,275 @@
1
+ import { canonicalizeIri, } from '@lde/pipeline';
2
+ import { DataFactory } from 'n3';
3
+ import { createHash } from 'node:crypto';
4
+ const { namedNode, literal, quad } = DataFactory;
5
+ const VOID = 'http://rdfs.org/ns/void#';
6
+ const VOID_EXT = 'http://ldf.fi/void-ext#';
7
+ const XSD_INTEGER = 'http://www.w3.org/2001/XMLSchema#integer';
8
+ const VOID_CLASS = `${VOID}class`;
9
+ const VOID_PROPERTY = `${VOID}property`;
10
+ const VOID_ENTITIES = `${VOID}entities`;
11
+ const VOID_TRIPLES = `${VOID}triples`;
12
+ const VOID_CLASS_PARTITION = `${VOID}classPartition`;
13
+ const VOID_PROPERTY_PARTITION = `${VOID}propertyPartition`;
14
+ const VOIDEXT_DATATYPE = `${VOID_EXT}datatype`;
15
+ const VOIDEXT_LANGUAGE = `${VOID_EXT}language`;
16
+ const VOIDEXT_DATATYPE_PARTITION = `${VOID_EXT}datatypePartition`;
17
+ const VOIDEXT_OBJECTCLASS_PARTITION = `${VOID_EXT}objectClassPartition`;
18
+ const VOIDEXT_LANGUAGE_PARTITION = `${VOID_EXT}languagePartition`;
19
+ // Predicates whose integer-literal objects are summed across merged partitions.
20
+ // `void:distinctObjects` is deliberately excluded: distinct-object sets overlap
21
+ // across namespace variants, so summing would over-count. It is normalized at
22
+ // query time in `class-properties-objects.rq` and never reaches this transform.
23
+ const NUMERIC_MEASURES = new Set([VOID_ENTITIES, VOID_TRIPLES]);
24
+ /** Structural links whose object is a child partition node. */
25
+ const CHILD_LINKS = new Set([
26
+ VOID_CLASS_PARTITION,
27
+ VOID_PROPERTY_PARTITION,
28
+ VOIDEXT_DATATYPE_PARTITION,
29
+ VOIDEXT_OBJECTCLASS_PARTITION,
30
+ VOIDEXT_LANGUAGE_PARTITION,
31
+ ]);
32
+ /** `void:class` / `void:property` objects are IRIs subject to canonicalization. */
33
+ const CANONICALIZED_OBJECT_PREDICATES = new Set([VOID_CLASS, VOID_PROPERTY]);
34
+ /**
35
+ * A {@link QuadTransform} that merges the `void:classPartition` /
36
+ * `void:propertyPartition` subtrees of namespace-alias variants (e.g.
37
+ * `http://schema.org/CreativeWork` and `https://schema.org/CreativeWork`) into
38
+ * a single partition per canonical class/property.
39
+ *
40
+ * VoID partitions are keyed by an opaque `MD5(class[, property[, …]])` IRI, so
41
+ * two namespace variants yield two partition nodes that both, after
42
+ * canonicalization, describe the same class. This transform re-mints every
43
+ * partition IRI from its **canonical** key components (replicating the queries’
44
+ * SPARQL `MD5(CONCAT(STR(…)))`), collapses the duplicates, and **sums** their
45
+ * numeric measures (`void:entities`, `void:triples`).
46
+ *
47
+ * ## Correctness assumptions
48
+ *
49
+ * Summing pre-aggregated counts is exact only under these assumptions; the
50
+ * queries whose measures cannot be safely summed keep their normalization at
51
+ * query time instead (notably `class-properties-objects.rq`’s
52
+ * `void:distinctObjects`, which this transform never sees):
53
+ *
54
+ * - **Subject/class disjointness** — no resource is typed under two namespace
55
+ * variants of the same class. Guards the `void:entities` sum on class
56
+ * partitions and every `void:triples` sum (a doubly-typed resource’s triples
57
+ * would otherwise count under both variants).
58
+ * - **Predicate-namespace disjointness** — no subject uses two namespace
59
+ * variants of the same property (e.g. both `http://schema.org/name` and
60
+ * `https://schema.org/name`). Guards the `void:entities` sum on property
61
+ * partitions.
62
+ *
63
+ * With no aliases configured the transform is a no-op.
64
+ *
65
+ * @see substituteNormalizationMarkers for the query-time normalization used
66
+ * where summing is not safe.
67
+ */
68
+ export function mergeNamespaceVariants(namespaceAliases) {
69
+ if (namespaceAliases.length === 0) {
70
+ return (quads) => quads;
71
+ }
72
+ return async function* (quads, { dataset }) {
73
+ const datasetIri = dataset.iri.toString();
74
+ const buffered = [];
75
+ for await (const q of quads) {
76
+ buffered.push(q);
77
+ }
78
+ yield* mergeBuffered(buffered, datasetIri, namespaceAliases);
79
+ };
80
+ }
81
+ function* mergeBuffered(buffered, datasetIri, namespaceAliases) {
82
+ const nodes = indexNodes(buffered);
83
+ const remap = buildRemap(nodes, datasetIri, namespaceAliases);
84
+ const measureSums = new Map();
85
+ const emitted = new Set();
86
+ const structural = [];
87
+ for (const original of buffered) {
88
+ const subject = remapTerm(original.subject, remap);
89
+ const object = canonicalizeObject(remapTerm(original.object, remap), original.predicate.value, namespaceAliases);
90
+ const rewritten = quad(subject, original.predicate, object, original.graph);
91
+ if (NUMERIC_MEASURES.has(original.predicate.value)) {
92
+ accumulateMeasure(measureSums, rewritten);
93
+ continue;
94
+ }
95
+ const key = quadKey(rewritten);
96
+ if (!emitted.has(key)) {
97
+ emitted.add(key);
98
+ structural.push(rewritten);
99
+ }
100
+ }
101
+ yield* structural;
102
+ for (const sum of measureSums.values()) {
103
+ yield reconstructMeasure(sum);
104
+ }
105
+ }
106
+ function indexNodes(buffered) {
107
+ const nodes = new Map();
108
+ const nodeOf = (value) => {
109
+ let node = nodes.get(value);
110
+ if (!node) {
111
+ node = { values: new Map() };
112
+ nodes.set(value, node);
113
+ }
114
+ return node;
115
+ };
116
+ for (const q of buffered) {
117
+ if (CHILD_LINKS.has(q.predicate.value) &&
118
+ q.object.termType === 'NamedNode') {
119
+ nodeOf(q.object.value).incomingLink = {
120
+ predicate: q.predicate.value,
121
+ parent: q.subject.value,
122
+ };
123
+ }
124
+ else {
125
+ nodeOf(q.subject.value).values.set(q.predicate.value, q.object);
126
+ }
127
+ }
128
+ return nodes;
129
+ }
130
+ /**
131
+ * Build a raw-IRI → canonical-IRI map for every partition node whose canonical
132
+ * key differs from its current IRI.
133
+ */
134
+ function buildRemap(nodes, datasetIri, namespaceAliases) {
135
+ const remap = new Map();
136
+ for (const [iri, node] of nodes) {
137
+ const canonicalIri = canonicalPartitionIri(node, nodes, datasetIri, namespaceAliases);
138
+ if (canonicalIri !== undefined && canonicalIri !== iri) {
139
+ remap.set(iri, canonicalIri);
140
+ }
141
+ }
142
+ return remap;
143
+ }
144
+ /**
145
+ * The canonical partition IRI for a node, or `undefined` if the node is not a
146
+ * partition (no incoming structural link). Replicates the queries’ minting:
147
+ * `<dataset>/.well-known/void#<prefix>-<MD5(STR(component)…)>`.
148
+ */
149
+ function canonicalPartitionIri(node, nodes, datasetIri, namespaceAliases) {
150
+ const link = node.incomingLink;
151
+ if (link === undefined)
152
+ return undefined;
153
+ const canon = (value) => canonicalizeIri(value, namespaceAliases);
154
+ const classOf = (partition) => nodes.get(partition)?.values.get(VOID_CLASS)?.value;
155
+ switch (link.predicate) {
156
+ case VOID_CLASS_PARTITION: {
157
+ const klass = node.values.get(VOID_CLASS)?.value;
158
+ return klass ? mint(datasetIri, 'class', [canon(klass)]) : undefined;
159
+ }
160
+ case VOID_PROPERTY_PARTITION: {
161
+ const klass = classOf(link.parent);
162
+ const property = node.values.get(VOID_PROPERTY)?.value;
163
+ return klass && property
164
+ ? mint(datasetIri, 'class-property', [canon(klass), canon(property)])
165
+ : undefined;
166
+ }
167
+ case VOIDEXT_DATATYPE_PARTITION: {
168
+ const [klass, property] = classProperty(link.parent, nodes);
169
+ const datatype = node.values.get(VOIDEXT_DATATYPE)?.value;
170
+ return klass && property && datatype
171
+ ? mint(datasetIri, 'datatype', [
172
+ canon(klass),
173
+ canon(property),
174
+ datatype,
175
+ ])
176
+ : undefined;
177
+ }
178
+ case VOIDEXT_OBJECTCLASS_PARTITION: {
179
+ const [klass, property] = classProperty(link.parent, nodes);
180
+ const objectClass = node.values.get(VOID_CLASS)?.value;
181
+ return klass && property && objectClass
182
+ ? mint(datasetIri, 'object-class', [
183
+ canon(klass),
184
+ canon(property),
185
+ canon(objectClass),
186
+ ])
187
+ : undefined;
188
+ }
189
+ case VOIDEXT_LANGUAGE_PARTITION: {
190
+ const [klass, property] = classProperty(link.parent, nodes);
191
+ const language = node.values.get(VOIDEXT_LANGUAGE)?.value;
192
+ return klass && property && language !== undefined
193
+ ? mint(datasetIri, 'language', [
194
+ canon(klass),
195
+ canon(property),
196
+ language,
197
+ ])
198
+ : undefined;
199
+ }
200
+ default:
201
+ return undefined;
202
+ }
203
+ }
204
+ /** The (class, property) of a property partition, following it up to its class partition. */
205
+ function classProperty(propertyPartition, nodes) {
206
+ const node = nodes.get(propertyPartition);
207
+ const property = node?.values.get(VOID_PROPERTY)?.value;
208
+ const classPartition = node?.incomingLink?.parent;
209
+ const klass = classPartition
210
+ ? nodes.get(classPartition)?.values.get(VOID_CLASS)?.value
211
+ : undefined;
212
+ return [klass, property];
213
+ }
214
+ function mint(datasetIri, prefix, components) {
215
+ const hash = createHash('md5').update(components.join('')).digest('hex');
216
+ return `${datasetIri}/.well-known/void#${prefix}-${hash}`;
217
+ }
218
+ function remapTerm(term, remap) {
219
+ if (term.termType === 'NamedNode') {
220
+ const canonical = remap.get(term.value);
221
+ if (canonical !== undefined)
222
+ return namedNode(canonical);
223
+ }
224
+ return term;
225
+ }
226
+ function canonicalizeObject(object, predicate, namespaceAliases) {
227
+ if (CANONICALIZED_OBJECT_PREDICATES.has(predicate) &&
228
+ object.termType === 'NamedNode') {
229
+ const canonical = canonicalizeIri(object.value, namespaceAliases);
230
+ if (canonical !== object.value)
231
+ return namedNode(canonical);
232
+ }
233
+ return object;
234
+ }
235
+ function accumulateMeasure(sums, measure) {
236
+ const key = `${measure.subject.value} ${measure.predicate.value} ${measure.graph.value}`;
237
+ const existing = sums.get(key);
238
+ if (existing === undefined) {
239
+ sums.set(key, {
240
+ subject: measure.subject,
241
+ predicate: measure.predicate,
242
+ graph: measure.graph,
243
+ datatype: measure.object.termType === 'Literal'
244
+ ? measure.object.datatype.value
245
+ : XSD_INTEGER,
246
+ total: parseInteger(measure.object),
247
+ });
248
+ }
249
+ else {
250
+ existing.total += parseInteger(measure.object);
251
+ }
252
+ }
253
+ function parseInteger(object) {
254
+ if (object.termType === 'Literal' && /^[+-]?\d+$/.test(object.value)) {
255
+ return BigInt(object.value);
256
+ }
257
+ return 0n;
258
+ }
259
+ function reconstructMeasure(sum) {
260
+ return quad(sum.subject, sum.predicate, literal(sum.total.toString(), namedNode(sum.datatype)), sum.graph);
261
+ }
262
+ function quadKey(q) {
263
+ return [
264
+ q.subject.value,
265
+ q.predicate.value,
266
+ termKey(q.object),
267
+ q.graph.value,
268
+ ].join(' ');
269
+ }
270
+ function termKey(term) {
271
+ if (term.termType === 'Literal') {
272
+ return `"${term.value}"^^${term.datatype.value}@${term.language}`;
273
+ }
274
+ return term.value;
275
+ }
package/dist/stage.d.ts CHANGED
@@ -1,4 +1,4 @@
1
- import { Stage, type ReaderContext, type QuadTransform } from '@lde/pipeline';
1
+ import { Stage, type ReaderContext, type NamespaceAlias, type QuadTransform } from '@lde/pipeline';
2
2
  import type { Quad } from '@rdfjs/types';
3
3
  /**
4
4
  * Stable names for every VoID stage, equal to the underlying query filename.
@@ -46,6 +46,27 @@ export interface VoidStageOptions {
46
46
  * with these, built-in first.
47
47
  */
48
48
  transform?: VoidStageTransform;
49
+ /**
50
+ * Namespace pairs to treat as equivalent when partitioning by class and
51
+ * property, so alias variants (e.g. `http://schema.org/CreativeWork` and
52
+ * `https://schema.org/CreativeWork`) merge into a single partition instead
53
+ * of two that both claim the canonical `void:class`.
54
+ *
55
+ * Merging happens after aggregation, in the {@link mergeNamespaceVariants}
56
+ * transform, which sums `void:entities` / `void:triples`; only
57
+ * `class-properties-objects.rq` normalizes at query time (its
58
+ * `void:distinctObjects` cannot be recovered by summing). Correct under the
59
+ * subject/class- and predicate-namespace-disjointness assumptions documented
60
+ * on {@link mergeNamespaceVariants}. The top-level property partitions
61
+ * ({@link detectVocabularies}’s `entity-properties.rq`) and `void:vocabulary`
62
+ * detection keep the source namespaces, so consumers can still see which
63
+ * namespace the dataset actually uses.
64
+ *
65
+ * Defaults to no aliases. To cover schema.org datasets that publish under
66
+ * both `http://schema.org/` and `https://schema.org/`, pass
67
+ * `[{ canonical: 'https://schema.org/', alias: 'http://schema.org/' }]`.
68
+ */
69
+ namespaceAliases?: readonly NamespaceAlias[];
49
70
  }
50
71
  /**
51
72
  * Options for per-class VoID stages that iterate over classes.
@@ -1 +1 @@
1
- {"version":3,"file":"stage.d.ts","sourceRoot":"","sources":["../src/stage.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,KAAK,EAKL,KAAK,aAAa,EAElB,KAAK,aAAa,EACnB,MAAM,eAAe,CAAC;AAEvB,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAC;AAezC;;;;;;GAMG;AACH,eAAO,MAAM,gBAAgB;;;;;;;;;;;;;;;;;CAiBnB,CAAC;AAEX,sDAAsD;AACtD,MAAM,MAAM,aAAa,GACvB,CAAC,OAAO,gBAAgB,CAAC,CAAC,MAAM,OAAO,gBAAgB,CAAC,CAAC;AAE3D,2EAA2E;AAC3E,MAAM,MAAM,kBAAkB,GAC1B,aAAa,CAAC,aAAa,CAAC,GAC5B,aAAa,CAAC,aAAa,CAAC,EAAE,CAAC;AAEnC;;;;;;;GAOG;AACH,MAAM,WAAW,gBAAgB;IAC/B;;;;;;OAMG;IACH,SAAS,CAAC,EAAE,kBAAkB,CAAC;CAChC;AAED;;;;;GAKG;AACH,MAAM,WAAW,wBAAyB,SAAQ,gBAAgB;IAChE,oEAAoE;IACpE,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,+DAA+D;IAC/D,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,iFAAiF;IACjF,QAAQ,CAAC,EAAE,OAAO,CAAC;CACpB;AAED;;;;;;GAMG;AACH,MAAM,WAAW,iBAAkB,SAAQ,IAAI,CAC7C,wBAAwB,EACxB,WAAW,CACZ;IACC,yEAAyE;IACzE,SAAS,CAAC,EAAE,WAAW,CAAC,MAAM,EAAE,SAAS,IAAI,EAAE,CAAC,CAAC;IACjD,mFAAmF;IACnF,YAAY,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;IACjC;;;;;;;OAOG;IACH,UAAU,CAAC,EAAE,OAAO,CAAC,MAAM,CAAC,aAAa,EAAE,kBAAkB,CAAC,CAAC,CAAC;CACjE;AAgED,wBAAgB,gBAAgB,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAE3E;AAED,wBAAgB,eAAe,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAE1E;AAOD,wBAAgB,mBAAmB,CACjC,OAAO,CAAC,EAAE,gBAAgB,GACzB,OAAO,CAAC,KAAK,CAAC,CAKhB;AAED,wBAAgB,eAAe,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAK1E;AAED,wBAAgB,eAAe,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAK1E;AAED,wBAAgB,aAAa,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAKxE;AAED,wBAAgB,YAAY,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAKvE;AAED,wBAAgB,qBAAqB,CACnC,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAKhB;AAED,wBAAgB,oBAAoB,CAClC,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAKhB;AAED,wBAAgB,cAAc,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAKzE;AAED,wBAAgB,cAAc,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAEzE;AAID,wBAAgB,qBAAqB,CACnC,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAKhB;AAED,wBAAgB,iBAAiB,CAC/B,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAKhB;AAED,wBAAgB,iBAAiB,CAC/B,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAKhB;AAID,wBAAgB,SAAS,CACvB,WAAW,EAAE,WAAW,CAAC,MAAM,EAAE,SAAS,IAAI,EAAE,CAAC,EACjD,OAAO,CAAC,EAAE,gBAAgB,GACzB,OAAO,CAAC,KAAK,CAAC,CAOhB;AAED,MAAM,WAAW,yBAA0B,SAAQ,gBAAgB;IACjE,mFAAmF;IACnF,YAAY,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;CAClC;AAED,wBAAgB,kBAAkB,CAChC,OAAO,CAAC,EAAE,yBAAyB,GAClC,OAAO,CAAC,KAAK,CAAC,CAQhB;AAED;;;;;;;GAOG;AACH,wBAAsB,UAAU,CAC9B,OAAO,CAAC,EAAE,iBAAiB,GAC1B,OAAO,CAAC,KAAK,EAAE,CAAC,CAgDlB"}
1
+ {"version":3,"file":"stage.d.ts","sourceRoot":"","sources":["../src/stage.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,KAAK,EAKL,KAAK,aAAa,EAElB,KAAK,cAAc,EACnB,KAAK,aAAa,EACnB,MAAM,eAAe,CAAC;AAEvB,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAC;AAqBzC;;;;;;GAMG;AACH,eAAO,MAAM,gBAAgB;;;;;;;;;;;;;;;;;CAiBnB,CAAC;AAEX,sDAAsD;AACtD,MAAM,MAAM,aAAa,GACvB,CAAC,OAAO,gBAAgB,CAAC,CAAC,MAAM,OAAO,gBAAgB,CAAC,CAAC;AAE3D,2EAA2E;AAC3E,MAAM,MAAM,kBAAkB,GAC1B,aAAa,CAAC,aAAa,CAAC,GAC5B,aAAa,CAAC,aAAa,CAAC,EAAE,CAAC;AAEnC;;;;;;;GAOG;AACH,MAAM,WAAW,gBAAgB;IAC/B;;;;;;OAMG;IACH,SAAS,CAAC,EAAE,kBAAkB,CAAC;IAC/B;;;;;;;;;;;;;;;;;;;OAmBG;IACH,gBAAgB,CAAC,EAAE,SAAS,cAAc,EAAE,CAAC;CAC9C;AAED;;;;;GAKG;AACH,MAAM,WAAW,wBAAyB,SAAQ,gBAAgB;IAChE,oEAAoE;IACpE,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,+DAA+D;IAC/D,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,iFAAiF;IACjF,QAAQ,CAAC,EAAE,OAAO,CAAC;CACpB;AAED;;;;;;GAMG;AACH,MAAM,WAAW,iBAAkB,SAAQ,IAAI,CAC7C,wBAAwB,EACxB,WAAW,CACZ;IACC,yEAAyE;IACzE,SAAS,CAAC,EAAE,WAAW,CAAC,MAAM,EAAE,SAAS,IAAI,EAAE,CAAC,CAAC;IACjD,mFAAmF;IACnF,YAAY,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;IACjC;;;;;;;OAOG;IACH,UAAU,CAAC,EAAE,OAAO,CAAC,MAAM,CAAC,aAAa,EAAE,kBAAkB,CAAC,CAAC,CAAC;CACjE;AAwFD,wBAAgB,gBAAgB,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAE3E;AAED,wBAAgB,eAAe,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAK1E;AAOD,wBAAgB,mBAAmB,CACjC,OAAO,CAAC,EAAE,gBAAgB,GACzB,OAAO,CAAC,KAAK,CAAC,CAKhB;AAED,wBAAgB,eAAe,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAK1E;AAED,wBAAgB,eAAe,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAK1E;AAED,wBAAgB,aAAa,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAKxE;AAED,wBAAgB,YAAY,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAKvE;AAED,wBAAgB,qBAAqB,CACnC,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAMhB;AAED,wBAAgB,oBAAoB,CAClC,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAKhB;AAED,wBAAgB,cAAc,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAKzE;AAED,wBAAgB,cAAc,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,OAAO,CAAC,KAAK,CAAC,CAEzE;AAID,wBAAgB,qBAAqB,CACnC,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAMhB;AAED,wBAAgB,iBAAiB,CAC/B,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAMhB;AAED,wBAAgB,iBAAiB,CAC/B,OAAO,CAAC,EAAE,wBAAwB,GACjC,OAAO,CAAC,KAAK,CAAC,CAMhB;AAID,wBAAgB,SAAS,CACvB,WAAW,EAAE,WAAW,CAAC,MAAM,EAAE,SAAS,IAAI,EAAE,CAAC,EACjD,OAAO,CAAC,EAAE,gBAAgB,GACzB,OAAO,CAAC,KAAK,CAAC,CAOhB;AAED,MAAM,WAAW,yBAA0B,SAAQ,gBAAgB;IACjE,mFAAmF;IACnF,YAAY,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;CAClC;AAED,wBAAgB,kBAAkB,CAChC,OAAO,CAAC,EAAE,yBAAyB,GAClC,OAAO,CAAC,KAAK,CAAC,CAQhB;AAED;;;;;;;GAOG;AACH,wBAAsB,UAAU,CAC9B,OAAO,CAAC,EAAE,iBAAiB,GAC1B,OAAO,CAAC,KAAK,EAAE,CAAC,CAgDlB"}
package/dist/stage.js CHANGED
@@ -4,6 +4,8 @@ import { resolve, dirname } from 'node:path';
4
4
  import { fileURLToPath } from 'node:url';
5
5
  import { withVocabularies, defaultVocabularies, } from './vocabularyTransform.js';
6
6
  import { withUriSpaces } from './uriSpaceTransform.js';
7
+ import { canonicalizeClassBindings, substituteNormalizationMarkers, withAliasVariantBindings, } from './namespaceAliases.js';
8
+ import { mergeNamespaceVariants } from './partitionMerge.js';
7
9
  const queriesDir = resolve(dirname(fileURLToPath(import.meta.url)), '..', 'queries');
8
10
  /**
9
11
  * Stable names for every VoID stage, equal to the underlying query filename.
@@ -31,15 +33,30 @@ export const VOID_STAGE_NAMES = {
31
33
  objectUriSpace: 'object-uri-space.rq',
32
34
  };
33
35
  async function createVoidStage(filename, options) {
34
- const query = await readQueryFile(resolve(queriesDir, filename));
36
+ const namespaceAliases = options?.namespaceAliases ?? [];
37
+ const query = substituteNormalizationMarkers(await readQueryFile(resolve(queriesDir, filename)), namespaceAliases);
38
+ const constructReader = new SparqlConstructReader({ query });
39
+ // Partition stages merge their namespace-alias variants (e.g. http:// and
40
+ // https://schema.org/) into one partition per canonical class/property.
41
+ // Runs first, so any consumer transform sees the already-merged output.
42
+ const mergeTransform = options?.mergePartitions && namespaceAliases.length > 0
43
+ ? [mergeNamespaceVariants(namespaceAliases)]
44
+ : [];
35
45
  const reader = {
36
- reader: new SparqlConstructReader({ query }),
37
- transform: options?.transform,
46
+ // Per-class stages receive canonical `?class` bindings from the selector;
47
+ // expand each into one `?class` binding per alias variant so the query
48
+ // (`?s a ?class`) matches instances under either namespace.
49
+ reader: options?.perClass
50
+ ? withAliasVariantBindings(constructReader, namespaceAliases)
51
+ : constructReader,
52
+ transform: [...mergeTransform, ...asTransforms(options?.transform)],
38
53
  };
39
54
  return new Stage({
40
55
  name: filename,
41
56
  readers: reader,
42
- itemSelector: options?.perClass ? classSelector() : undefined,
57
+ itemSelector: options?.perClass
58
+ ? classSelector(namespaceAliases)
59
+ : undefined,
43
60
  batchSize: options?.batchSize,
44
61
  maxConcurrency: options?.maxConcurrency,
45
62
  expectsOutput: options?.expectsOutput,
@@ -51,7 +68,7 @@ function asTransforms(transform) {
51
68
  return [];
52
69
  return Array.isArray(transform) ? [...transform] : [transform];
53
70
  }
54
- function classSelector() {
71
+ function classSelector(namespaceAliases) {
55
72
  return {
56
73
  // Forward `options` so the Pipeline’s per-dataset TimeoutPolicy
57
74
  // reaches the inner SparqlItemSelector — without this the adaptive
@@ -69,9 +86,10 @@ function classSelector() {
69
86
  `WHERE { ${subjectFilter} ?s a ?class . }`,
70
87
  'LIMIT 1000',
71
88
  ].join('\n');
72
- return new SparqlItemSelector({
89
+ const rows = new SparqlItemSelector({
73
90
  query: selectorQuery,
74
91
  }).select(distribution, batchSize, options);
92
+ return canonicalizeClassBindings(rows, namespaceAliases);
75
93
  },
76
94
  };
77
95
  }
@@ -80,7 +98,10 @@ export function subjectUriSpaces(options) {
80
98
  return createVoidStage(VOID_STAGE_NAMES.subjectUriSpace, options);
81
99
  }
82
100
  export function classPartitions(options) {
83
- return createVoidStage(VOID_STAGE_NAMES.classPartitions, options);
101
+ return createVoidStage(VOID_STAGE_NAMES.classPartitions, {
102
+ ...options,
103
+ mergePartitions: true,
104
+ });
84
105
  }
85
106
  // Scalar-aggregate counts: each query is a single COUNT with no GROUP BY/HAVING,
86
107
  // so it always returns exactly one row. Zero output therefore means the endpoint
@@ -120,6 +141,7 @@ export function classPropertySubjects(options) {
120
141
  return createVoidStage(VOID_STAGE_NAMES.classPropertySubjects, {
121
142
  ...options,
122
143
  perClass: options?.perClass ?? true,
144
+ mergePartitions: true,
123
145
  });
124
146
  }
125
147
  export function classPropertyObjects(options) {
@@ -142,18 +164,21 @@ export function perClassObjectClasses(options) {
142
164
  return createVoidStage(VOID_STAGE_NAMES.perClassObjectClasses, {
143
165
  ...options,
144
166
  perClass: options?.perClass ?? true,
167
+ mergePartitions: true,
145
168
  });
146
169
  }
147
170
  export function perClassDatatypes(options) {
148
171
  return createVoidStage(VOID_STAGE_NAMES.perClassDatatypes, {
149
172
  ...options,
150
173
  perClass: options?.perClass ?? true,
174
+ mergePartitions: true,
151
175
  });
152
176
  }
153
177
  export function perClassLanguages(options) {
154
178
  return createVoidStage(VOID_STAGE_NAMES.perClassLanguages, {
155
179
  ...options,
156
180
  perClass: options?.perClass ?? true,
181
+ mergePartitions: true,
157
182
  });
158
183
  }
159
184
  // Stages with a built-in transform
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@lde/pipeline-void",
3
- "version": "0.31.3",
3
+ "version": "0.31.4",
4
4
  "description": "VOiD (Vocabulary of Interlinked Datasets) statistical analysis for RDF datasets",
5
5
  "repository": {
6
6
  "url": "git+https://github.com/ldelements/lde.git",
@@ -34,6 +34,6 @@
34
34
  },
35
35
  "peerDependencies": {
36
36
  "@lde/dataset": "^0.7.8",
37
- "@lde/pipeline": "^0.33.3"
37
+ "@lde/pipeline": "^0.33.4"
38
38
  }
39
39
  }
@@ -5,17 +5,27 @@ CONSTRUCT {
5
5
  }
6
6
  WHERE {
7
7
  # Object counts only. Subject counts in class-properties-subjects.rq.
8
+ #
9
+ # Unlike the other per-class queries, this one normalizes the class and
10
+ # property to their canonical namespace *before* the COUNT(DISTINCT ?o):
11
+ # merging distinct-object counts by summing would over-count objects shared
12
+ # between namespace variants (e.g. the same name under http:name and
13
+ # https:name), so the dedup must happen here rather than in the
14
+ # partition-merge transform. `?class` is bound per namespace variant by the
15
+ # injected VALUES clause; `?property` is discovered.
8
16
  {
9
- SELECT ?class ?p (COUNT(DISTINCT ?o) AS ?objects) {
17
+ SELECT ?canonicalClass ?canonicalProperty (COUNT(DISTINCT ?o) AS ?objects) {
10
18
  {
11
- SELECT ?class ?p ?o {
19
+ SELECT ?canonicalClass ?canonicalProperty ?o {
12
20
  #subjectFilter#
13
- ?s a ?class ; ?p ?o .
21
+ ?s a ?class ; ?property ?o .
22
+ BIND(#normalized:class# AS ?canonicalClass)
23
+ BIND(#normalized:property# AS ?canonicalProperty)
14
24
  }
15
25
  }
16
26
  }
17
- GROUP BY ?class ?p
27
+ GROUP BY ?canonicalClass ?canonicalProperty
18
28
  }
19
- BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-property-", MD5(CONCAT(STR(?class), STR(?p))))) AS ?propertyPartition)
29
+ BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-property-", MD5(CONCAT(STR(?canonicalClass), STR(?canonicalProperty))))) AS ?propertyPartition)
20
30
  }
21
31
  LIMIT 100000
@@ -2,7 +2,17 @@ PREFIX void: <http://rdfs.org/ns/void#>
2
2
  PREFIX void-ext: <http://ldf.fi/void-ext#>
3
3
 
4
4
  CONSTRUCT {
5
- ?propertyPartition void-ext:datatypePartition ?datatypePartition .
5
+ # Self-describing parent chain so the namespace-merge transform can re-key
6
+ # this partition from its (class, property) components. The
7
+ # `?dataset void:classPartition ?classPartition` link is what gives the class
8
+ # partition an incoming edge, so the transform re-keys it too (without it, the
9
+ # alias-variant class partition leaks unmerged).
10
+ ?dataset a void:Dataset ;
11
+ void:classPartition ?classPartition .
12
+ ?classPartition void:class ?class ;
13
+ void:propertyPartition ?propertyPartition .
14
+ ?propertyPartition void:property ?p ;
15
+ void-ext:datatypePartition ?datatypePartition .
6
16
  ?datatypePartition
7
17
  void-ext:datatype ?dt ;
8
18
  void:triples ?count .
@@ -22,6 +32,7 @@ WHERE {
22
32
  }
23
33
  GROUP BY ?class ?p ?dt
24
34
  }
35
+ BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-", MD5(STR(?class)))) AS ?classPartition)
25
36
  BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-property-", MD5(CONCAT(STR(?class), STR(?p))))) AS ?propertyPartition)
26
37
  BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#datatype-", MD5(CONCAT(STR(?class), STR(?p), STR(?dt))))) AS ?datatypePartition)
27
38
  }
@@ -2,7 +2,17 @@ PREFIX void: <http://rdfs.org/ns/void#>
2
2
  PREFIX void-ext: <http://ldf.fi/void-ext#>
3
3
 
4
4
  CONSTRUCT {
5
- ?propertyPartition void-ext:languagePartition ?languagePartition .
5
+ # Self-describing parent chain so the namespace-merge transform can re-key
6
+ # this partition from its (class, property) components. The
7
+ # `?dataset void:classPartition ?classPartition` link is what gives the class
8
+ # partition an incoming edge, so the transform re-keys it too (without it, the
9
+ # alias-variant class partition leaks unmerged).
10
+ ?dataset a void:Dataset ;
11
+ void:classPartition ?classPartition .
12
+ ?classPartition void:class ?class ;
13
+ void:propertyPartition ?propertyPartition .
14
+ ?propertyPartition void:property ?p ;
15
+ void-ext:languagePartition ?languagePartition .
6
16
  ?languagePartition
7
17
  void-ext:language ?lang ;
8
18
  void:triples ?count .
@@ -24,6 +34,7 @@ WHERE {
24
34
  }
25
35
  GROUP BY ?class ?p ?lang
26
36
  }
37
+ BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-", MD5(STR(?class)))) AS ?classPartition)
27
38
  BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-property-", MD5(CONCAT(STR(?class), STR(?p))))) AS ?propertyPartition)
28
39
  BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#language-", MD5(CONCAT(STR(?class), STR(?p), ?lang)))) AS ?languagePartition)
29
40
  }
@@ -2,7 +2,17 @@ PREFIX void: <http://rdfs.org/ns/void#>
2
2
  PREFIX void-ext: <http://ldf.fi/void-ext#>
3
3
 
4
4
  CONSTRUCT {
5
- ?propertyPartition void-ext:objectClassPartition ?objectClassPartition .
5
+ # Self-describing parent chain so the namespace-merge transform can re-key
6
+ # this partition from its (class, property) components. The
7
+ # `?dataset void:classPartition ?classPartition` link is what gives the class
8
+ # partition an incoming edge, so the transform re-keys it too (without it, the
9
+ # alias-variant class partition leaks unmerged).
10
+ ?dataset a void:Dataset ;
11
+ void:classPartition ?classPartition .
12
+ ?classPartition void:class ?class ;
13
+ void:propertyPartition ?propertyPartition .
14
+ ?propertyPartition void:property ?p ;
15
+ void-ext:objectClassPartition ?objectClassPartition .
6
16
  ?objectClassPartition
7
17
  void:class ?objectClass ;
8
18
  void:triples ?count .
@@ -22,6 +32,7 @@ WHERE {
22
32
  }
23
33
  GROUP BY ?class ?p ?objectClass
24
34
  }
35
+ BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-", MD5(STR(?class)))) AS ?classPartition)
25
36
  BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#class-property-", MD5(CONCAT(STR(?class), STR(?p))))) AS ?propertyPartition)
26
37
  BIND(URI(CONCAT(STR(?dataset), "/.well-known/void#object-class-", MD5(CONCAT(STR(?class), STR(?p), STR(?objectClass))))) AS ?objectClassPartition)
27
38
  }