keepset 0.29.1 → 0.29.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/dist/{bundled-Ql23qj97.js → bundled-B_XXp-TS.js} +5 -5
- package/dist/{categories-DC0uTkzn.d.ts → categories-IrB8M6Ne.d.ts} +10 -10
- package/dist/index.d.ts +3 -3
- package/dist/index.js +3 -3
- package/dist/node/fetch-dataset.d.ts +3 -3
- package/dist/node/fetch-dataset.js +4 -4
- package/dist-cli/keepset.mjs +2 -2
- package/package.json +2 -2
- package/schemas/entry.schema.json +1 -1
- package/schemas/manifest.schema.json +1 -1
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://github.com/Mearman/keepset) [](https://www.npmjs.com/package/keepset) [](https://github.com/Mearman/keepset/releases/latest) [](https://github.com/Mearman/keepset/actions)
|
|
4
4
|
|
|
5
|
-
A catalogue of datasets, each with a machine-readable recipe that says how to fetch it and how to normalise it into the shape its category names. The `keepset` package is the consumable library over that catalogue; the CLI and the statically generated site are thin surfaces on top of it. Entries are expressed with [
|
|
5
|
+
A catalogue of datasets, each with a machine-readable recipe that says how to fetch it and how to normalise it into the shape its category names. The `keepset` package is the consumable library over that catalogue; the CLI and the statically generated site are thin surfaces on top of it. Entries are expressed with [crossmap](https://github.com/ExaDev/crossmap) crosswalks, so fetching a dataset yields a normalised shape directly in user code.
|
|
6
6
|
|
|
7
7
|
## Usage
|
|
8
8
|
|
|
@@ -36,7 +36,7 @@ npx keepset validate cora
|
|
|
36
36
|
npx keepset roundtrip cora
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
-
`namespaces [path]` lists the namespaces directly beneath a path (the outermost ones without one) with how many datasets each holds, and the site has a page for each, linked from every entry's namespace. `fetch` writes the normalised result with its provenance beside it; `validate` checks the entry against the entry schema and its crosswalk against
|
|
39
|
+
`namespaces [path]` lists the namespaces directly beneath a path (the outermost ones without one) with how many datasets each holds, and the site has a page for each, linked from every entry's namespace. `fetch` writes the normalised result with its provenance beside it; `validate` checks the entry against the entry schema and its crosswalk against crossmap's validation without the network; `roundtrip` fetches the dataset and checks that decoding the encoded native document gives it back. Exit status is 0 for an answer, 1 for a negative one (an invalid entry, a round trip that does not hold) and 2 for no answer (bad usage, an unknown or ambiguous name or namespace, a recipe that failed).
|
|
40
40
|
|
|
41
41
|
## Names, aliases and namespaces
|
|
42
42
|
|
|
@@ -57,7 +57,7 @@ An alias that is a path, such as `foo/graphs/bar`, files the entry in a second g
|
|
|
57
57
|
|
|
58
58
|
## The entry format
|
|
59
59
|
|
|
60
|
-
An entry is a JSON document in `catalogue/entries/`: a descriptor (stable id, name, description, domain, subjects, tags), the category whose normalisation target the entry resolves to, its sources (each with a retrieval date, and for a dead source a pinned Wayback snapshot that travels with the URL because the snapshot timestamp is part of the recipe's reproducibility), its licence and citations, the recipe that turns downloaded bytes into the dataset's native JSON shape, and a
|
|
60
|
+
An entry is a JSON document in `catalogue/entries/`: a descriptor (stable id, name, description, domain, subjects, tags), the category whose normalisation target the entry resolves to, its sources (each with a retrieval date, and for a dead source a pinned Wayback snapshot that travels with the URL because the snapshot timestamp is part of the recipe's reproducibility), its licence and citations, the recipe that turns downloaded bytes into the dataset's native JSON shape, and a crossmap crosswalk mapping that native shape onto the category target. The format is defined by Zod schemas, and the generated JSON Schema is published with the package as `keepset/schemas/entry.schema.json`, with the catalogue manifest's beside it. The domain is the entry's topic, its tags are the node fields it carries, its namespace is the source it comes from, and its subjects are the fields of study it is relevant to (for example `operations-research`), as slugs a search filters by (`keepset search --subject operations-research`), a listing page per subject on the site and keywords in the JSON-LD.
|
|
61
61
|
|
|
62
62
|
The catalogue itself is a manifest (`catalogue/manifest.json`: id, version, repository) plus the entries and their crosswalks. Contribution is by forking with lineage: a fork is its own catalogue, its manifest names the catalogue and version it was forked from, and entry ids stay stable across forks so two catalogues can be compared entry by entry.
|
|
63
63
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { CrosswalkSchema, validateCrosswalk } from "
|
|
1
|
+
import { CrosswalkSchema, validateCrosswalk } from "crossmap";
|
|
2
2
|
import { z } from "zod";
|
|
3
3
|
//#region src/errors.ts
|
|
4
4
|
/** The root of this package's error taxonomy: every error keepset throws is a `KeepsetError`, so callers can catch the family with one check. */
|
|
@@ -235,7 +235,7 @@ function maybe(schema) {
|
|
|
235
235
|
return schema.nullable().optional();
|
|
236
236
|
}
|
|
237
237
|
/**
|
|
238
|
-
* A task: a place or case to visit or serve, such as a stop of a route or a case in a register, flat so a
|
|
238
|
+
* A task: a place or case to visit or serve, such as a stop of a route or a case in a register, flat so a crossmap crosswalk can map a table's columns onto it (a wildcard path must keep the same shape on both sides). The target fixes only what the datasets share and lets a crosswalk map any further column through under its own name, as graph/v1 does. `lat` and `lon` are WGS84 degrees and `x` and `y` are planar coordinates in the dataset's own system; `duration` and `travelTime` are seconds, `prize` is the dataset's own reward measure, and `prizeProbability` is the chance the reward is collected. `skill` and `territory` name what is needed to serve the node and where it sits, `status` is where it is in its lifecycle, and `opened`, `closed` and `deadline` are its lifecycle times as the source writes them, a deadline being a number where the source gives a time of day.
|
|
239
239
|
*/
|
|
240
240
|
const TaskSchema = z.looseObject({
|
|
241
241
|
id: z.string().min(1),
|
|
@@ -535,7 +535,7 @@ const ArchiveSchema = z.union([
|
|
|
535
535
|
const ColumnSchema = z.int().min(0);
|
|
536
536
|
/** The text encoding of the unpacked file. Decoding is strict: bytes that are not valid in the stated encoding fail the recipe rather than turning into replacement characters. */
|
|
537
537
|
const EncodingSchema = z.enum(["utf-8", "latin1"]);
|
|
538
|
-
/** A column of a table as the table parser names it: the header with every run of characters outside letters, digits and underscore replaced by one underscore, so it is a valid segment of a
|
|
538
|
+
/** A column of a table as the table parser names it: the header with every run of characters outside letters, digits and underscore replaced by one underscore, so it is a valid segment of a crossmap path. */
|
|
539
539
|
const ColumnNameSchema = z.string().regex(/^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$/, { message: "must be a column name as the parser spells it: letters, digits and underscores, starting with a letter and not ending with an underscore" });
|
|
540
540
|
/** The parser that reads the unpacked file into the dataset's native JSON shape, with the options that format needs. Every option is stated in the entry, so a recipe reads the same way wherever it runs. */
|
|
541
541
|
const ParserSchema = z.discriminatedUnion("format", [
|
|
@@ -675,7 +675,7 @@ const RecipeSchema = z.discriminatedUnion("status", [z.object({
|
|
|
675
675
|
status: z.literal("not-fetchable"),
|
|
676
676
|
reason: z.string().min(1)
|
|
677
677
|
}).strict()]);
|
|
678
|
-
/** The
|
|
678
|
+
/** The crossmap crosswalk from the native shape to the category target: the document inline, or a `.json` file path relative to the catalogue root, so entries that share a native shape share one crosswalk. */
|
|
679
679
|
const CrosswalkReferenceSchema = z.union([CrosswalkSchema, z.object({ file: RelativePathSchema.refine((path) => path.endsWith(".json"), { message: "must name a .json file" }) }).strict()]);
|
|
680
680
|
/** The meta keys a fetched result takes from the entry's own descriptor, which its `properties` therefore may not also set. */
|
|
681
681
|
const RESERVED_META_KEYS = ["id", "name"];
|
|
@@ -819,7 +819,7 @@ function parseManifest(doc) {
|
|
|
819
819
|
//#endregion
|
|
820
820
|
//#region src/catalogue.ts
|
|
821
821
|
/**
|
|
822
|
-
* A catalogue index: the manifest, every entry, and every crosswalk file of the catalogue, keyed by its catalogue-relative path. The build assembles it from a catalogue directory; a consumer holding its own catalogue (a fork, say) can pass one to `createCatalogue`. Entry ids are unique, every fully qualified name (the whole path of an id or alias) identifies one entry and is not another entry's id, every namespace segment the manifest gives aliases to is some entry's segment and no alias is itself one, every crosswalk passes
|
|
822
|
+
* A catalogue index: the manifest, every entry, and every crosswalk file of the catalogue, keyed by its catalogue-relative path. The build assembles it from a catalogue directory; a consumer holding its own catalogue (a fork, say) can pass one to `createCatalogue`. Entry ids are unique, every fully qualified name (the whole path of an id or alias) identifies one entry and is not another entry's id, every namespace segment the manifest gives aliases to is some entry's segment and no alias is itself one, every crosswalk passes crossmap's structural validation, every crosswalk file reference resolves, and every crosswalk's target is the category of each entry that uses it.
|
|
823
823
|
*/
|
|
824
824
|
const CatalogueIndexSchema = define(z.object({
|
|
825
825
|
manifest: ManifestSchema,
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { Crosswalk } from "
|
|
1
|
+
import { Crosswalk } from "crossmap";
|
|
2
2
|
import { z } from "zod";
|
|
3
3
|
//#region src/entry.d.ts
|
|
4
4
|
/** A stable identifier: an entry id, a catalogue id. It starts with a letter or digit, so no id can be `__proto__` or any other name that begins with an underscore. Entry ids stay the same across forks, which is what lets two catalogues be compared entry by entry. */
|
|
@@ -131,7 +131,7 @@ type DocumentArchive = Exclude<Archive, {
|
|
|
131
131
|
}>;
|
|
132
132
|
/** An archive that gives every file a collection is read from. */
|
|
133
133
|
type CollectionArchive = z.infer<typeof CollectionArchiveSchema>;
|
|
134
|
-
/** A column of a table as the table parser names it: the header with every run of characters outside letters, digits and underscore replaced by one underscore, so it is a valid segment of a
|
|
134
|
+
/** A column of a table as the table parser names it: the header with every run of characters outside letters, digits and underscore replaced by one underscore, so it is a valid segment of a crossmap path. */
|
|
135
135
|
declare const ColumnNameSchema: z.ZodString;
|
|
136
136
|
/** The parser that reads the unpacked file into the dataset's native JSON shape, with the options that format needs. Every option is stated in the entry, so a recipe reads the same way wherever it runs. */
|
|
137
137
|
declare const ParserSchema: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
@@ -393,9 +393,9 @@ declare const RecipeSchema: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
|
393
393
|
reason: z.ZodString;
|
|
394
394
|
}, z.core.$strict>], "status">;
|
|
395
395
|
type Recipe = z.infer<typeof RecipeSchema>;
|
|
396
|
-
/** The
|
|
397
|
-
declare const CrosswalkReferenceSchema: z.ZodUnion<readonly [z.ZodType<import("
|
|
398
|
-
is: (value: unknown) => value is import("
|
|
396
|
+
/** The crossmap crosswalk from the native shape to the category target: the document inline, or a `.json` file path relative to the catalogue root, so entries that share a native shape share one crosswalk. */
|
|
397
|
+
declare const CrosswalkReferenceSchema: z.ZodUnion<readonly [z.ZodType<import("crossmap").Crosswalk, unknown, z.core.$ZodTypeInternals<import("crossmap").Crosswalk, unknown>> & {
|
|
398
|
+
is: (value: unknown) => value is import("crossmap").Crosswalk;
|
|
399
399
|
}, z.ZodObject<{
|
|
400
400
|
file: z.ZodString;
|
|
401
401
|
}, z.core.$strict>]>;
|
|
@@ -600,8 +600,8 @@ declare const EntrySchema: z.ZodObject<{
|
|
|
600
600
|
status: z.ZodLiteral<"not-fetchable">;
|
|
601
601
|
reason: z.ZodString;
|
|
602
602
|
}, z.core.$strict>], "status">;
|
|
603
|
-
crosswalk: z.ZodUnion<readonly [z.ZodType<import("
|
|
604
|
-
is: (value: unknown) => value is import("
|
|
603
|
+
crosswalk: z.ZodUnion<readonly [z.ZodType<import("crossmap").Crosswalk, unknown, z.core.$ZodTypeInternals<import("crossmap").Crosswalk, unknown>> & {
|
|
604
|
+
is: (value: unknown) => value is import("crossmap").Crosswalk;
|
|
605
605
|
}, z.ZodObject<{
|
|
606
606
|
file: z.ZodString;
|
|
607
607
|
}, z.core.$strict>]>;
|
|
@@ -744,7 +744,7 @@ declare const EntrySchema: z.ZodObject<{
|
|
|
744
744
|
status: "not-fetchable";
|
|
745
745
|
reason: string;
|
|
746
746
|
};
|
|
747
|
-
crosswalk: import("
|
|
747
|
+
crosswalk: import("crossmap").Crosswalk | {
|
|
748
748
|
file: string;
|
|
749
749
|
};
|
|
750
750
|
expectedCounts: Record<string, number>;
|
|
@@ -816,7 +816,7 @@ declare function parseManifest(doc: unknown): Manifest;
|
|
|
816
816
|
//#endregion
|
|
817
817
|
//#region src/catalogue.d.ts
|
|
818
818
|
/**
|
|
819
|
-
* A catalogue index: the manifest, every entry, and every crosswalk file of the catalogue, keyed by its catalogue-relative path. The build assembles it from a catalogue directory; a consumer holding its own catalogue (a fork, say) can pass one to `createCatalogue`. Entry ids are unique, every fully qualified name (the whole path of an id or alias) identifies one entry and is not another entry's id, every namespace segment the manifest gives aliases to is some entry's segment and no alias is itself one, every crosswalk passes
|
|
819
|
+
* A catalogue index: the manifest, every entry, and every crosswalk file of the catalogue, keyed by its catalogue-relative path. The build assembles it from a catalogue directory; a consumer holding its own catalogue (a fork, say) can pass one to `createCatalogue`. Entry ids are unique, every fully qualified name (the whole path of an id or alias) identifies one entry and is not another entry's id, every namespace segment the manifest gives aliases to is some entry's segment and no alias is itself one, every crosswalk passes crossmap's structural validation, every crosswalk file reference resolves, and every crosswalk's target is the category of each entry that uses it.
|
|
820
820
|
*/
|
|
821
821
|
declare const CatalogueIndexSchema: z.ZodObject<{
|
|
822
822
|
manifest: z.ZodObject<{
|
|
@@ -1498,7 +1498,7 @@ declare function collectionOf<Document extends object>(document: z.ZodType<Docum
|
|
|
1498
1498
|
//#endregion
|
|
1499
1499
|
//#region src/targets/tasks-v1.d.ts
|
|
1500
1500
|
/**
|
|
1501
|
-
* A task: a place or case to visit or serve, such as a stop of a route or a case in a register, flat so a
|
|
1501
|
+
* A task: a place or case to visit or serve, such as a stop of a route or a case in a register, flat so a crossmap crosswalk can map a table's columns onto it (a wildcard path must keep the same shape on both sides). The target fixes only what the datasets share and lets a crosswalk map any further column through under its own name, as graph/v1 does. `lat` and `lon` are WGS84 degrees and `x` and `y` are planar coordinates in the dataset's own system; `duration` and `travelTime` are seconds, `prize` is the dataset's own reward measure, and `prizeProbability` is the chance the reward is collected. `skill` and `territory` name what is needed to serve the node and where it sits, `status` is where it is in its lifecycle, and `opened`, `closed` and `deadline` are its lifecycle times as the source writes them, a deadline being a number where the source gives a time of day.
|
|
1502
1502
|
*/
|
|
1503
1503
|
declare const TaskSchema: z.ZodObject<{
|
|
1504
1504
|
id: z.ZodString;
|
package/dist/index.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { $ as CrosswalkReference, A as Catalogue, B as ManifestSchema, C as JsonScalarSchema, Ct as waybackUrl, D as GraphNodeSchema, E as GraphNode, F as NamespaceNode, G as ArchiveMember, H as parseManifest, I as createCatalogue, J as Citation, K as ArchiveMemberSchema, L as Lineage, M as CatalogueIndexSchema, N as Dataset, O as GraphV1, P as DatasetQuery, Q as ColumnNameSchema, R as LineageSchema, S as JsonScalar, St as parseEntry, T as GraphEdgeSchema, U as AliasSchema, V as SemVerSchema, W as Archive, X as CollectionArchive, Y as CitationSchema, Z as CollectionArchiveSchema, _ as TasksV1, _t as SnapshotSchema, a as CategoryTargets, at as IdSchema, b as Instance, bt as SubjectSchema, c as countTarget, ct as Namespace, d as getCollectionTarget, dt as ParserSchema, et as CrosswalkReferenceSchema, f as getTarget, ft as RESERVED_META_KEYS, g as TaskSchema, gt as Snapshot, h as Task, ht as RelativePathSchema, i as CategoryTarget, it as HttpUrlSchema, j as CatalogueIndex, k as GraphV1Schema, l as countedKeys, lt as NamespaceSchema, m as listCategories, mt as RecipeSchema, n as CategoryId, nt as Entry, o as CollectionTarget, ot as Licence, p as kindOf, pt as Recipe, q as ArchiveSchema, r as CategoryIdSchema, rt as EntrySchema, s as CountedKey, st as LicenceSchema, t as Category, tt as DocumentArchive, u as getCategory, ut as Parser, v as TasksV1Schema, vt as Source, w as GraphEdge, x as collectionOf, xt as WaybackTimestampSchema, y as Collection, yt as SourceSchema, z as Manifest } from "./categories-
|
|
2
|
-
import { Crosswalk } from "
|
|
1
|
+
import { $ as CrosswalkReference, A as Catalogue, B as ManifestSchema, C as JsonScalarSchema, Ct as waybackUrl, D as GraphNodeSchema, E as GraphNode, F as NamespaceNode, G as ArchiveMember, H as parseManifest, I as createCatalogue, J as Citation, K as ArchiveMemberSchema, L as Lineage, M as CatalogueIndexSchema, N as Dataset, O as GraphV1, P as DatasetQuery, Q as ColumnNameSchema, R as LineageSchema, S as JsonScalar, St as parseEntry, T as GraphEdgeSchema, U as AliasSchema, V as SemVerSchema, W as Archive, X as CollectionArchive, Y as CitationSchema, Z as CollectionArchiveSchema, _ as TasksV1, _t as SnapshotSchema, a as CategoryTargets, at as IdSchema, b as Instance, bt as SubjectSchema, c as countTarget, ct as Namespace, d as getCollectionTarget, dt as ParserSchema, et as CrosswalkReferenceSchema, f as getTarget, ft as RESERVED_META_KEYS, g as TaskSchema, gt as Snapshot, h as Task, ht as RelativePathSchema, i as CategoryTarget, it as HttpUrlSchema, j as CatalogueIndex, k as GraphV1Schema, l as countedKeys, lt as NamespaceSchema, m as listCategories, mt as RecipeSchema, n as CategoryId, nt as Entry, o as CollectionTarget, ot as Licence, p as kindOf, pt as Recipe, q as ArchiveSchema, r as CategoryIdSchema, rt as EntrySchema, s as CountedKey, st as LicenceSchema, t as Category, tt as DocumentArchive, u as getCategory, ut as Parser, v as TasksV1Schema, vt as Source, w as GraphEdge, x as collectionOf, xt as WaybackTimestampSchema, y as Collection, yt as SourceSchema, z as Manifest } from "./categories-IrB8M6Ne.js";
|
|
2
|
+
import { Crosswalk } from "crossmap";
|
|
3
3
|
//#region src/errors.d.ts
|
|
4
4
|
/** The root of this package's error taxonomy: every error keepset throws is a `KeepsetError`, so callers can catch the family with one check. */
|
|
5
5
|
export declare class KeepsetError extends Error {
|
|
@@ -111,7 +111,7 @@ export declare function serialise(doc: Entry | Manifest | Crosswalk): string;
|
|
|
111
111
|
//#endregion
|
|
112
112
|
//#region src/validate.d.ts
|
|
113
113
|
/**
|
|
114
|
-
* Checks a dataset as the catalogue holds it: the entry against the entry schema, and its crosswalk against
|
|
114
|
+
* Checks a dataset as the catalogue holds it: the entry against the entry schema, and its crosswalk against crossmap's structural validation and the entry's category. A catalogue index is checked the same way when it is built, so a bundled dataset is valid unless the package itself is broken; the check is what a contributor runs on an entry they are writing.
|
|
115
115
|
* @param dataset - The dataset.
|
|
116
116
|
* @returns Every issue found, with paths from the dataset's root (`entry.` or `crosswalk.`); empty when the dataset is valid.
|
|
117
117
|
*/
|
package/dist/index.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { $ as ArchiveError, A as WaybackTimestampSchema, B as countedKeys, C as ParserSchema, D as SnapshotSchema, E as RelativePathSchema, F as groupsOf, G as listCategories, H as getCollectionTarget, I as namesOf, J as collectionOf, K as TaskSchema, M as toIssues, N as waybackUrl, O as SourceSchema, P as NAME_SEPARATOR, Q as JsonScalarSchema, R as CategoryIdSchema, S as NamespaceSchema, T as RecipeSchema, U as getTarget, V as getCategory, W as kindOf, X as GraphNodeSchema, Y as GraphEdgeSchema, Z as GraphV1Schema, _ as CrosswalkReferenceSchema, a as CatalogueIndexSchema, at as RecipeError, b as IdSchema, c as ManifestSchema, d as AliasSchema, et as CatalogueSyntaxError, f as ArchiveMemberSchema, g as ColumnNameSchema, h as CollectionArchiveSchema, i as listNamespaces, it as NativeParseError, j as parseEntry, k as SubjectSchema, l as SemVerSchema, m as CitationSchema, n as getManifest, nt as DatasetNotFoundError, o as createCatalogue, ot as UnknownCategoryError, p as ArchiveSchema, q as TasksV1Schema, r as listDatasets, rt as KeepsetError, s as LineageSchema, t as getDataset, tt as DatasetAmbiguousError, u as parseManifest, v as EntrySchema, w as RESERVED_META_KEYS, x as LicenceSchema, y as HttpUrlSchema, z as countTarget } from "./bundled-
|
|
2
|
-
import { validateCrosswalk } from "
|
|
1
|
+
import { $ as ArchiveError, A as WaybackTimestampSchema, B as countedKeys, C as ParserSchema, D as SnapshotSchema, E as RelativePathSchema, F as groupsOf, G as listCategories, H as getCollectionTarget, I as namesOf, J as collectionOf, K as TaskSchema, M as toIssues, N as waybackUrl, O as SourceSchema, P as NAME_SEPARATOR, Q as JsonScalarSchema, R as CategoryIdSchema, S as NamespaceSchema, T as RecipeSchema, U as getTarget, V as getCategory, W as kindOf, X as GraphNodeSchema, Y as GraphEdgeSchema, Z as GraphV1Schema, _ as CrosswalkReferenceSchema, a as CatalogueIndexSchema, at as RecipeError, b as IdSchema, c as ManifestSchema, d as AliasSchema, et as CatalogueSyntaxError, f as ArchiveMemberSchema, g as ColumnNameSchema, h as CollectionArchiveSchema, i as listNamespaces, it as NativeParseError, j as parseEntry, k as SubjectSchema, l as SemVerSchema, m as CitationSchema, n as getManifest, nt as DatasetNotFoundError, o as createCatalogue, ot as UnknownCategoryError, p as ArchiveSchema, q as TasksV1Schema, r as listDatasets, rt as KeepsetError, s as LineageSchema, t as getDataset, tt as DatasetAmbiguousError, u as parseManifest, v as EntrySchema, w as RESERVED_META_KEYS, x as LicenceSchema, y as HttpUrlSchema, z as countTarget } from "./bundled-B_XXp-TS.js";
|
|
2
|
+
import { validateCrosswalk } from "crossmap";
|
|
3
3
|
//#region src/serialise.ts
|
|
4
4
|
/**
|
|
5
5
|
* Copies a JSON value with every object's keys in ascending code-unit order, the order the house JSON lint (`json/sort-keys`) requires of committed JSON. `Object.fromEntries` defines each key as an own data property, so a key such as `__proto__` stays a key.
|
|
@@ -30,7 +30,7 @@ function serialise(doc) {
|
|
|
30
30
|
//#endregion
|
|
31
31
|
//#region src/validate.ts
|
|
32
32
|
/**
|
|
33
|
-
* Checks a dataset as the catalogue holds it: the entry against the entry schema, and its crosswalk against
|
|
33
|
+
* Checks a dataset as the catalogue holds it: the entry against the entry schema, and its crosswalk against crossmap's structural validation and the entry's category. A catalogue index is checked the same way when it is built, so a bundled dataset is valid unless the package itself is broken; the check is what a contributor runs on an entry they are writing.
|
|
34
34
|
* @param dataset - The dataset.
|
|
35
35
|
* @returns Every issue found, with paths from the dataset's root (`entry.` or `crosswalk.`); empty when the dataset is valid.
|
|
36
36
|
*/
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { N as Dataset, gt as Snapshot, i as CategoryTarget, nt as Entry, o as CollectionTarget, vt as Source } from "../categories-
|
|
1
|
+
import { N as Dataset, gt as Snapshot, i as CategoryTarget, nt as Entry, o as CollectionTarget, vt as Source } from "../categories-IrB8M6Ne.js";
|
|
2
2
|
//#region src/node/fetch-dataset.d.ts
|
|
3
3
|
/** The network port a recipe downloads through: `downloadRaw` by default, which returns the bytes exactly as served, or a stand-in under test. */
|
|
4
4
|
export type FetchPort = (url: string, init: {
|
|
@@ -43,7 +43,7 @@ export interface FetchedDataset {
|
|
|
43
43
|
*/
|
|
44
44
|
export declare function downloadUrl(source: Source): string;
|
|
45
45
|
/**
|
|
46
|
-
* Runs a dataset's recipe: downloads the source (from its pinned snapshot when the source is dead), unpacks the archive, parses the native shape, applies the
|
|
46
|
+
* Runs a dataset's recipe: downloads the source (from its pinned snapshot when the source is dead), unpacks the archive, parses the native shape, applies the crossmap crosswalk and validates the result against the category target. The native document the crosswalk receives is the parser's output with `meta` built from the entry.
|
|
47
47
|
* @param dataset - A dataset from any catalogue.
|
|
48
48
|
* @param options - The network port, abort signal and clock.
|
|
49
49
|
* @returns The resolved target with its provenance and any count warnings.
|
|
@@ -69,7 +69,7 @@ export type RoundTripReport = {
|
|
|
69
69
|
readonly provenance: Provenance;
|
|
70
70
|
};
|
|
71
71
|
/**
|
|
72
|
-
* Checks a dataset's crosswalk on its real data: runs the recipe to the native document, then
|
|
72
|
+
* Checks a dataset's crosswalk on its real data: runs the recipe to the native document, then crossmap's `assertRoundTrip`, which encodes it, decodes the result and compares the two over every path the crosswalk promises to preserve.
|
|
73
73
|
* @param dataset - A dataset from any catalogue.
|
|
74
74
|
* @param options - The network port, abort signal and clock.
|
|
75
75
|
* @returns Whether the round trip holds, and where it first differs when it does not.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { $ as ArchiveError, H as getCollectionTarget, L as getOwn, U as getTarget, at as RecipeError, it as NativeParseError, rt as KeepsetError, t as getDataset, z as countTarget } from "../bundled-
|
|
2
|
-
import { RoundTripMismatchError, assertRoundTrip, encode } from "
|
|
1
|
+
import { $ as ArchiveError, H as getCollectionTarget, L as getOwn, U as getTarget, at as RecipeError, it as NativeParseError, rt as KeepsetError, t as getDataset, z as countTarget } from "../bundled-B_XXp-TS.js";
|
|
2
|
+
import { RoundTripMismatchError, assertRoundTrip, encode } from "crossmap";
|
|
3
3
|
import { z } from "zod";
|
|
4
4
|
import { createHash } from "node:crypto";
|
|
5
5
|
import { gunzipSync } from "node:zlib";
|
|
@@ -2945,7 +2945,7 @@ async function runToNative(dataset, options) {
|
|
|
2945
2945
|
};
|
|
2946
2946
|
}
|
|
2947
2947
|
/**
|
|
2948
|
-
* Runs a dataset's recipe: downloads the source (from its pinned snapshot when the source is dead), unpacks the archive, parses the native shape, applies the
|
|
2948
|
+
* Runs a dataset's recipe: downloads the source (from its pinned snapshot when the source is dead), unpacks the archive, parses the native shape, applies the crossmap crosswalk and validates the result against the category target. The native document the crosswalk receives is the parser's output with `meta` built from the entry.
|
|
2949
2949
|
* @param dataset - A dataset from any catalogue.
|
|
2950
2950
|
* @param options - The network port, abort signal and clock.
|
|
2951
2951
|
* @returns The resolved target with its provenance and any count warnings.
|
|
@@ -2990,7 +2990,7 @@ async function fetchDataset(id, options = {}) {
|
|
|
2990
2990
|
return fetchEntry(getDataset(id), options);
|
|
2991
2991
|
}
|
|
2992
2992
|
/**
|
|
2993
|
-
* Checks a dataset's crosswalk on its real data: runs the recipe to the native document, then
|
|
2993
|
+
* Checks a dataset's crosswalk on its real data: runs the recipe to the native document, then crossmap's `assertRoundTrip`, which encodes it, decodes the result and compares the two over every path the crosswalk promises to preserve.
|
|
2994
2994
|
* @param dataset - A dataset from any catalogue.
|
|
2995
2995
|
* @param options - The network port, abort signal and clock.
|
|
2996
2996
|
* @returns Whether the round trip holds, and where it first differs when it does not.
|
package/dist-cli/keepset.mjs
CHANGED
|
@@ -4,7 +4,7 @@ import { KeepsetError, NAME_SEPARATOR, countTarget, getDataset, listDatasets, li
|
|
|
4
4
|
import { fetchDataset, roundTripDataset } from "keepset/node";
|
|
5
5
|
import { parseArgs } from "node:util";
|
|
6
6
|
//#region package.json
|
|
7
|
-
var version = "0.29.
|
|
7
|
+
var version = "0.29.3";
|
|
8
8
|
//#endregion
|
|
9
9
|
//#region cli/run.ts
|
|
10
10
|
/**
|
|
@@ -36,7 +36,7 @@ Commands:
|
|
|
36
36
|
to <file> (default <id>.json), with its provenance beside
|
|
37
37
|
it in <file without .json>.provenance.json
|
|
38
38
|
validate <dataset> check the entry against the entry schema and its
|
|
39
|
-
crosswalk against
|
|
39
|
+
crosswalk against crossmap's validation
|
|
40
40
|
roundtrip <dataset> fetch the dataset and check that decoding the encoded
|
|
41
41
|
native document gives it back
|
|
42
42
|
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "keepset",
|
|
3
3
|
"description": "Library, CLI and statically generated catalogue for dataset fetch-and-normalise recipes",
|
|
4
|
-
"version": "0.29.
|
|
4
|
+
"version": "0.29.3",
|
|
5
5
|
"bin": {
|
|
6
6
|
"keepset": "./dist-cli/keepset.mjs"
|
|
7
7
|
},
|
|
@@ -10,10 +10,10 @@
|
|
|
10
10
|
"@mantine/hooks": "9.7.2",
|
|
11
11
|
"@tanstack/react-table": "9.2.8",
|
|
12
12
|
"@vanilla-extract/css": "1.21.3",
|
|
13
|
+
"crossmap": "0.2.0",
|
|
13
14
|
"fflate": "0.8.3",
|
|
14
15
|
"react": "19.3.0",
|
|
15
16
|
"react-dom": "19.3.0",
|
|
16
|
-
"twindex": "0.1.4",
|
|
17
17
|
"zod": "4.6.6"
|
|
18
18
|
},
|
|
19
19
|
"devDependencies": {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"$defs":{"Crosswalk":{"$ref":"https://cdn.jsdelivr.net/npm/twindex@0.1.4/schemas/twindex.schema.json"},"Entry":{"additionalProperties":false,"properties":{"aliases":{"items":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*(?:\\/[A-Za-z0-9][A-Za-z0-9._-]*)*$","type":"string"},"minItems":1,"type":"array"},"category":{"enum":["graph/v1","tasks/v1"],"type":"string"},"citations":{"items":{"additionalProperties":false,"properties":{"doi":{"pattern":"^10\\.\\d{4,9}\\/\\S+$","type":"string"},"text":{"minLength":1,"type":"string"},"url":{"format":"uri","type":"string"}},"required":["text"],"type":"object"},"type":"array"},"crosswalk":{"anyOf":[{"$ref":"#/$defs/Crosswalk"},{"additionalProperties":false,"properties":{"file":{"type":"string"}},"required":["file"],"type":"object"}]},"description":{"minLength":1,"type":"string"},"domain":{"minLength":1,"type":"string"},"expectedCounts":{"additionalProperties":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"propertyNames":{"pattern":"^[A-Za-z][A-Za-z0-9]*$","type":"string"},"type":"object"},"id":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"licence":{"oneOf":[{"additionalProperties":false,"properties":{"id":{"pattern":"^[A-Za-z0-9][A-Za-z0-9.+-]*$","type":"string"},"kind":{"const":"spdx","type":"string"}},"required":["kind","id"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"custom","type":"string"},"name":{"minLength":1,"type":"string"},"url":{"format":"uri","type":"string"}},"required":["kind","name","url"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"unrecorded","type":"string"}},"required":["kind"],"type":"object"}]},"name":{"minLength":1,"type":"string"},"namespace":{"items":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"minItems":1,"type":"array"},"properties":{"additionalProperties":{"type":["string","number","boolean","null"]},"propertyNames":{"pattern":"^[A-Za-z][A-Za-z0-9]*$","type":"string"},"type":"object"},"recipe":{"oneOf":[{"additionalProperties":false,"properties":{"archive":{"anyOf":[{"additionalProperties":false,"properties":{"kind":{"const":"none","type":"string"}},"required":["kind"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"gzip","type":"string"}},"required":["kind"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"enum":["zip","tar","tar.gz"],"type":"string"},"member":{"anyOf":[{"additionalProperties":false,"properties":{"path":{"type":"string"}},"required":["path"],"type":"object"},{"additionalProperties":false,"properties":{"extensions":{"items":{"pattern":"^\\.[A-Za-z0-9][A-Za-z0-9.]*$","type":"string"},"minItems":1,"type":"array"}},"required":["extensions"],"type":"object"},{"additionalProperties":false,"properties":{"pattern":{"minLength":1,"type":"string"}},"required":["pattern"],"type":"object"}]}},"required":["kind","member"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"enum":["zip","tar","tar.gz"],"type":"string"},"members":{"anyOf":[{"additionalProperties":false,"properties":{"path":{"type":"string"}},"required":["path"],"type":"object"},{"additionalProperties":false,"properties":{"extensions":{"items":{"pattern":"^\\.[A-Za-z0-9][A-Za-z0-9.]*$","type":"string"},"minItems":1,"type":"array"}},"required":["extensions"],"type":"object"},{"additionalProperties":false,"properties":{"pattern":{"minLength":1,"type":"string"}},"required":["pattern"],"type":"object"}]}},"required":["kind","members"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"xlsx","type":"string"},"sheet":{"minLength":1,"type":"string"}},"required":["kind","sheet"],"type":"object"}]},"paging":{"oneOf":[{"additionalProperties":false,"properties":{"kind":{"const":"arcgis-query","type":"string"},"orderBy":{"pattern":"^[A-Za-z_][A-Za-z0-9_]*$","type":"string"}},"required":["kind","orderBy"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"socrata-resource","type":"string"},"pageSize":{"maximum":9007199254740991,"minimum":1,"type":"integer"}},"required":["kind","pageSize"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"ckan-datastore","type":"string"},"pageSize":{"maximum":9007199254740991,"minimum":1,"type":"integer"}},"required":["kind","pageSize"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"openfema","type":"string"},"pageSize":{"maximum":9007199254740991,"minimum":1,"type":"integer"}},"required":["kind","pageSize"],"type":"object"}]},"parser":{"oneOf":[{"additionalProperties":false,"properties":{"commentPrefixes":{"items":{"minLength":1,"type":"string"},"type":"array"},"delimiter":{"enum":["detect","whitespace","tab","comma","space"],"type":"string"},"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"edge-list","type":"string"},"skipLines":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"sourceColumn":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"targetColumn":{"maximum":9007199254740991,"minimum":0,"type":"integer"}},"required":["format","encoding","delimiter","skipLines","commentPrefixes","sourceColumn","targetColumn"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"matrix-market","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"gml","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"pajek","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"customer-points","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"solomon","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"top","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"toptw","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"tsplib","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"const":"utf-8","type":"string"},"format":{"const":"verolog-2019","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"const":"utf-8","type":"string"},"format":{"const":"vrp-rep","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"columns":{"items":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"type":"array"},"delimiter":{"enum":["comma","tab","semicolon"],"type":"string"},"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"csv-table","type":"string"},"idColumn":{"anyOf":[{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},{"type":"null"}]},"nullValues":{"items":{"minLength":1,"type":"string"},"type":"array"},"numericColumns":{"items":{"additionalProperties":false,"properties":{"column":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"factor":{"exclusiveMinimum":0,"type":"number"}},"required":["column","factor"],"type":"object"},"type":"array"},"skipLines":{"maximum":9007199254740991,"minimum":0,"type":"integer"}},"required":["format","encoding","delimiter","skipLines","idColumn","numericColumns"],"type":"object"},{"additionalProperties":false,"properties":{"columns":{"items":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"minItems":1,"type":"array"},"dateColumns":{"items":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"type":"array"},"encoding":{"const":"utf-8","type":"string"},"format":{"const":"geojson","type":"string"},"idColumn":{"anyOf":[{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},{"type":"null"}]},"numericColumns":{"items":{"additionalProperties":false,"properties":{"column":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"factor":{"exclusiveMinimum":0,"type":"number"}},"required":["column","factor"],"type":"object"},"type":"array"}},"required":["format","encoding","idColumn","numericColumns","dateColumns","columns"],"type":"object"}]},"source":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"status":{"const":"fetchable","type":"string"}},"required":["status","source","archive","parser"],"type":"object"},{"additionalProperties":false,"properties":{"reason":{"minLength":1,"type":"string"},"status":{"const":"not-fetchable","type":"string"}},"required":["status","reason"],"type":"object"}]},"sources":{"items":{"additionalProperties":false,"properties":{"retrieved":{"format":"date","pattern":"^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))$","type":"string"},"snapshot":{"additionalProperties":false,"properties":{"timestamp":{"pattern":"^\\d{8}(?:\\d{6})?$","type":"string"},"url":{"format":"uri","type":"string"}},"required":["url","timestamp"],"type":"object"},"status":{"enum":["live","dead"],"type":"string"},"url":{"format":"uri","type":"string"}},"required":["url","retrieved","status"],"type":"object"},"minItems":1,"type":"array"},"subjects":{"items":{"pattern":"^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$","type":"string"},"minItems":1,"type":"array"},"tags":{"items":{"minLength":1,"type":"string"},"type":"array"}},"required":["id","name","tags","category","sources","licence","citations","recipe","crosswalk","expectedCounts","properties"],"type":"object"}},"$id":"https://cdn.jsdelivr.net/npm/keepset@0.29.1/schemas/entry.schema.json","$ref":"#/$defs/Entry","$schema":"https://json-schema.org/draft/2020-12/schema"}
|
|
1
|
+
{"$defs":{"Crosswalk":{"$ref":"https://cdn.jsdelivr.net/npm/crossmap@0.2.0/schemas/crossmap.schema.json"},"Entry":{"additionalProperties":false,"properties":{"aliases":{"items":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*(?:\\/[A-Za-z0-9][A-Za-z0-9._-]*)*$","type":"string"},"minItems":1,"type":"array"},"category":{"enum":["graph/v1","tasks/v1"],"type":"string"},"citations":{"items":{"additionalProperties":false,"properties":{"doi":{"pattern":"^10\\.\\d{4,9}\\/\\S+$","type":"string"},"text":{"minLength":1,"type":"string"},"url":{"format":"uri","type":"string"}},"required":["text"],"type":"object"},"type":"array"},"crosswalk":{"anyOf":[{"$ref":"#/$defs/Crosswalk"},{"additionalProperties":false,"properties":{"file":{"type":"string"}},"required":["file"],"type":"object"}]},"description":{"minLength":1,"type":"string"},"domain":{"minLength":1,"type":"string"},"expectedCounts":{"additionalProperties":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"propertyNames":{"pattern":"^[A-Za-z][A-Za-z0-9]*$","type":"string"},"type":"object"},"id":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"licence":{"oneOf":[{"additionalProperties":false,"properties":{"id":{"pattern":"^[A-Za-z0-9][A-Za-z0-9.+-]*$","type":"string"},"kind":{"const":"spdx","type":"string"}},"required":["kind","id"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"custom","type":"string"},"name":{"minLength":1,"type":"string"},"url":{"format":"uri","type":"string"}},"required":["kind","name","url"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"unrecorded","type":"string"}},"required":["kind"],"type":"object"}]},"name":{"minLength":1,"type":"string"},"namespace":{"items":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"minItems":1,"type":"array"},"properties":{"additionalProperties":{"type":["string","number","boolean","null"]},"propertyNames":{"pattern":"^[A-Za-z][A-Za-z0-9]*$","type":"string"},"type":"object"},"recipe":{"oneOf":[{"additionalProperties":false,"properties":{"archive":{"anyOf":[{"additionalProperties":false,"properties":{"kind":{"const":"none","type":"string"}},"required":["kind"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"gzip","type":"string"}},"required":["kind"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"enum":["zip","tar","tar.gz"],"type":"string"},"member":{"anyOf":[{"additionalProperties":false,"properties":{"path":{"type":"string"}},"required":["path"],"type":"object"},{"additionalProperties":false,"properties":{"extensions":{"items":{"pattern":"^\\.[A-Za-z0-9][A-Za-z0-9.]*$","type":"string"},"minItems":1,"type":"array"}},"required":["extensions"],"type":"object"},{"additionalProperties":false,"properties":{"pattern":{"minLength":1,"type":"string"}},"required":["pattern"],"type":"object"}]}},"required":["kind","member"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"enum":["zip","tar","tar.gz"],"type":"string"},"members":{"anyOf":[{"additionalProperties":false,"properties":{"path":{"type":"string"}},"required":["path"],"type":"object"},{"additionalProperties":false,"properties":{"extensions":{"items":{"pattern":"^\\.[A-Za-z0-9][A-Za-z0-9.]*$","type":"string"},"minItems":1,"type":"array"}},"required":["extensions"],"type":"object"},{"additionalProperties":false,"properties":{"pattern":{"minLength":1,"type":"string"}},"required":["pattern"],"type":"object"}]}},"required":["kind","members"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"xlsx","type":"string"},"sheet":{"minLength":1,"type":"string"}},"required":["kind","sheet"],"type":"object"}]},"paging":{"oneOf":[{"additionalProperties":false,"properties":{"kind":{"const":"arcgis-query","type":"string"},"orderBy":{"pattern":"^[A-Za-z_][A-Za-z0-9_]*$","type":"string"}},"required":["kind","orderBy"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"socrata-resource","type":"string"},"pageSize":{"maximum":9007199254740991,"minimum":1,"type":"integer"}},"required":["kind","pageSize"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"ckan-datastore","type":"string"},"pageSize":{"maximum":9007199254740991,"minimum":1,"type":"integer"}},"required":["kind","pageSize"],"type":"object"},{"additionalProperties":false,"properties":{"kind":{"const":"openfema","type":"string"},"pageSize":{"maximum":9007199254740991,"minimum":1,"type":"integer"}},"required":["kind","pageSize"],"type":"object"}]},"parser":{"oneOf":[{"additionalProperties":false,"properties":{"commentPrefixes":{"items":{"minLength":1,"type":"string"},"type":"array"},"delimiter":{"enum":["detect","whitespace","tab","comma","space"],"type":"string"},"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"edge-list","type":"string"},"skipLines":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"sourceColumn":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"targetColumn":{"maximum":9007199254740991,"minimum":0,"type":"integer"}},"required":["format","encoding","delimiter","skipLines","commentPrefixes","sourceColumn","targetColumn"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"matrix-market","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"gml","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"pajek","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"customer-points","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"solomon","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"top","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"toptw","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"tsplib","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"const":"utf-8","type":"string"},"format":{"const":"verolog-2019","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"encoding":{"const":"utf-8","type":"string"},"format":{"const":"vrp-rep","type":"string"}},"required":["format","encoding"],"type":"object"},{"additionalProperties":false,"properties":{"columns":{"items":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"type":"array"},"delimiter":{"enum":["comma","tab","semicolon"],"type":"string"},"encoding":{"enum":["utf-8","latin1"],"type":"string"},"format":{"const":"csv-table","type":"string"},"idColumn":{"anyOf":[{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},{"type":"null"}]},"nullValues":{"items":{"minLength":1,"type":"string"},"type":"array"},"numericColumns":{"items":{"additionalProperties":false,"properties":{"column":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"factor":{"exclusiveMinimum":0,"type":"number"}},"required":["column","factor"],"type":"object"},"type":"array"},"skipLines":{"maximum":9007199254740991,"minimum":0,"type":"integer"}},"required":["format","encoding","delimiter","skipLines","idColumn","numericColumns"],"type":"object"},{"additionalProperties":false,"properties":{"columns":{"items":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"minItems":1,"type":"array"},"dateColumns":{"items":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"type":"array"},"encoding":{"const":"utf-8","type":"string"},"format":{"const":"geojson","type":"string"},"idColumn":{"anyOf":[{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},{"type":"null"}]},"numericColumns":{"items":{"additionalProperties":false,"properties":{"column":{"pattern":"^[A-Za-z](?:[A-Za-z0-9_]*[A-Za-z0-9])?$","type":"string"},"factor":{"exclusiveMinimum":0,"type":"number"}},"required":["column","factor"],"type":"object"},"type":"array"}},"required":["format","encoding","idColumn","numericColumns","dateColumns","columns"],"type":"object"}]},"source":{"maximum":9007199254740991,"minimum":0,"type":"integer"},"status":{"const":"fetchable","type":"string"}},"required":["status","source","archive","parser"],"type":"object"},{"additionalProperties":false,"properties":{"reason":{"minLength":1,"type":"string"},"status":{"const":"not-fetchable","type":"string"}},"required":["status","reason"],"type":"object"}]},"sources":{"items":{"additionalProperties":false,"properties":{"retrieved":{"format":"date","pattern":"^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))$","type":"string"},"snapshot":{"additionalProperties":false,"properties":{"timestamp":{"pattern":"^\\d{8}(?:\\d{6})?$","type":"string"},"url":{"format":"uri","type":"string"}},"required":["url","timestamp"],"type":"object"},"status":{"enum":["live","dead"],"type":"string"},"url":{"format":"uri","type":"string"}},"required":["url","retrieved","status"],"type":"object"},"minItems":1,"type":"array"},"subjects":{"items":{"pattern":"^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$","type":"string"},"minItems":1,"type":"array"},"tags":{"items":{"minLength":1,"type":"string"},"type":"array"}},"required":["id","name","tags","category","sources","licence","citations","recipe","crosswalk","expectedCounts","properties"],"type":"object"}},"$id":"https://cdn.jsdelivr.net/npm/keepset@0.29.3/schemas/entry.schema.json","$ref":"#/$defs/Entry","$schema":"https://json-schema.org/draft/2020-12/schema"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"$defs":{"Manifest":{"additionalProperties":false,"properties":{"description":{"minLength":1,"type":"string"},"id":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"lineage":{"additionalProperties":false,"properties":{"catalogue":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"repository":{"format":"uri","type":"string"},"version":{"pattern":"^(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)(?:-[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?(?:\\+[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?$","type":"string"}},"required":["catalogue","version","repository"],"type":"object"},"name":{"minLength":1,"type":"string"},"namespaceAliases":{"additionalProperties":{"items":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"minItems":1,"type":"array"},"propertyNames":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"type":"object"},"repository":{"format":"uri","type":"string"},"version":{"pattern":"^(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)(?:-[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?(?:\\+[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?$","type":"string"}},"required":["id","name","description","version","repository"],"type":"object"}},"$id":"https://cdn.jsdelivr.net/npm/keepset@0.29.
|
|
1
|
+
{"$defs":{"Manifest":{"additionalProperties":false,"properties":{"description":{"minLength":1,"type":"string"},"id":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"lineage":{"additionalProperties":false,"properties":{"catalogue":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"repository":{"format":"uri","type":"string"},"version":{"pattern":"^(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)(?:-[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?(?:\\+[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?$","type":"string"}},"required":["catalogue","version","repository"],"type":"object"},"name":{"minLength":1,"type":"string"},"namespaceAliases":{"additionalProperties":{"items":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"minItems":1,"type":"array"},"propertyNames":{"pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","type":"string"},"type":"object"},"repository":{"format":"uri","type":"string"},"version":{"pattern":"^(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)(?:-[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?(?:\\+[0-9A-Za-z-]+(?:\\.[0-9A-Za-z-]+)*)?$","type":"string"}},"required":["id","name","description","version","repository"],"type":"object"}},"$id":"https://cdn.jsdelivr.net/npm/keepset@0.29.3/schemas/manifest.schema.json","$ref":"#/$defs/Manifest","$schema":"https://json-schema.org/draft/2020-12/schema"}
|