keepset 0.37.1 → 0.39.0

package/README.md

@@ -63,7 +63,7 @@ The catalogue itself is a manifest (`catalogue/manifest.json`: id, version, repo
 
 ## Categories and normalisation targets
 
-A category names a normalisation target: the shape every entry in that category resolves to, whatever its native shape. `graph/v1` is `{ meta, nodes, edges }` with unique node ids and edges whose endpoints name nodes; its entries are graph datasets taken from upstream collections such as SuiteSparse, Netzschleuder, SNAP, KONECT and OGB, read by one parser per format (delimited edge lists, Matrix Market, GML, Pajek, UCINet DL full matrices, Graph Drawing contest files, KONECT edge files, rating and count files, knowledge graph triples and Heterogeneous Graph Benchmark node and link files), each giving the graph's topology and, where the format states them, node and edge types, weights, times and the split each edge belongs to. `tasks/v1` is `{ meta, nodes }`, a set of tasks as the places or cases it offers to visit or serve (a routing benchmark, or an open data register of permits and complaints), each flat with optional coordinates, prize, prize probability, duration, travel time, skill, territory, lifecycle status and times, and deadline, read from tabular sources by the `csv-table` parser (a delimited file, a member of an archive, a sheet of an Excel workbook, or an open data portal read a page at a time) and from ArcGIS feature layers by the `geojson` parser. Its entries come from a survey of open operations-research datasets, imported with a curation that records, for each source, what its download gives and how its columns map onto the node; a dataset whose source has no usable table is imported without a recipe and says why. Any category can be fetched as a collection: a recipe that selects every matching archive member with `members` (by path, by the extensions its name ends with, or by a regular expression its whole path matches) resolves to `{ meta, instances }`, where each instance is a document of the entry's category under the path of the member it came from, the schema derived from the category's own, so a category needs nothing of its own to be collected. The parser reads each member, the crosswalk is applied to each, and a member that does not parse or does not validate fails the fetch and is named. Instance files are read by one parser per format (Solomon and TOPTW time window files, Chao team orienteering files, TSPLIB style orienteering files, VRP-REP XML, VeRoLog 2019 files and customer point lists), which gives each node the columns the file has and each instance a header of its own; a curation says which of them fill the `tasks/v1` node's fields, which pass through under their own names and which are dropped, so what an entry leaves out is written down, and a source whose time quantities are not seconds keeps them under the source's names with the unit recorded as the entry's `timeUnit`. A new category starts by defining its target schema in `src/targets/` and registering it in the category registry; an entry cannot name a category that does not exist. The statically generated site renders every entry with schema.org Dataset JSON-LD, so the catalogue surfaces in the dataset discovery layers.
+A category names a normalisation target: the shape every entry in that category resolves to, whatever its native shape. `graph/v1` is `{ meta, nodes, edges }` with unique node ids and edges whose endpoints name nodes; its entries are graph datasets taken from upstream collections such as SuiteSparse, Netzschleuder, SNAP, KONECT and OGB, read by one parser per format (delimited edge lists, which can also be a sheet of a spreadsheet, Matrix Market, GML, Pajek, UCINet DL full matrices, Graph Drawing contest files, Witvliet connectome synapse files, Open Graph Benchmark raw directories read together from the several files of a dataset's zip, KONECT edge files, rating and count files, knowledge graph triples and Heterogeneous Graph Benchmark node and link files), each giving the graph's topology and, where the format states them, node and edge types, weights, times and the split each edge belongs to. `tasks/v1` is `{ meta, nodes }`, a set of tasks as the places or cases it offers to visit or serve (a routing benchmark, or an open data register of permits and complaints), each flat with optional coordinates, prize, prize probability, duration, travel time, skill, territory, lifecycle status and times, and deadline, read from tabular sources by the `csv-table` parser (a delimited file, a member of an archive, a sheet of an Excel workbook, a legacy binary xls workbook or an OpenDocument spreadsheet, or an open data portal read a page at a time) and from ArcGIS feature layers by the `geojson` parser. Its entries come from a survey of open operations-research datasets, imported with a curation that records, for each source, what its download gives and how its columns map onto the node; a dataset whose source has no usable table is imported without a recipe and says why. Any category can be fetched as a collection: a recipe that selects every matching archive member with `members` (by path, by the extensions its name ends with, or by a regular expression its whole path matches) resolves to `{ meta, instances }`, where each instance is a document of the entry's category under the path of the member it came from, the schema derived from the category's own, so a category needs nothing of its own to be collected. The parser reads each member, the crosswalk is applied to each, and a member that does not parse or does not validate fails the fetch and is named. Instance files are read by one parser per format (Solomon and TOPTW time window files, Chao team orienteering files, TSPLIB style orienteering files, VRP-REP XML, VeRoLog 2019 files and customer point lists), which gives each node the columns the file has and each instance a header of its own; a curation says which of them fill the `tasks/v1` node's fields, which pass through under their own names and which are dropped, so what an entry leaves out is written down, and a source whose time quantities are not seconds keeps them under the source's names with the unit recorded as the entry's `timeUnit`. A new category starts by defining its target schema in `src/targets/` and registering it in the category registry; an entry cannot name a category that does not exist. The statically generated site renders every entry with schema.org Dataset JSON-LD, so the catalogue surfaces in the dataset discovery layers.
 
 ## Licensing