vis-chronicle 0.0.2

package/.vscode/settings.json

@@ -0,0 +1,5 @@
+{
+	"editor.insertSpaces": false,
+	"editor.tabSize": 4,
+	"editor.detectIndentation": false
+}

package/LICENSE.md

@@ -0,0 +1,5 @@
+Copyright 2025 Brian MacIntosh
+
+Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED “AS IS” AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

package/README.md

@@ -0,0 +1,95 @@
+# vis-chronicle
+
+vis-chronicle is a tool to generate web-based timelines from Wikidata SPARQL queries. You feed the tool a specification JSON file that describes what items to put on the timeline. The tool gathers the needed data from Wikidata and produces a JSON file suitable for feeding directly to [vis-timeline](https://github.com/visjs/vis-timeline).
+
+[vis-chronicle-demo](https://github.com/BrianMacIntosh/vis-chronicle-demo) is a demo application using this tool.
+
+## Example
+
+This specification file generates a timeline containing the lifespans of every United States president.
+
+```json
+{
+	"groups":
+	[
+		{
+			"id": "presidents",
+			"content": ""
+		}
+	],
+	"options":
+	{
+		"width": "100%",
+		"preferZoom": false,
+		"showCurrentTime": true,
+		"stack": false,
+		"stackSubgroups": false,
+		"verticalScroll": true
+	},
+	"queryTemplates":
+	{
+		"dateOfBirth": "{entity} p:P569 ?_prop. ?_prop psv:P569 ?_value.",
+		"dateOfDeath": "{entity} p:P570 ?_prop. ?_prop psv:P570 ?_value."
+	},
+	"items":
+	[
+		{
+			"comment": "All United States presidents",
+			"itemQuery": "?_node wdt:P39 wd:Q11696. ?_node wdt:P31 wd:Q5.",
+			"group": "presidents",
+
+			"startQuery": "#dateOfBirth",
+			"endQuery": "#dateOfDeath"
+		},
+		{
+			"label": "Ed Kealty (fictional)",
+			"entity": "Q5335019",
+			"group": "presidents",
+			"startQuery": "#dateOfBirth",
+			"endQuery": "#dateOfDeath"
+		}
+	]
+}
+
+```
+
+`groups` and `options` are transparently passed through for vis-timeline - see the vis-timeline documentation. `"stack": false` and `"stackSubgroups": false` false are necessary in order for chronicle's open-ended ranges to display correctly.
+
+`queryTemplates` contains template queries to be re-used by multiple items. vis-chronicle comes with a few template queries by default. Queries need to match a property (`p:`) using the variable `?_prop` and a value (such as `psv:` or `pqv:`) using the variable `?_value`. Also, any format variable like `{format}` will be replaced by the same-named value on the item entry, allowing templates to be parameterized.
+
+`items` contains any number of items to generate. An item can be a single, literal item, or a multi-item generator (using `itemQuery` or `items`).
+
+Item properties:
+* `label`: Literal items only. Display label for the item. Can contain HTML. For generators, wildcards can be used:
+    * `{_LABEL}` will be replaced by the Wikidata label.
+	* `{_QID}` will be replaced by the Wikidata Q-id.
+* `itemQuery`: Generators only. A SPARQL query segment to select all the items that should be generated. `?_node` stands in for the ouput variable. The `entity` property is added to each item with the item's entity id. Item labels are automatically fetched from Wikidata.
+* `items`: Generators only. An array of Wikidata ids. Works like `itemQuery` but with an explicit list.
+* `itemRange`: Generators only. Only generates items whose time spans fall at least partially within the provided range.
+	* `min`
+    * `max`
+* `group`: The vis-timeline group to put the item(s) in.
+* `className`: CSS class to apply to the timeline div.
+* `type`: vis-timeline type of the item.
+* `startQuery`: A SPARQL query segment that selects the desired start time of the object. These should select Wikidata properties (not statements or values). Can be a query term like `{entity} p:P569 ?_prop. ?_prop psv:P569 ?_value.` or a template query like `#dateOfBirth`.
+* `endQuery`: Same as `startQuery`, but for the end time.
+* `startEndQuery`: Use instead of `startQuery` and `endQuery` separately if the start and end times are from qualifiers. This guarantees they will be drawn from the same statement.
+    * `general`: General query segment, usually for selecting the item and property.
+	* `start`: Query segment for selecting the start value.
+	* `end`: Query segment for selecting the end value.
+* `expectedDuration`: Describes the expected duration, for hinting if the start or end is missing.
+    * `min`: The absolute minimum duration.
+	* `max`: The absolute maximum duration.
+	* `avg`: The average duration.
+
+## Parameters
+
+The tool can be run as a package script like `vis-chronicle ./src/timeline.json -v`.
+
+The tool takes these parameters on the command line:
+* (Mandatory) The name of the input file or directory. If a directory, all JSON files in the directory will be combined together.
+* (Optional) The name of the output file. Defaults to `intermediate/timeline.json`.
+* `-v[erbose]`: Print verbose output, including the actual queries being run.
+* `-skip-wd-cache`: Do not read anything from the local cache, query all data fresh.
+* `-q[uery-url]`: The URL of the SPARQL endpoint. Defaults to `https://query.wikidata.org/sparql`.
+* `-l[ang]`: Language priorities to use when fetching labels. Defaults to `en,mul`. See (SPARQL/SERVICE - Label)[https://en.wikibooks.org/wiki/SPARQL/SERVICE_-_Label].

package/package.json

@@ -0,0 +1,30 @@
+{
+  "name": "vis-chronicle",
+  "version": "0.0.2",
+  "description": "Generates JSON for populating a vis.js timeline from Wikidata queries.",
+  "keywords": [
+	"wikidata", "visjs", "timeline"
+  ],
+  "homepage": "https://github.com/BrianMacIntosh/vis-chronicle",
+  "bugs": "https://github.com/BrianMacIntosh/vis-chronicle/issues",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/BrianMacIntosh/vis-chronicle.git"
+  },
+  "bin": {
+	"vis-chronicle": "src/fetch.js"
+  },
+  "scripts": {
+  },
+  "author": {
+	"name": "Brian MacIntosh",
+	"email": "brianamacintosh@gmail.com",
+	"url": "https://brianmacintosh.com"
+  },
+  "license": "ISC",
+  "dependencies": {
+  },
+  "devDependencies": {
+    "moment": "2.30.1"
+  }
+}

package/src/fetch.js

@@ -0,0 +1,544 @@
+#!/usr/bin/env node
+
+// Uses an input specification file to produce an output file for vis.js Timeline.
+
+const path = require('path')
+const nodeutil = require('node:util')
+const assert = require('node:assert/strict');
+
+// parse args
+const { values, positionals } = nodeutil.parseArgs({
+	allowPositionals: true,
+	options: {
+		"verbose": { type: 'boolean', short: 'v', default: false },
+		"skip-wd-cache": { type: 'boolean', default: false },
+		"query-url": { type: 'string', short: 'q', default: "https://query.wikidata.org/sparql" },
+		"lang": { type: 'string', short: 'l', default: "en,mul" },
+		"cachebuster": { type: 'string', default: undefined}
+	}})
+	
+var specFile = positionals[0]
+if (!specFile)
+{
+	console.error(`No specification file or directory provided.`)
+	console.log(`Usage: vis-chronicle IN [OUT]`)
+}
+else
+{
+	console.log(`Fetching using spec '${specFile}'.`)
+}
+
+var outputFile = positionals[1]
+if (!outputFile)
+{
+	outputFile = "intermediate/timeline.json"
+}
+
+const moment = require('moment')
+const fs = require('fs');
+const wikidata = require('./wikidata.js')
+const mypath = require("./mypath.js")
+const globalData = require("./global-data.json")
+
+function postprocessDuration(duration)
+{
+	if (duration.min)
+		duration.min = moment.duration(duration.min)
+	if (duration.max)
+		duration.max = moment.duration(duration.max)
+	if (duration.avg)
+		duration.avg = moment.duration(duration.avg)
+	return duration
+}
+function postprocessGlobalData()
+{
+	for (const expectation of globalData.expectations)
+	{
+		if (expectation.duration)
+		{
+			postprocessDuration(expectation.duration)
+		}
+	}
+}
+postprocessGlobalData()
+
+wikidata.skipCache = values["skip-wd-cache"]
+wikidata.cacheBuster = values["cachebuster"]
+wikidata.sparqlUrl = values["query-url"]
+wikidata.verboseLogging = values["verbose"]
+wikidata.setLang(values["lang"])
+wikidata.initialize()
+
+// produces a moment from a Wikidata time
+function wikidataToMoment(inTime)
+{
+	if (!inTime || !inTime.value)
+	{
+		// missing value
+		return undefined
+	}
+
+	// moment has trouble with negative years unless they're six digits
+	const date = moment.utc(inTime.value, 'YYYYYY-MM-DDThh:mm:ss')
+
+	//TODO: do something nice in the GUI to indicate imprecision of times
+	switch (inTime.precision)
+	{
+		case 0: case 1: case 2: case 3: case 4: case 5: case 6: case 7: case 8: case 9:
+			const yearBase = Math.pow(10, 9 - inTime.precision)
+			const roundedYear = Math.round(date.year() / yearBase) * yearBase
+			return date.year(roundedYear).startOf('year')
+		case 10: // month precision
+			return date.startOf('month')
+		case 11: // day precision
+			return date.startOf('day')
+		case 12: // hour precision
+			return date.startOf('hour')
+		case 13: // minute precision
+			return date.startOf('minute')
+		case 14: // second precision
+			return date.startOf('second')
+		default:
+			throw `Unrecognized date precision ${inTime.precision}`
+	}
+}
+
+function getExpectation(item)
+{
+	if (item.expectedDuration)
+	{
+		return { "duration": item.expectedDuration };
+	}
+
+	for (const expectation of globalData.expectations)
+	{
+		if (expectation.startQuery && item.startQuery != expectation.startQuery)
+		{
+			continue;
+		}
+		if (expectation.endQuery && item.endQuery != expectation.endQuery)
+		{
+			continue;
+		}
+		if (expectation.startEndQuery && item.startEndQuery != expectation.startEndQuery)
+		{
+			continue;
+		}
+		return expectation;
+	}
+	assert(false) // expect at least a universal fallback expectation
+	return undefined
+}
+
+// produces JSON output from the queried data
+function produceOutput(items)
+{
+	const finalizeItem = function(item)
+	{
+		if (item.start)
+			item.start = item.start.format("YYYYYY-MM-DDThh:mm:ss")
+		if (item.end)
+			item.end = item.end.format("YYYYYY-MM-DDThh:mm:ss")
+		outputObject.items.push(item)
+	}
+
+	var outputObject = { items: [], groups: wikidata.inputSpec.groups, options: wikidata.inputSpec.options }
+	for (const item of items)
+	{
+		var outputItem = {
+			id: item.id,
+			content: item.label,
+			className: item.className,
+			comment: item.comment,
+			type: item.type
+		}
+		if (item.group)
+		{
+			outputItem.group = item.group
+			outputItem.subgroup = item.subgroup ? item.subgroup : item.entity
+		}
+
+		const isRangeType = !outputItem.type || outputItem.type == "range" || outputItem.type == "background"
+
+		// for debugging
+		outputItem.className = [ outputItem.className, item.entity ].join(' ')
+		
+		// look up duration expectations
+		const expectation = getExpectation(item)
+
+		assert(expectation && expectation.duration && expectation.duration.avg) // expect at least a universal fallback expectation
+
+		if (!item.start && !item.end
+			&& !item.start_min && !item.start_max
+			&& !item.end_min && !item.end_max)
+		{
+			console.warn(`Item ${item.id} has no date data at all.`)
+			continue
+		}
+		
+		outputItem.start = wikidataToMoment(item.start)
+		outputItem.end = wikidataToMoment(item.end)
+		const start_min = item.start_min ? wikidataToMoment(item.start_min) : outputItem.start
+		const start_max = item.start_max ? wikidataToMoment(item.start_max) : outputItem.start
+		const end_min = item.end_min ? wikidataToMoment(item.end_min) : outputItem.end
+		const end_max = item.end_max ? wikidataToMoment(item.end_max) : outputItem.end
+
+		// exclude items that violate itemRange constraints
+		//OPT: do this at an earlier stage? (e.g. when running the first query)
+		if (item.itemRange)
+		{
+			if (item.itemRange.min && moment(item.itemRange.min).isAfter(end_max))
+				continue
+			if (item.itemRange.max && moment(item.itemRange.max).isBefore(end_max))
+				continue
+		}
+
+		// no certainty at all
+		if (start_max >= end_min)
+		{
+			outputItem.className = [ outputItem.className, 'visc-uncertain' ].join(' ')
+			outputItem.start = start_min
+			outputItem.end = end_max
+
+			finalizeItem(outputItem)
+			continue
+		}
+
+		if (!isRangeType)
+		{
+			finalizeItem(outputItem)
+			continue
+		}
+
+		// handle end date
+		if (end_min && end_max && end_min < end_max)
+		{
+			// uncertain end
+
+			// find lower bound of uncertain region
+			var uncertainMin
+			if (item.end_min)
+				uncertainMin = end_min
+			else if (outputItem.end)
+				uncertainMin = outputItem.end
+			else
+				uncertainMin = outputItem.start //TODO: use min/max start
+			assert(uncertainMin)
+			
+			// add uncertain range
+			outputObject.items.push({
+				id: outputItem.id + "-unc-end",
+				className: [outputItem.className, "visc-uncertain", "visc-left-connection"].join(' '),
+				content: item.label ? "&nbsp;" : "",
+				start: uncertainMin.format("YYYYYY-MM-DDThh:mm:ss"),
+				end: end_max.format("YYYYYY-MM-DDThh:mm:ss"),
+				group: item.group,
+				subgroup: outputItem.subgroup
+			})
+
+			// adjust normal range to match
+			outputItem.end = uncertainMin
+			outputItem.className = [ outputItem.className, 'visc-right-connection' ].join(' ')
+		}
+		else if (!outputItem.end)
+		{
+			// open-ended end
+			const useMax = expectation.duration.max ? expectation.duration.max : moment(expectation.duration.avg.asMilliseconds() * 2)
+			if (outputItem.start.clone().add(useMax) < moment())
+			{
+				// 'max' is less than 'now'; it is likely this duration is not ongoing but has an unknown end
+				//TODO: accomodate wikidata special 'no value'
+				outputItem.end = outputItem.start.clone().add(expectation.duration.avg)
+			}
+			else
+			{
+				// 'now' is within 'max' and so it is a reasonable guess that this duration is ongoing
+				outputItem.end = moment()
+			}
+
+			const avgDuration = moment.duration(expectation.duration.avg) //HACK: TODO: consistently postprocess expectations, or don't
+			const actualDuration = moment.duration(outputItem.end.diff(outputItem.start))
+			var excessDuration = moment.duration(avgDuration.asMilliseconds()).subtract(actualDuration)
+			excessDuration = moment.duration(Math.max(excessDuration.asMilliseconds(), avgDuration.asMilliseconds() * 0.25))
+
+			// add a "tail" item after the end
+			outputObject.items.push({
+				id: outputItem.id + "-tail",
+				className: [outputItem.className, "visc-right-tail"].join(' '),
+				content: item.label ? "&nbsp;" : "",
+				start: outputItem.end.format("YYYYYY-MM-DDThh:mm:ss"),
+				end: outputItem.end.clone().add(excessDuration).format("YYYYYY-MM-DDThh:mm:ss"), //HACK: magic number
+				group: item.group,
+				subgroup: outputItem.subgroup
+			})
+
+			outputItem.className = [ outputItem.className, 'visc-right-connection' ].join(' ')
+		}
+		
+		// handle start date
+		if (start_min && start_max && start_max > start_min)
+		{
+			// uncertain start
+			
+			// find upper bound of uncertain region
+			var uncertainMax
+			if (item.start_max)
+				uncertainMax = start_max
+			else if (outputItem.start)
+				uncertainMax = outputItem.start
+			else
+				uncertainMax = outputItem.end //TODO: use min/max end
+			assert(uncertainMax)
+
+			// add uncertain range
+			outputObject.items.push({
+				id: outputItem.id + "-unc-start",
+				className: [outputItem.className, "visc-uncertain", "visc-right-connection"].join(' '),
+				content: item.label ? "&nbsp;" : "",
+				start: start_min.format("YYYYYY-MM-DDThh:mm:ss"),
+				end: uncertainMax.format("YYYYYY-MM-DDThh:mm:ss"),
+				group: item.group,
+				subgroup: outputItem.subgroup
+			})
+
+			// adjust normal range to match
+			outputItem.start = uncertainMax
+			outputItem.className = [ outputItem.className, 'visc-left-connection' ].join(' ')
+		}
+		else if (!outputItem.start)
+		{
+			// open-ended start
+			outputItem.start = outputItem.end.clone().subtract(expectation.duration.avg)
+			outputItem.className = [outputItem.className, "visc-open-left"].join(' ')
+		}
+		
+		//TODO: missing death dates inside expected duration: solid to NOW, fade after NOW
+		//TODO: accept expected durations and place uncertainly before/after those
+
+		finalizeItem(outputItem)
+	}
+	return JSON.stringify(outputObject, undefined, "\t") //TODO: configure space
+}
+
+async function entryPoint() {}
+
+entryPoint()
+.then(async () => {
+
+	await wikidata.readInputSpec(path.join(process.cwd(), specFile));
+
+})
+.then(async () => {
+
+	await wikidata.readCache();
+
+})
+.then(async () => {
+
+	// replace template items using their item-generating queries
+	for (var i = wikidata.inputSpec.items.length - 1; i >= 0; --i)
+	{
+		var templateItem = wikidata.inputSpec.items[i]
+		if (templateItem.itemQuery || templateItem.items)
+		{
+			//TODO: caching for item queries
+			wikidata.inputSpec.items.splice(i, 1)
+			const newItems = await wikidata.createTemplateItems(templateItem)
+			for (const newItem of newItems)
+				wikidata.inputSpec.items.push(newItem)
+		}
+	}
+
+})
+.then(() => {
+
+	var hasError = false;
+
+	// collect all existing ids
+	var itemIds = new Set()
+	for (const item of wikidata.inputSpec.items)
+	{
+		if (item.id)
+		{
+			if (itemIds.has(item.id))
+			{
+				itemIds.add(item.id)
+			}
+			else
+			{
+				console.error(`Item id '${item.id}' appears multiple times.`)
+				hasError = true
+			}
+		}
+	}
+
+	// generate ids for items that don't have one
+	for (const item of wikidata.inputSpec.items)
+	{
+		if (!item.id)
+		{
+			const baseId = item.entity ? item.entity : "anonymous"
+			var prospectiveId = baseId
+			var i = 0
+			while (itemIds.has(prospectiveId))
+			{
+				i++
+				prospectiveId = baseId + "-" + i
+			}
+			item.id = prospectiveId
+			itemIds.add(prospectiveId)
+		}
+	}
+
+	if (hasError) throw "Error generating item ids."
+
+})
+.then(() => {
+	
+	// batch up queries using templates
+	//TODO:
+
+})
+.then(async () => {
+
+	// run all the Wikidata queries
+
+	const isQueryProperty = function(key)
+	{
+		if (key == "startEndQuery" || key == "startQuery" || key == "endQuery") return true
+
+		//TODO: look at the queries for parameter names
+		return key != "comment"
+			&& key != "id"
+			&& key != "content"
+			&& key != "group"
+			&& key != "subgroup"
+			&& key != "itemQuery"
+			&& key != "type"
+			&& key != "label"
+			&& key != "className"
+			&& key != "entity"
+			&& key != "skipCache"
+	}
+
+	// bundle items that use the same queries
+	const queryBundles = {}
+	for (const item of wikidata.inputSpec.items)
+	{
+		if (item.finished) continue
+
+		// the bundle key is the queries, as well as any wildcard parameters
+		const keyObject = {}
+		for (const itemKey in item)
+		{
+			if (isQueryProperty(itemKey))
+			{
+				keyObject[itemKey] = item[itemKey]
+			}
+		}
+
+		const keyStr = JSON.stringify(keyObject)
+		var targetBundle = queryBundles[keyStr]
+		if (targetBundle)
+		{
+			targetBundle.push(item)
+		}
+		else
+		{
+			targetBundle = [ item ]
+			queryBundles[keyStr] = targetBundle
+		}
+	}
+
+	console.log(`There are ${Object.keys(queryBundles).length} query bundles.`)
+	for (const bundleKey in queryBundles)
+	{
+		const bundle = queryBundles[bundleKey]
+		console.log(`\tBundle (${bundle.length} items): ${bundleKey}.`)
+
+		const representativeItem = bundle[0]
+
+		const copyResult = function(result, func)
+		{
+			for (const entityId in result)
+			{
+				var entityResult = result[entityId]
+
+				// there may be multiple source items making the same query
+				for (const item of bundle)
+				{
+					if (item.entity == entityId)
+					{
+						if (entityResult instanceof Array)
+						{
+							// clone the item for each result beyond the first
+							for (var i = 1; i < entityResult.length; ++i)
+							{
+								const newItem = structuredClone(item)
+								newItem.id = `${newItem.id}-v${i}`
+								wikidata.inputSpec.items.push(newItem) //HACK: modifying original array
+								func(newItem, entityResult[i])
+								newItem.finished = true
+							}
+
+							// populate the first result into the original item
+							entityResult = entityResult[0]
+						}
+						
+						func(item, entityResult)
+						item.finished = true
+					}
+				}
+			}
+		}
+
+		if (representativeItem.startEndQuery)
+		{
+			const result = await wikidata.runTimeQueryTerm(representativeItem.startEndQuery, bundle)
+			copyResult(result, function(item, result) {
+				Object.assign(item, result)
+			})
+		}
+		else
+		{
+			if (representativeItem.startQuery)
+			{
+				const result = await wikidata.runTimeQueryTerm(representativeItem.startQuery, bundle)
+				copyResult(result, function(item, result) {
+					item.start = result.value;
+					item.start_min = result.min;
+					item.start_max = result.max;
+				})
+			}
+			if (representativeItem.endQuery)
+			{
+				const result = await wikidata.runTimeQueryTerm(representativeItem.endQuery, bundle)
+				copyResult(result, function(item, result) {
+					item.end = result.value;
+					item.end_min = result.min;
+					item.end_max = result.max;
+				})
+			}
+		}
+	}
+})
+.then(async () => {
+
+	await mypath.ensureDirectoryForFile(outputFile)
+
+	// write the output file
+	const output = produceOutput(wikidata.inputSpec.items)
+	await fs.writeFile(outputFile, output, err => {
+		if (err) {
+			console.error(`Error writing output file:`)
+			console.error(err)
+		}
+	});
+
+})
+.catch((reason) => {
+
+	console.error(reason)
+
+})
+.finally(async () => { await wikidata.writeCache(); })

package/src/global-data.json

@@ -0,0 +1,52 @@
+{
+	"queryTemplates":
+	{
+		"dateOfBirth": "{entity} p:P569 ?_prop. ?_prop psv:P569 ?_value.",
+		"dateOfDeath": "{entity} p:P570 ?_prop. ?_prop psv:P570 ?_value.",
+
+		"inception": "{entity} p:P571 ?_prop. ?_prop psv:P571 ?_value.",
+		"dissolved": "{entity} p:P576 ?_prop. ?_prop psv:P576 ?_value.",
+
+		"startTime": "{entity} p:P580 ?_prop. ?_prop psv:P580 ?_value.",
+		"endTime": "{entity} p:P582 ?_prop. ?_prop psv:P582 ?_value.",
+
+		"pointInTime": "{entity} p:P585 ?_prop. ?_prop psv:P585 ?_value.",
+
+		"positionHeldStartEnd": {
+			"general": "{entity} p:P39 ?_prop. ?_prop ps:P39 {position}.",
+			"start": "?_prop pqv:P580 ?_start_value.",
+			"start_min": "?_prop pqv:P1319 ?_start_min_value.",
+			"start_max": "?_prop pqv:P8555 ?_start_max_value.",
+			"end": "?_prop pqv:P582 ?_end_value.",
+			"end_min": "?_prop pqv:P8554 ?_end_min_value.",
+			"end_max": "?_prop pqv:P12506 ?_end_max_value."
+		},
+		"positionJurisdictionHeldStartEnd": {
+			"general": "{entity} p:P39 ?_prop. ?_prop ps:P39 {position}; pq:P1001 {jurisdiction}.",
+			"start": "?_prop pqv:P580 ?_start_value.",
+			"start_min": "?_prop pqv:P1319 ?_start_min_value.",
+			"start_max": "?_prop pqv:P8555 ?_start_max_value.",
+			"end": "?_prop pqv:P582 ?_end_value.",
+			"end_min": "?_prop pqv:P8554 ?_end_min_value.",
+			"end_max": "?_prop pqv:P12506 ?_end_max_value."
+		}
+	},
+	"itemQueryTemplates":
+	{
+		"withPosition": "?_node wdt:P39 {position}.",
+
+		"humansWithPosition": "?_node wdt:P39 {position}. ?_node wdt:P31 wd:Q5.",
+		"humansWithPositionJurisdiction": "?_node p:P39 [ ps:P39 {position}; pq:P1001 {jurisdiction} ]. ?_node wdt:P31 wd:Q5."
+	},
+	"expectations":
+	[
+		{
+			"startQuery": "#dateOfBirth",
+			"endQuery": "#dateOfDeath",
+			"duration": { "avg": "P80Y", "max": "P150Y" }
+		},
+		{
+			"duration": { "avg": "P100Y" }
+		}
+	]
+}

package/src/mypath.js

@@ -0,0 +1,19 @@
+
+const path = require('path')
+const fs = require('fs');
+
+module.exports = {
+	
+	ensureDirectoryForFile: async function(filePath)
+	{
+		const directoryName = path.dirname(filePath);
+		await fs.mkdir(directoryName, { recursive: true }, (err) => {
+			if (err)
+			{
+				console.error(`Error creating output directory: ${err}`);
+				throw err;
+			}
+		});
+	}
+	
+}

package/src/sparql-builder.js

@@ -0,0 +1,73 @@
+
+const assert = require('node:assert/strict')
+
+module.exports = class SparqlBuilder
+{
+	constructor()
+	{
+		this.outParams = []
+		this.queryTerms = []
+	}
+
+	// Adds an output parameter to the query
+	addOutParam(paramName)
+	{
+		//TODO: validate name more
+		assert(paramName)
+		if (this.outParams.indexOf(paramName) < 0)
+		{
+			this.outParams.push(paramName)
+		}
+	}
+
+	addQueryTerm(term)
+	{
+		assert(term)
+		this.queryTerms.push(term)
+	}
+
+	addWikibaseLabel(lang)
+	{
+		if (!lang) lang = "mul"
+		this.addQueryTerm(`SERVICE wikibase:label{bd:serviceParam wikibase:language "${lang}".}`)
+	}
+
+	addOptionalQueryTerm(term)
+	{
+		assert(term)
+		this.queryTerms.push(`OPTIONAL{${term}}`)
+	}
+
+	addTimeTerm(term, valueVar, timeVar, precisionVar)
+	{
+		assert(term)
+		
+		this.addOutParam(timeVar)
+		this.addOutParam(precisionVar)
+		this.addQueryTerm(`${term} ${valueVar} wikibase:timeValue ${timeVar}. ${valueVar} wikibase:timePrecision ${precisionVar}.`)
+	}
+
+	addOptionalTimeTerm(term, valueVar, timeVar, precisionVar)
+	{
+		assert(term)
+		
+		this.addOutParam(timeVar)
+		this.addOutParam(precisionVar)
+		this.addOptionalQueryTerm(`${term} ${valueVar} wikibase:timeValue ${timeVar}. ${valueVar} wikibase:timePrecision ${precisionVar}.`)
+	}
+
+	addCacheBuster(str)
+	{
+		if (str)
+			this.addQueryTerm(`#${str}\n`)
+	}
+
+	build()
+	{
+		assert(this.outParams.length > 0)
+		assert(this.queryTerms.length > 0)
+
+		//TODO: prevent injection
+		return `SELECT ${this.outParams.join(" ")} WHERE{${this.queryTerms.join(" ")}}`
+	}
+}

package/src/wikidata.js

@@ -0,0 +1,484 @@
+
+const mypath = require("./mypath.js")
+const fs = require('fs');
+const nodepath = require('node:path');
+const globalData = require("./global-data.json")
+const assert = require('node:assert/strict')
+const SparqlBuilder = require("./sparql-builder.js")
+
+const wikidata = module.exports = {
+
+	inputSpec: null,
+	verboseLogging: false,
+
+	cache: {},
+	skipCache: false,
+	cacheBuster: undefined,
+	termCacheFile: "intermediate/wikidata-term-cache.json",
+
+	sparqlUrl: "https://query.wikidata.org/sparql",
+	lang: "en,mul",
+
+	rankDeprecated: "http://wikiba.se/ontology#DeprecatedRank",
+	rankNormal: "http://wikiba.se/ontology#NormalRank",
+	rankPreferred: "http://wikiba.se/ontology#PreferredRank",
+
+	initialize: function()
+	{
+		const chroniclePackage = require("../package.json")
+		this.options = {
+			method: 'POST',
+			headers: {
+				'User-Agent': `vis-chronicle/${chroniclePackage.version} (https://github.com/BrianMacIntosh/vis-chronicle) Node.js/${process.version}`,
+				'Content-Type': 'application/sparql-query',
+				'Accept': 'application/sparql-results+json'
+			}
+		}
+	},
+
+	setLang: function(inLang)
+	{
+		//TODO: escape
+		this.lang = inLang
+	},
+
+	readInputSpec: async function(path)
+	{
+		const pathStat = await fs.promises.stat(path)
+		if (pathStat.isDirectory())
+		{
+			// read multi-file config
+			this.inputSpec = {}
+
+			const files = await fs.promises.readdir(path)
+			for (const fileName of files)
+			{
+				if (fileName.endsWith(".json"))
+				{
+					const contents = await fs.promises.readFile(nodepath.join(path, fileName))
+					const specPart = JSON.parse(contents)
+					for (const key in specPart)
+					{
+						if (specPart[key] instanceof Array)
+						{
+							if (this.inputSpec[key])
+							{
+								this.inputSpec[key].push(...specPart[key])
+							}
+							else
+							{
+								this.inputSpec[key] = specPart[key]
+							}
+						}
+						else if (this.inputSpec[key])
+						{
+							throw `Key '${key}' appears in multiple input spec files.`
+						}
+						else
+						{
+							this.inputSpec[key] = specPart[key]
+						}
+					}
+				}
+			}
+		}
+		else
+		{
+			// read single-file config
+			const contents = await fs.promises.readFile(path)
+			this.inputSpec = JSON.parse(contents)
+		}
+	},
+
+	readCache: async function()
+	{
+		try
+		{
+			const contents = await fs.promises.readFile(this.termCacheFile)
+			this.cache = JSON.parse(contents)
+		}
+		catch
+		{
+			// cache doesn't exist or is invalid; continue without it
+		}
+	},
+
+	writeCache: async function()
+	{
+		await mypath.ensureDirectoryForFile(this.termCacheFile)
+
+		fs.writeFile(this.termCacheFile, JSON.stringify(this.cache), err => {
+			if (err) {
+				console.error(`Error writing wikidata cache:`)
+				console.error(err)
+			}
+		})
+	},
+
+	getQueryTemplate: function(templateName, templateSetName)
+	{
+		assert(templateName)
+		assert(templateSetName)
+		if (this.inputSpec[templateSetName] && this.inputSpec[templateSetName][templateName])
+		{
+			return this.inputSpec[templateSetName][templateName]
+		}
+		else if (globalData && globalData[templateSetName] && globalData[templateSetName][templateName])
+		{
+			return globalData[templateSetName][templateName]
+		}
+		else
+		{
+			return undefined
+		}
+	},
+
+	postprocessQueryTerm: function(context, term, item)
+	{
+		if (!term)
+		{
+			return term;
+		}
+
+		// replace query wildcards
+		for (const key in item)
+		{
+			var insertValue = item[key]
+			if (typeof insertValue === "string" && insertValue.startsWith("Q"))
+				insertValue = "wd:" + insertValue
+			term = term.replaceAll(`{${key}}`, insertValue)
+		}
+
+		// detect unreplaced wildcards
+		//TODO:
+
+		// terminate term
+		if (!term.trim().endsWith("."))
+		{
+			term += "."
+		}
+
+		return term
+	},
+
+	// Dereferences the query term if it's a pointer to a template.
+	// Expects simple string terms (start or end)
+	getQueryTermHelper: function(inQueryTerm, item, tempateSetName)
+	{
+		var queryTerm
+
+		if (inQueryTerm.startsWith("#"))
+		{
+			const templateName = inQueryTerm.substring(1).trim()
+			var queryTemplate = this.getQueryTemplate(templateName, tempateSetName);
+			if (queryTemplate)
+			{
+				queryTerm = queryTemplate
+			}
+			else
+			{
+				throw `Query template '${templateName}' not found (on item ${item.id}).`
+			}
+		}
+		else
+		{
+			queryTerm = inQueryTerm
+		}
+
+		//TODO: validate query has required wildcards
+		
+		queryTerm = this.postprocessQueryTerm(inQueryTerm, queryTerm, item)
+		return queryTerm
+	},
+
+	// Dereferences the query term if it's a pointer to a template.
+	// Expects item-generating terms
+	getItemQueryTerm: function(queryTerm, item)
+	{
+		return this.getQueryTermHelper(queryTerm, item, "itemQueryTemplates")
+	},
+
+	// Dereferences the query term if it's a pointer to a template.
+	getValueQueryTerm: function(inQueryTerm, item)
+	{
+		var queryTerm
+
+		if (inQueryTerm.startsWith && inQueryTerm.startsWith("#"))
+		{
+			const templateName = inQueryTerm.substring(1).trim()
+			var queryTemplate = this.getQueryTemplate(templateName, "queryTemplates");
+			if (queryTemplate)
+			{
+				queryTerm = queryTemplate
+			}
+			else
+			{
+				throw `Query template '${templateName}' not found (on item ${item.id}).`
+			}
+		}
+		else
+		{
+			queryTerm = inQueryTerm
+		}
+
+		if (typeof queryTerm === 'string' || queryTerm instanceof String)
+		{
+			return {
+				value: this.postprocessQueryTerm(inQueryTerm, queryTerm, item),
+				min: "?_prop pqv:P1319 ?_min_value.",
+				max: "?_prop pqv:P1326 ?_max_value."
+			}
+		}
+		else
+		{
+			const result = {}
+			for (const key in queryTerm)
+			{
+				result[key] = this.postprocessQueryTerm(inQueryTerm, queryTerm[key], item)
+			}
+			return result
+		}
+	},
+
+	extractQidFromUrl: function(url)
+	{
+		const lastSlashIndex = url.lastIndexOf("/")
+		if (lastSlashIndex >= 0)
+			return url.substring(lastSlashIndex + 1)
+		else
+			return url
+	},
+
+	// Sequential list of filters to narrow down a list of bindings to the best result
+	bindingFilters: [
+		(binding, index, array) => {
+			return binding._rank.value != wikidata.rankDeprecated;
+		},
+		(binding, index, array) => {
+			return binding._rank.value === wikidata.rankPreferred;
+		},
+	],
+
+	// From the specified set of bindings, returns only the best ones to use
+	filterBestBindings: function(inBindings)
+	{
+		// filter the values down until there are none
+		var lastBindings = inBindings.slice()
+		for (const filter of this.bindingFilters)
+		{
+			workingBindings = lastBindings.filter(filter)
+			if (workingBindings.length == 0)
+			{
+				break
+			}
+			lastBindings = workingBindings
+		}
+		return lastBindings
+	},
+
+	// runs a SPARQL query for time values on an item or set of items
+	runTimeQueryTerm: async function (queryTermStr, items)
+	{
+		const entityVarName = '_entity'
+		const entityVar = `?${entityVarName}`
+		const propVar = '?_prop'
+		const rankVar = '?_rank'
+
+		const queryBuilder = new SparqlBuilder()
+		queryBuilder.addCacheBuster(this.cacheBuster)
+
+		// create a dummy item representing the collective items
+		//TODO: validate that they match
+		item = { ...items[0] }
+		item.entity = entityVar
+		item.id = "DUMMY"
+
+		queryTerm = this.getValueQueryTerm(queryTermStr, item)
+		
+		// assembly query targets
+		const targetEntities = new Set()
+		for (const item of items)
+		{
+			targetEntities.add(`wd:${item.entity}`)
+		}
+		queryBuilder.addQueryTerm(`VALUES ${entityVar}{${[...targetEntities].join(' ')}}`)
+		queryBuilder.addOutParam(entityVar)
+
+		queryBuilder.addOutParam(rankVar)
+		if (queryTerm.general)
+		{
+			queryBuilder.addQueryTerm(queryTerm.general)
+		}
+		if (queryTerm.value) //TODO: could unify better with loop below?
+		{
+			queryBuilder.addTimeTerm(queryTerm.value, "?_value", "?_value_ti", "?_value_pr")
+		}
+		for (const termKey in queryTerm) //TODO: only pass recognized terms
+		{
+			if (termKey == "general" || termKey == "value") continue
+			queryBuilder.addOptionalTimeTerm(queryTerm[termKey], `?_${termKey}_value`, `?_${termKey}_ti`, `?_${termKey}_pr`)
+		}
+		queryBuilder.addOptionalQueryTerm(`${propVar} wikibase:rank ${rankVar}.`)
+
+		const query = queryBuilder.build()
+
+		// read cache
+		const cacheKey = query
+		if (!this.skipCache && !item.skipCache && this.cache[cacheKey])
+		{
+			return this.cache[cacheKey]
+		}
+		
+		const data = await this.runQuery(query)
+		console.log(`\tQuery for ${item.id} returned ${data.results.bindings.length} results.`)
+
+		const readBinding = function(binding)
+		{
+			const result = {}
+			for (const termKey in queryTerm) //TODO: only use recognized terms
+			{
+				if (binding[`_${termKey}_ti`])
+				{
+					result[termKey] = {
+						value: binding[`_${termKey}_ti`].value,
+						precision: parseInt(binding[`_${termKey}_pr`].value)
+					}
+				}
+			}
+			return result
+		}
+
+		const readBindings = function(bindings)
+		{
+			const results = []
+			for (const binding of bindings)
+			{
+				results.push(readBinding(binding))
+			}
+			return results
+		}
+
+		// arrays of bindings, grouped by entity id
+		const bindingsByEntity = {}
+
+		// sort out the bindings by entity
+		for (const binding of data.results.bindings)
+		{
+			// read entity id
+			assert(binding[entityVarName].type == 'uri')
+			const entityId = this.extractQidFromUrl(binding[entityVarName].value)
+
+			// get array for entity-specific results
+			var entityBindings = bindingsByEntity[entityId]
+			if (!entityBindings)
+			{
+				entityBindings = []
+				bindingsByEntity[entityId] = entityBindings
+			}
+
+			entityBindings.push(binding)
+		}
+
+		// filter results down to best per entity
+		const result = {}
+		for (const entityId in bindingsByEntity)
+		{
+			const entityBindings = bindingsByEntity[entityId]
+			if (entityBindings.length == 1)
+			{
+				result[entityId] = readBinding(entityBindings[0])
+			}
+			else // entityBindings.length > 1
+			{
+				var lastBindings = this.filterBestBindings(entityBindings)
+				result[entityId] = readBindings(lastBindings)
+			}
+		}
+
+		this.cache[cacheKey] = result;
+		return result;
+	},
+
+	// runs a SPARQL query that generates multiple items
+	createTemplateItems: async function(templateItem)
+	{
+		//TODO: caching
+
+		const itemVarName = "_node"
+		const itemVar = `?${itemVarName}`
+		templateItem.entity = itemVar
+
+		var itemQueryTerm
+		if (templateItem.itemQuery)
+		{
+			if (templateItem.items)
+			{
+				console.warn(`Template item '${templateItem.comment}' has both 'itemQuery' and 'items': 'items' will be ignored.`)
+			}
+			itemQueryTerm = this.getItemQueryTerm(templateItem.itemQuery, templateItem)
+		}
+		else if (templateItem.items)
+		{
+			itemQueryTerm = `VALUES ${itemVar}{${templateItem.items.map(id => `wd:${id}` ).join(' ')}}`
+		}
+		else
+		{
+			console.error(`Template item '${templateItem.comment}' has no 'itemQuery' or 'items'.`)
+			return []
+		}
+
+		const queryBuilder = new SparqlBuilder()
+		queryBuilder.addCacheBuster(this.cacheBuster)
+		queryBuilder.addOutParam(itemVar)
+		queryBuilder.addOutParam(itemVar + "Label")
+		queryBuilder.addQueryTerm(itemQueryTerm)
+		queryBuilder.addWikibaseLabel(this.lang)
+		
+		const query = queryBuilder.build()
+		const data = await this.runQuery(query)
+
+		const newItems = []
+
+		// clone the template for each result
+		for (const binding of data.results.bindings)
+		{
+			const newItem = structuredClone(templateItem)
+			delete newItem.comment
+			delete newItem.itemQuery
+			delete newItem.items
+			newItem.entity = this.extractQidFromUrl(binding[itemVarName].value)
+			newItem.generated = true
+			const wikidataLabel = binding[itemVarName + "Label"].value
+			newItem.label = templateItem.label
+				? templateItem.label.replaceAll("{_LABEL}", wikidataLabel).replaceAll("{_QID}", newItem.entity)
+				: `<a target="_blank" href="https://www.wikidata.org/wiki/${newItem.entity}">${wikidataLabel}</a>`
+			newItems.push(newItem)
+		}
+
+		templateItem.finished = true
+		console.log(`Item template '${templateItem.comment}' created ${newItems.length} items.`)
+
+		return newItems;
+	},
+
+	// runs a SPARQL query
+	runQuery: async function(query)
+	{
+		if (this.verboseLogging) console.log(query)
+
+		assert(this.options)
+		const options = { ...this.options, body: query }
+		const response = await fetch(this.sparqlUrl, options)
+		if (response.status != 200)
+		{
+			console.log(response)
+			console.error(`${response.status}: ${response.statusText}`)
+			return null
+		}
+		else
+		{
+			const data = await response.json()
+			return data
+		}
+	}
+} 

package/styles/style.css

@@ -0,0 +1,40 @@
+.vis-item.vis-range.visc-open-right
+{
+	mask-image: linear-gradient(to right, rgba(0, 0, 0, 1) 50%, rgba(0, 0, 0, 0));
+}
+.vis-item.vis-range.visc-open-left
+{
+	mask-image: linear-gradient(to right, rgba(0, 0, 0, 0), rgba(0, 0, 0, 1) 50%);
+}
+.vis-item.vis-range.visc-open-both
+{
+	mask-image: linear-gradient(to right, rgba(0, 0, 0, 0), rgba(0, 0, 0, 1) 50%, rgba(0, 0, 0, 0));
+}
+.visc-uncertain
+{
+	background: repeating-linear-gradient(45deg, #00000060, #00000060 8px, transparent 8px, transparent 16px)
+}
+.vis-item.vis-range.visc-right-connection, .vis-item.vis-range.visc-left-tail
+{
+	border-right-style: none;
+	border-top-right-radius: 0;
+	border-bottom-right-radius: 0;
+}
+.vis-item.vis-range.visc-right-tail
+{
+	mask-image: linear-gradient(to right, rgba(0, 0, 0, 1), rgba(0, 0, 0, 0));
+}
+.vis-item.vis-range.visc-left-connection, .vis-item.vis-range.visc-right-tail
+{
+	border-left-style: none;
+	border-top-left-radius: 0;
+	border-bottom-left-radius: 0;
+}
+.vis-item.vis-range.visc-left-tail
+{
+	mask-image: linear-gradient(to right, rgba(0, 0, 0, 0), rgba(0, 0, 0, 1));
+}
+.vis-item-content a {
+  color: inherit;
+  text-decoration: inherit;
+}