@suss/datalog 0.10.0 → 0.11.0

package/README.md

@@ -76,6 +76,130 @@ you re-run with negated rules, the engine throws away what the previous
 pass derived and works the answer out again from the base facts. Either
 way, the database ends up with the answer for the facts it has now.
 
+## Tagging derivations
+
+A fact is normally there or not there. Pass `evaluate` a `TagAlgebra`
+and every derived fact also gets a tag: `combine` turns the body's
+tags into the head's, and `merge` decides what to keep when the same
+fact is derived a second time. This is a provenance semiring in the
+sense of Green, Karvounarakis, and Tannen (2007); Soufflé's provenance
+mode stores a minimal proof height with the same two operations. Rules
+never mention tags, and a call without an algebra runs as before.
+
+```ts
+const cost: TagAlgebra<number> = {
+  asserted: 0, // what an untagged fact contributes
+  absent: 0, // what a matched negated literal contributes
+  combine: (tags) => tags.reduce((sum, tag) => sum + tag, 1),
+  merge: (stored, incoming) => (incoming < stored ? incoming : stored),
+};
+
+evaluate(db, rules, cost);
+db.tagOf("reaches", ["a", "c"]); // the cheapest derivation's cost
+```
+
+`combine` is also handed the derivation itself: the `Rule` that fired,
+and one entry per body literal saying which fact it matched, or which
+grounded tuple a negated literal checked and did not find. An algebra
+that only folds tags, like the cost above, ignores it; the witness
+algebra below is built out of it.
+
+Tags are stored beside the relation, keyed the same way as the facts,
+never in the tuple. A tuple is its own identity: were the tag part of it, an
+improved tag would read as a new fact and the fixpoint would never
+close. `Database.add` reports "added", "improved", or "unchanged", and
+an improved fact re-enters the delta so conclusions built on it
+recompute with the better tag.
+
+The contract on `merge`:
+
+- Return the stored tag itself, the same value by `===`, when the new
+  derivation does not improve on it. Anything else is stored and
+  re-derives downstream.
+- For recursive rule sets, merge must be a bounded meet: repeated
+  merges must stop improving after finitely many steps, the way `min`
+  over numbers does. The engine does not check this; an algebra that
+  keeps improving keeps evaluating.
+- `undefined` is not a valid tag. The store uses it to mean "no tag",
+  and `tagOf` returns it for untagged facts.
+
+Two caveats. Supply the same algebra on every evaluation of a rule set
+over a database: a resumed run derives only from facts added since, so
+it tags only what those reach. And rules rewritten by `deriveOnDemand`
+refuse an algebra: tags would follow the demand-transformed rules
+rather than the ones you wrote, so `evaluate` throws instead.
+
+## Witnesses, and proofs on demand
+
+The `witnesses` algebra keeps one derivation per derived fact: the rule
+that fired and one entry per body literal. Its merge keeps whatever is
+already stored, so no fact ever re-enters the delta, the fixpoint
+behaves exactly as it does untagged, and the tag stays one small object
+per fact. `proofOf` then rebuilds the tree for one fact by walking
+those witnesses backward, without re-running any rule:
+
+```ts
+import { evaluate, proofOf, ruleLabel, witnesses } from "@suss/datalog";
+
+evaluate(db, rules, witnesses);
+const proof = proofOf(db, "reachable", ["util"]);
+// {
+//   kind: "derived",
+//   rule: <the Rule that fired>,
+//   premises: [
+//     { kind: "derived", ... },              // walked further down
+//     { kind: "fact", relation: "calls", tuple: ["helper", "util"] },
+//   ],
+// }
+```
+
+A proof node is one of four kinds. `derived` has the rule and one
+premise per body literal, in rule-body order. `fact` is a leaf with no
+witness: the caller asserted it, or it was derived without the
+algebra. `absence` is a leaf for a tuple missing from the database, at
+the root when you ask about a fact that was never derived, and under a
+derivation where a negated literal relied on the tuple being missing;
+that absence is recorded at derivation time, so a proof can say "and no
+handles(x) existed" without checking anything again. `truncated` is
+where the walk stopped, at the depth cap (`maxDepth`, 128 unless you
+say otherwise) or at a cycle.
+
+First-wins gives you *a* proof, not the shortest one. When nine
+derivations reach the same fact, the one stored is the one evaluation
+found first. A different merge, say keeping the witness with the
+smaller proof height, fits the same algebra interface without touching
+the walk.
+
+A proof shows each rule as `ruleLabel` renders it: the `name` you
+passed as `rule`'s fourth argument, or the head and body relations, as
+in `path :- path, !blocked`. The witness keeps the `Rule` object
+itself, so two rules that render alike stay distinct.
+
+## Confidence levels
+
+The `confidence` algebra propagates how sure the run is of each fact,
+as one of `"high"`, `"medium"`, or `"low"`. A rule firing takes the
+minimum level across its body, so a conclusion is only as sure as its
+weakest premise, and a fact derived twice keeps the better level. An
+asserted fact without a tag counts as high; to say less, assert it
+with a level:
+
+```ts
+import { confidence, evaluate } from "@suss/datalog";
+
+db.add("edge", ["a", "b"]);            // high: read from source
+db.add("edge", ["b", "c"], "medium");  // a guess somebody should check
+evaluate(db, rules, confidence);
+db.tagOf("path", ["a", "c"]); // "medium"
+```
+
+A rule can be a heuristic itself, and then its conclusions should not
+outrank it. `confidenceWith` takes a level per rule and folds it into
+the minimum; rules the callback returns `undefined` for count as
+exact. Both operations are idempotent, so ten medium steps come out
+medium, and a matched negation counts as high, because negation here
+is exact over the database as computed.
+
 ## Deriving only what somebody asked for
 
 A rule set written for a whole program derives every conclusion its