tiny-datalog 0.4.0__tar.gz → 0.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (24) hide show
  1. {tiny_datalog-0.4.0/tiny_datalog.egg-info → tiny_datalog-0.5.0}/PKG-INFO +19 -13
  2. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/README.md +18 -12
  3. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/pyproject.toml +2 -1
  4. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/__init__.py +3 -2
  5. tiny_datalog-0.5.0/tiny_datalog/chase.py +397 -0
  6. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/core.py +2 -1
  7. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/datalog.py +4 -1
  8. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/engines.py +4 -3
  9. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0/tiny_datalog.egg-info}/PKG-INFO +19 -13
  10. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog.egg-info/SOURCES.txt +1 -0
  11. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog.egg-info/entry_points.txt +1 -0
  12. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/LICENSE +0 -0
  13. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/setup.cfg +0 -0
  14. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/containment.py +0 -0
  15. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/defeasible.py +0 -0
  16. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/incremental.py +0 -0
  17. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/magic.py +0 -0
  18. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/prolog.py +0 -0
  19. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/semantics.py +0 -0
  20. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/semiring.py +0 -0
  21. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/subsumption.py +0 -0
  22. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog/tabling.py +0 -0
  23. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog.egg-info/dependency_links.txt +0 -0
  24. {tiny_datalog-0.4.0 → tiny_datalog-0.5.0}/tiny_datalog.egg-info/top_level.txt +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: tiny-datalog
3
- Version: 0.4.0
3
+ Version: 0.5.0
4
4
  Summary: A Datalog engine whose core you can read in an afternoon, and a course built on it
5
5
  Author: Andrew Goodchild
6
6
  License-Expression: MIT
@@ -127,7 +127,7 @@ Nothing to install:
127
127
 
128
128
  ```sh
129
129
  git clone https://github.com/andrewgoodchild/tiny-datalog && cd tiny-datalog
130
- python3 tests.py # 303 tests, ~12s
130
+ python3 tests.py # 326 tests, ~12s
131
131
  ```
132
132
 
133
133
  ## Why the language choice decides what you can ask later
@@ -249,8 +249,12 @@ the current research threads. The field's own recent lecture notes
249
249
  observe that the literature advises people building Datalog engines
250
250
  better than people trying to *use* one; this course does both halves
251
251
  on purpose — sixteen lessons where the engine is the explanation, then
252
- a lesson on authoring rules that survive review, and one on a logic
253
- built for rules with exceptions. And it is built to be
252
+ a lesson on authoring rules that survive review, one on a logic
253
+ built for rules with exceptions, and three where the engine goes to
254
+ work: [authorization](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/20-authorization.md),
255
+ [program analysis](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/21-program-analysis.md),
256
+ and [knowledge graphs and the chase](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/22-knowledge-graphs.md).
257
+ And it is built to be
254
258
  inherited: `git clone`, no dependencies, no hosted anything, and every
255
259
  quoted transcript re-verified by CI — the exercises cannot rot. (For where each
256
260
  technique ships — CodeQL, RDFox, Feldera, SNOMED and the rest —
@@ -384,18 +388,20 @@ tiny_datalog/ the code you read, in three tiers:
384
388
  subsumption.py KL-ONE-style EL classifier, compiled to Datalog
385
389
  containment.py query containment and minimisation by homomorphism
386
390
  defeasible.py defeasible logic: exceptions, priorities, defeaters
391
+ chase.py existential rules: the chase, labelled nulls, weak
392
+ acyclicity
387
393
  *.py three-line launchers, so `python3 datalog.py ...` works
388
394
  straight from a checkout with nothing installed
389
395
  programs/ teaching programs, numbered by the lesson that uses
390
396
  them (00-* are the README's examples)
391
- lessons/ getting started, glossary, and lessons 0–19
397
+ lessons/ getting started, glossary, and lessons 0–22
392
398
  exercises/ worked answers, verified by the test suite
393
399
  cases/ golden test cases — add one without writing Python
394
400
  conformance/ the external datalog-conformance corpus (Soufflé, Nemo,
395
401
  Crepe, SPINdle), run against every evaluation strategy
396
402
  benchmarks/ scaled input generators (chain/tree/clique/grid, an EL
397
403
  ontology, and the N-queens puzzle)
398
- tests.py 303 tests: every shipped program and exercise answer is
404
+ tests.py 326 tests: every shipped program and exercise answer is
399
405
  executed, a conformance suite runs every query through
400
406
  every engine that handles it, and a seeded fuzzer checks
401
407
  the same property on random programs
@@ -412,7 +418,7 @@ This project started as an attempt at a tiny implementation of
412
418
  Datalog, small enough to read in an afternoon. It grew with the
413
419
  course. Each lesson that taught a new technique — magic sets, stable models,
414
420
  semirings, incremental maintenance, tabling, subsumption, containment,
415
- defeasible logic — brought a module to run it, and the repository is
421
+ defeasible logic, the chase — brought a module to run it, and the repository is
416
422
  now several thousand lines. The core idea stayed small, and the code
417
423
  is laid out so you can still read just that: the common core and the
418
424
  base engine, below, are the original tiny Datalog. Everything else is
@@ -424,19 +430,19 @@ Measured in lines, comments, docstrings and blank lines included:
424
430
  |---|---|
425
431
  | the common core (`core.py`: the language, evaluating nothing) | 785 |
426
432
  | the base engine's evaluator (`datalog.py` up to its CLI section: the module docstring, the import list that re-exports the core, `Program`, `Engine`, aggregation folding, `run_program`) | 395 |
427
- | `--explain`, why-not, printing and the CLI (the rest of `datalog.py`) | 535 |
433
+ | `--explain`, why-not, printing and the CLI (the rest of `datalog.py`) | 538 |
428
434
  | the other pluggable engines: magic sets (`magic.py`) and tabling (`tabling.py`), plus the `engines.py` registry (naive evaluation is a flag on the base engine) | 549 |
429
- | the seven extensions | 2,425 |
435
+ | the eight extensions | 2,822 |
430
436
 
431
437
  So the smallest complete Datalog system here — parse, check, stratify,
432
438
  evaluate semi-naively — is the core plus the evaluator, about 1,200
433
- lines. All of it is 4.7k lines of toolkit (every module but the
439
+ lines. All of it is 5.1k lines of toolkit (every module but the
434
440
  re-export list in `__init__.py`), roughly a quarter of it commentary,
435
- and 3.4k of tests.
441
+ and 3.6k of tests.
436
442
 
437
443
  "Tiny" is a claim about that core and evaluator, and about each module
438
- built on them singly: none of the ten (`magic.py`, `tabling.py`,
439
- `engines.py` and the seven extensions) exceeds 500 lines, which a test asserts. It is not a
444
+ built on them singly: none of the eleven (`magic.py`, `tabling.py`,
445
+ `engines.py` and the eight extensions) exceeds 500 lines, which a test asserts. It is not a
440
446
  claim about the repository, which is a dozen modules because it
441
447
  teaches a dozen things.
442
448
 
@@ -94,7 +94,7 @@ Nothing to install:
94
94
 
95
95
  ```sh
96
96
  git clone https://github.com/andrewgoodchild/tiny-datalog && cd tiny-datalog
97
- python3 tests.py # 303 tests, ~12s
97
+ python3 tests.py # 326 tests, ~12s
98
98
  ```
99
99
 
100
100
  ## Why the language choice decides what you can ask later
@@ -216,8 +216,12 @@ the current research threads. The field's own recent lecture notes
216
216
  observe that the literature advises people building Datalog engines
217
217
  better than people trying to *use* one; this course does both halves
218
218
  on purpose — sixteen lessons where the engine is the explanation, then
219
- a lesson on authoring rules that survive review, and one on a logic
220
- built for rules with exceptions. And it is built to be
219
+ a lesson on authoring rules that survive review, one on a logic
220
+ built for rules with exceptions, and three where the engine goes to
221
+ work: [authorization](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/20-authorization.md),
222
+ [program analysis](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/21-program-analysis.md),
223
+ and [knowledge graphs and the chase](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/22-knowledge-graphs.md).
224
+ And it is built to be
221
225
  inherited: `git clone`, no dependencies, no hosted anything, and every
222
226
  quoted transcript re-verified by CI — the exercises cannot rot. (For where each
223
227
  technique ships — CodeQL, RDFox, Feldera, SNOMED and the rest —
@@ -351,18 +355,20 @@ tiny_datalog/ the code you read, in three tiers:
351
355
  subsumption.py KL-ONE-style EL classifier, compiled to Datalog
352
356
  containment.py query containment and minimisation by homomorphism
353
357
  defeasible.py defeasible logic: exceptions, priorities, defeaters
358
+ chase.py existential rules: the chase, labelled nulls, weak
359
+ acyclicity
354
360
  *.py three-line launchers, so `python3 datalog.py ...` works
355
361
  straight from a checkout with nothing installed
356
362
  programs/ teaching programs, numbered by the lesson that uses
357
363
  them (00-* are the README's examples)
358
- lessons/ getting started, glossary, and lessons 0–19
364
+ lessons/ getting started, glossary, and lessons 0–22
359
365
  exercises/ worked answers, verified by the test suite
360
366
  cases/ golden test cases — add one without writing Python
361
367
  conformance/ the external datalog-conformance corpus (Soufflé, Nemo,
362
368
  Crepe, SPINdle), run against every evaluation strategy
363
369
  benchmarks/ scaled input generators (chain/tree/clique/grid, an EL
364
370
  ontology, and the N-queens puzzle)
365
- tests.py 303 tests: every shipped program and exercise answer is
371
+ tests.py 326 tests: every shipped program and exercise answer is
366
372
  executed, a conformance suite runs every query through
367
373
  every engine that handles it, and a seeded fuzzer checks
368
374
  the same property on random programs
@@ -379,7 +385,7 @@ This project started as an attempt at a tiny implementation of
379
385
  Datalog, small enough to read in an afternoon. It grew with the
380
386
  course. Each lesson that taught a new technique — magic sets, stable models,
381
387
  semirings, incremental maintenance, tabling, subsumption, containment,
382
- defeasible logic — brought a module to run it, and the repository is
388
+ defeasible logic, the chase — brought a module to run it, and the repository is
383
389
  now several thousand lines. The core idea stayed small, and the code
384
390
  is laid out so you can still read just that: the common core and the
385
391
  base engine, below, are the original tiny Datalog. Everything else is
@@ -391,19 +397,19 @@ Measured in lines, comments, docstrings and blank lines included:
391
397
  |---|---|
392
398
  | the common core (`core.py`: the language, evaluating nothing) | 785 |
393
399
  | the base engine's evaluator (`datalog.py` up to its CLI section: the module docstring, the import list that re-exports the core, `Program`, `Engine`, aggregation folding, `run_program`) | 395 |
394
- | `--explain`, why-not, printing and the CLI (the rest of `datalog.py`) | 535 |
400
+ | `--explain`, why-not, printing and the CLI (the rest of `datalog.py`) | 538 |
395
401
  | the other pluggable engines: magic sets (`magic.py`) and tabling (`tabling.py`), plus the `engines.py` registry (naive evaluation is a flag on the base engine) | 549 |
396
- | the seven extensions | 2,425 |
402
+ | the eight extensions | 2,822 |
397
403
 
398
404
  So the smallest complete Datalog system here — parse, check, stratify,
399
405
  evaluate semi-naively — is the core plus the evaluator, about 1,200
400
- lines. All of it is 4.7k lines of toolkit (every module but the
406
+ lines. All of it is 5.1k lines of toolkit (every module but the
401
407
  re-export list in `__init__.py`), roughly a quarter of it commentary,
402
- and 3.4k of tests.
408
+ and 3.6k of tests.
403
409
 
404
410
  "Tiny" is a claim about that core and evaluator, and about each module
405
- built on them singly: none of the ten (`magic.py`, `tabling.py`,
406
- `engines.py` and the seven extensions) exceeds 500 lines, which a test asserts. It is not a
411
+ built on them singly: none of the eleven (`magic.py`, `tabling.py`,
412
+ `engines.py` and the eight extensions) exceeds 500 lines, which a test asserts. It is not a
407
413
  claim about the repository, which is a dozen modules because it
408
414
  teaches a dozen things.
409
415
 
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "tiny-datalog"
7
- version = "0.4.0"
7
+ version = "0.5.0"
8
8
  description = "A Datalog engine whose core you can read in an afternoon, and a course built on it"
9
9
  readme = "README.md"
10
10
  license = "MIT"
@@ -52,6 +52,7 @@ tiny-datalog-incremental = "tiny_datalog.incremental:main"
52
52
  tiny-datalog-subsumption = "tiny_datalog.subsumption:main"
53
53
  tiny-datalog-containment = "tiny_datalog.containment:main"
54
54
  tiny-datalog-defeasible = "tiny_datalog.defeasible:main"
55
+ tiny-datalog-chase = "tiny_datalog.chase:main"
55
56
 
56
57
  [tool.setuptools.packages.find]
57
58
  include = ["tiny_datalog*"]
@@ -16,7 +16,8 @@ question of a program: `semantics` (stable and well-founded models),
16
16
  `semiring` (provenance-weighted evaluation), `incremental` (maintenance
17
17
  under updates), `subsumption` (KL-ONE style classification),
18
18
  `containment` (query containment), `defeasible` (rules with exceptions
19
- and priorities), and `prolog` (a Prolog reader for the same syntax).
19
+ and priorities), `chase` (existential rules and labelled nulls), and
20
+ `prolog` (a Prolog reader for the same syntax).
20
21
  Import those by name: `from tiny_datalog import semiring`.
21
22
  """
22
23
 
@@ -35,7 +36,7 @@ from tiny_datalog.datalog import (
35
36
  format_atom, format_fact,
36
37
  )
37
38
 
38
- __version__ = "0.4.0"
39
+ __version__ = "0.5.0"
39
40
 
40
41
  __all__ = [
41
42
  "Var", "Const", "Struct", "Atom", "Literal", "Rule",
@@ -0,0 +1,397 @@
1
+ #!/usr/bin/env python3
2
+ """
3
+ chase.py — existential rules and the chase. (Lesson 22.)
4
+
5
+ Datalog's safety condition says every head variable must be bound by
6
+ the body, and that is what keeps the universe of values closed: no rule
7
+ can mention anything the database did not already contain. An
8
+ **existential rule** (a tuple-generating dependency, in database
9
+ theory's name for it) drops exactly that condition:
10
+
11
+ works_in(E, D) :- employee(E). % every employee works in
12
+ % *some* department
13
+
14
+ D is bound by nothing, so it is read as "there exists a D". Nobody
15
+ knows which department, so the engine invents a placeholder for it —
16
+ a **labelled null**, printed `_:n1` (RDF writes the same idea as a blank
17
+ node, `_:b1`) — and carries on. That procedure is **the chase**, and
18
+ this module is it:
19
+
20
+ 1. saturate the ordinary rules, with the base engine from datalog.py;
21
+ 2. fire every existential rule whose body matches, inventing a fresh
22
+ null for each existential variable;
23
+ 3. repeat until a round invents nothing.
24
+
25
+ Two variants, chosen with --variant:
26
+
27
+ * **restricted** (the default): a trigger fires only if its head is not
28
+ already satisfied — if *some* department for alice is already known,
29
+ the rule has nothing to add. Ordinary rules run first in every
30
+ round (the "Datalog-first" order), so the check sees everything they
31
+ imply.
32
+ * **oblivious**: every trigger fires once, satisfied or not. Simpler,
33
+ and it invents nulls the restricted chase would not.
34
+
35
+ Neither is guaranteed to stop: inventing values is precisely what the
36
+ safety condition was there to prevent (lesson 11's boundary, crossed
37
+ from the database side). Whether the chase terminates is undecidable
38
+ in general, so the module does two things about it. Before running,
39
+ it checks **weak acyclicity** (Fagin, Kolaitis, Miller and Popa, 2005),
40
+ a static condition that *guarantees* termination: draw a graph over
41
+ argument positions, mark the edges along which a rule invents a null,
42
+ and look for a cycle through a marked edge — a graph problem in the
43
+ same spirit as stratification. And while running, it stops after
44
+ --max-nulls inventions and says which cycle is the likely culprit.
45
+
46
+ Answers that mention a null are reported, but are not **certain
47
+ answers**: they are true in every model only up to *which* value the
48
+ null stands for. `manages(_:n2, sales)` says sales has a manager, not
49
+ who.
50
+
51
+ Where this stops
52
+ ----------------
53
+ Positive rules only: no `not`, no aggregates, one atom per head (write
54
+ a second ordinary rule over the invented value where a conjunctive head
55
+ would be used). No equality-generating dependencies, so two nulls
56
+ known to be the same thing are never merged; lesson 22's sameAs
57
+ section is about why that is the hard part.
58
+
59
+ CLI
60
+ ---
61
+ python3 chase.py programs/org-chart.dl
62
+ python3 chase.py -q 'manages(M, D)' programs/org-chart.dl
63
+ python3 chase.py --variant oblivious programs/org-chart.dl
64
+ """
65
+
66
+ from __future__ import annotations
67
+
68
+ import argparse
69
+ import sys
70
+
71
+ from collections import defaultdict, deque
72
+ from dataclasses import dataclass
73
+
74
+ from tiny_datalog.core import (
75
+ _format_value, aggregate_of, Atom, cli, Const, DatalogError, Literal,
76
+ match, parse, parse_query_atom, read_program, Rule, Var, validate)
77
+ from tiny_datalog.datalog import Engine, Program
78
+
79
+
80
+ @dataclass(frozen=True)
81
+ class Null:
82
+ """A labelled null: a value the chase invented to witness an
83
+ existential. Two nulls are equal only if they are the same null."""
84
+ n: int
85
+
86
+ def __str__(self):
87
+ return "_:n%d" % self.n
88
+
89
+
90
+ def _format(v):
91
+ return str(v) if isinstance(v, Null) else _format_value(v)
92
+
93
+
94
+ def format_atom(pred, tup):
95
+ if not tup:
96
+ return pred
97
+ return "%s(%s)" % (pred, ", ".join(_format(v) for v in tup))
98
+
99
+
100
+ def _key(tup):
101
+ # constants as core.sort_key orders them, then nulls in invention order
102
+ return tuple((2, v.n, "") if isinstance(v, Null)
103
+ else (0, v, "") if isinstance(v, (int, float))
104
+ else (1, 0, str(v)) for v in tup)
105
+
106
+
107
+ def has_null(tup):
108
+ return any(isinstance(v, Null) for v in tup)
109
+
110
+
111
+ # ---------------------------------------------------------------------------
112
+ # The rules
113
+ # ---------------------------------------------------------------------------
114
+
115
+ def _body_vars(rule):
116
+ return {a.name for lit in rule.body for a in lit.atom.args
117
+ if isinstance(a, Var)}
118
+
119
+
120
+ def existentials(rule):
121
+ """The head variables no body literal binds, in head order."""
122
+ bound = _body_vars(rule)
123
+ seen = []
124
+ for a in rule.head.args:
125
+ if isinstance(a, Var) and a.name not in bound and a.name not in seen:
126
+ seen.append(a.name)
127
+ return seen
128
+
129
+
130
+ def check(clauses):
131
+ """Validate a program of existential rules; return {pred: arity}.
132
+
133
+ The core's checks apply to a *skeleton* of each rule in which the
134
+ existential variables are replaced by a placeholder constant: that
135
+ is exactly "safe, except for the variables we mean to invent"."""
136
+ for c in clauses:
137
+ if c.retract:
138
+ raise DatalogError("a retraction is an update, not a statement:"
139
+ " %s" % c)
140
+ if any(lit.negated for lit in c.body):
141
+ raise DatalogError(
142
+ "negation is not supported by the chase: %s. With "
143
+ "invented values, `not` would have to range over values "
144
+ "not invented yet" % c)
145
+ if c.body and aggregate_of(c.head):
146
+ raise DatalogError("aggregates are not supported by the chase:"
147
+ " %s" % c)
148
+ skeleton = []
149
+ for c in clauses:
150
+ ex = set(existentials(c)) if c.body else set()
151
+ head = Atom(c.head.pred, tuple(
152
+ Const(None) if isinstance(a, Var) and a.name in ex else a
153
+ for a in c.head.args))
154
+ skeleton.append(Rule(head, c.body, c.weight))
155
+ return validate(skeleton)
156
+
157
+
158
+ # ---------------------------------------------------------------------------
159
+ # Weak acyclicity: termination as a graph problem
160
+ # ---------------------------------------------------------------------------
161
+
162
+ def position_graph(rules):
163
+ """Edges between argument positions (pred, i), as Fagin et al. draw
164
+ them. For each variable a rule copies from body to head: a *normal*
165
+ edge from each body position it occupies to each head position it
166
+ occupies, and a *special* edge from each of those body positions to
167
+ every head position holding an existential — the value there is
168
+ invented *because* the body matched."""
169
+ normal, special = set(), set()
170
+ for rule in rules:
171
+ body_pos = defaultdict(list)
172
+ for lit in rule.body:
173
+ for i, a in enumerate(lit.atom.args):
174
+ if isinstance(a, Var):
175
+ body_pos[a.name].append((lit.atom.pred, i))
176
+ ex = set(existentials(rule))
177
+ ex_pos = [(rule.head.pred, i) for i, a in enumerate(rule.head.args)
178
+ if isinstance(a, Var) and a.name in ex]
179
+ for i, a in enumerate(rule.head.args):
180
+ if isinstance(a, Var) and a.name in body_pos:
181
+ for p in body_pos[a.name]:
182
+ normal.add((p, (rule.head.pred, i)))
183
+ special.update((p, q) for q in ex_pos)
184
+ return normal, special
185
+
186
+
187
+ def _path(edges, start, goal):
188
+ """A shortest path start -> goal in the graph, as a node list."""
189
+ succ = defaultdict(set)
190
+ for u, v in edges:
191
+ succ[u].add(v)
192
+ prev, todo = {start: None}, deque([start])
193
+ while todo:
194
+ u = todo.popleft()
195
+ if u == goal:
196
+ path = []
197
+ while u is not None:
198
+ path.append(u)
199
+ u = prev[u]
200
+ return path[::-1]
201
+ for v in sorted(succ[u]):
202
+ if v not in prev:
203
+ prev[v] = u
204
+ todo.append(v)
205
+ return None
206
+
207
+
208
+ def dangerous_cycle(rules):
209
+ """None if the rules are weakly acyclic; otherwise a cycle through a
210
+ special edge, as a list of (node, edge_kind_into_next) pairs."""
211
+ normal, special = position_graph(rules)
212
+ edges = normal | special
213
+ for p, q in sorted(special):
214
+ back = _path(edges, q, p)
215
+ if back is not None:
216
+ nodes = [p] + back # p =*=> q -> ... -> p
217
+ kinds = ["special"] + [
218
+ "normal" if (u, v) in normal else "special"
219
+ for u, v in zip(back, back[1:])]
220
+ return list(zip(nodes, kinds + [None]))
221
+ return None
222
+
223
+
224
+ def format_cycle(cycle):
225
+ out = []
226
+ for (pred, i), kind in cycle:
227
+ out.append("%s[%d]" % (pred, i + 1))
228
+ if kind:
229
+ out.append("=*=>" if kind == "special" else "->")
230
+ return " ".join(out)
231
+
232
+
233
+ # ---------------------------------------------------------------------------
234
+ # The chase
235
+ # ---------------------------------------------------------------------------
236
+
237
+ class ChaseBudget(DatalogError):
238
+ pass
239
+
240
+
241
+ def _solutions(body, rels):
242
+ """Every substitution satisfying the body: nested-loop joins, in a
243
+ fixed order so that null numbering is the same on every run."""
244
+ substs = [{}]
245
+ for lit in body:
246
+ rel = sorted(rels.get(lit.atom.pred, ()), key=_key)
247
+ substs = [s2 for s in substs for tup in rel
248
+ for s2 in (match(lit.atom.args, tup, s),) if s2 is not None]
249
+ return substs
250
+
251
+
252
+ def _saturate(rules, rels):
253
+ """The ordinary rules, to fixpoint, by the base engine. Facts may
254
+ hold nulls: to the engine they are values like any other."""
255
+ if not rules:
256
+ return rels
257
+ facts = [Rule(Atom(pred, tuple(Const(v) for v in tup)), ())
258
+ for pred, tups in rels.items() for tup in tups]
259
+ engine = Engine(Program(facts + list(rules)))
260
+ engine.run()
261
+ return engine.rels
262
+
263
+
264
+ class Chase:
265
+ def __init__(self, clauses, variant="restricted", max_nulls=1000):
266
+ if variant not in ("restricted", "oblivious"):
267
+ raise DatalogError("unknown chase variant %r" % variant)
268
+ self.arity = check(clauses)
269
+ self.variant = variant
270
+ self.max_nulls = max_nulls
271
+ self.facts = [c for c in clauses if not c.body]
272
+ rules = [c for c in clauses if c.body]
273
+ self.plain = [r for r in rules if not existentials(r)]
274
+ self.tgds = [r for r in rules if existentials(r)]
275
+ self.idb = {r.head.pred for r in rules}
276
+ self.cycle = dangerous_cycle(rules)
277
+ self.rels = defaultdict(set)
278
+ self.nulls = 0
279
+ self.rounds = 0
280
+
281
+ def _satisfied(self, head, s):
282
+ """Is there already a fact matching the head, for *some* values
283
+ of its existential variables? (match leaves them unbound in s,
284
+ so it binds them to whatever the fact holds.)"""
285
+ return any(match(head.args, tup, s) is not None
286
+ for tup in self.rels.get(head.pred, ()))
287
+
288
+ def _fresh(self):
289
+ if self.nulls >= self.max_nulls:
290
+ if self.cycle:
291
+ why = ("the rules are not weakly acyclic -- %s -- so the "
292
+ "chase may never stop" % format_cycle(self.cycle))
293
+ else:
294
+ why = "the rules are weakly acyclic, so a larger budget " \
295
+ "will reach the end"
296
+ raise ChaseBudget(
297
+ "no fixpoint after inventing %d labelled nulls: %s. "
298
+ "(--max-nulls raises the budget.)" % (self.max_nulls, why))
299
+ self.nulls += 1
300
+ return Null(self.nulls)
301
+
302
+ def run(self):
303
+ for f in self.facts:
304
+ self.rels[f.head.pred].add(tuple(a.value for a in f.head.args))
305
+ fired = set()
306
+ while True:
307
+ self.rels = _saturate(self.plain, self.rels)
308
+ self.rounds += 1
309
+ invented = False
310
+ for idx, rule in enumerate(self.tgds):
311
+ for s in _solutions(rule.body, self.rels):
312
+ trigger = (idx, tuple(sorted(s.items(), key=str)))
313
+ if trigger in fired:
314
+ continue
315
+ fired.add(trigger)
316
+ if self.variant == "restricted" \
317
+ and self._satisfied(rule.head, s):
318
+ continue
319
+ s = dict(s)
320
+ for name in existentials(rule):
321
+ s[name] = self._fresh()
322
+ self.rels[rule.head.pred].add(tuple(
323
+ a.value if isinstance(a, Const) else s[a.name]
324
+ for a in rule.head.args))
325
+ invented = True
326
+ if not invented:
327
+ return self.rels
328
+
329
+ def answers(self, atom):
330
+ return sorted((tup for tup in self.rels.get(atom.pred, ())
331
+ if match(atom.args, tup, {}) is not None), key=_key)
332
+
333
+
334
+ # ---------------------------------------------------------------------------
335
+ # CLI
336
+ # ---------------------------------------------------------------------------
337
+
338
+ @cli
339
+ def main(argv=None):
340
+ ap = argparse.ArgumentParser(
341
+ description="The chase: Datalog with existential rules. A head "
342
+ "variable the body does not bind is invented, as a "
343
+ "labelled null.")
344
+ ap.add_argument("file", help="program (.dl)")
345
+ ap.add_argument("-q", "--query", action="append", default=[],
346
+ metavar="ATOM", help="query (repeatable)")
347
+ ap.add_argument("-a", "--all", action="store_true",
348
+ help="print input relations too")
349
+ ap.add_argument("--variant", choices=("restricted", "oblivious"),
350
+ default="restricted")
351
+ ap.add_argument("--max-nulls", type=int, default=1000, metavar="N",
352
+ help="give up after inventing N nulls (default 1000)")
353
+ args = ap.parse_args(argv)
354
+
355
+ chase = Chase(parse(read_program(args.file)), args.variant,
356
+ args.max_nulls)
357
+ if chase.cycle:
358
+ print("not weakly acyclic: %s" % format_cycle(chase.cycle))
359
+ print(" (a null can feed the rule that invents the next one; "
360
+ "termination is not guaranteed)")
361
+ else:
362
+ print("weakly acyclic: the chase terminates on every database")
363
+ chase.run()
364
+ print("%s chase: %d round%s, %d labelled null%s invented"
365
+ % (chase.variant, chase.rounds, "" if chase.rounds == 1 else "s",
366
+ chase.nulls, "" if chase.nulls == 1 else "s"))
367
+ print()
368
+
369
+ if args.query:
370
+ for q in args.query:
371
+ atom = parse_query_atom(q, chase.arity)
372
+ found = chase.answers(atom)
373
+ print("?- %s" % atom)
374
+ for tup in found:
375
+ print(" %s.%s" % (format_atom(atom.pred, tup),
376
+ " [not certain: mentions a null]"
377
+ if has_null(tup) else ""))
378
+ certain = sum(1 for t in found if not has_null(t))
379
+ print(" (%d answer%s, %d certain)"
380
+ % (len(found), "" if len(found) == 1 else "s", certain))
381
+ return 0
382
+
383
+ preds = sorted(set(chase.arity) if args.all else chase.idb)
384
+ for pred in preds:
385
+ tuples = sorted(chase.rels.get(pred, ()), key=_key)
386
+ kind = "derived" if pred in chase.idb else "input"
387
+ print("%% %s/%d (%s) — %d fact%s" % (
388
+ pred, chase.arity[pred], kind, len(tuples),
389
+ "" if len(tuples) == 1 else "s"))
390
+ for tup in tuples:
391
+ print(format_atom(pred, tup) + ".")
392
+ print()
393
+ return 0
394
+
395
+
396
+ if __name__ == "__main__":
397
+ sys.exit(main())
@@ -18,7 +18,8 @@ The package is built in three tiers:
18
18
  well-founded models (semantics.py), semiring weights (semiring.py),
19
19
  maintenance under updates (incremental.py), defeasible logic
20
20
  (defeasible.py), Prolog with function symbols (prolog.py), concept
21
- subsumption (subsumption.py) and query containment (containment.py).
21
+ subsumption (subsumption.py), query containment (containment.py) and
22
+ existential rules (chase.py).
22
23
 
23
24
  Every engine and extension imports from here; nothing here imports
24
25
  them.
@@ -555,7 +555,10 @@ def _earlier_solution(engine, rule, seed, stamp, index):
555
555
  new.append(m)
556
556
  substs = new
557
557
  known |= {a.name for a in args if isinstance(a, Var)}
558
- for s in substs:
558
+ # several derivations may qualify; the index tables are built from
559
+ # sets, whose order varies with the hash seed, so pick in a fixed
560
+ # order or the same --explain prints a different tree from run to run
561
+ for s in sorted(substs, key=lambda s: sort_key([s[k] for k in sorted(s)])):
559
562
  if all(engine.instantiate(lit.atom, s)
560
563
  not in engine.rels.get(lit.atom.pred, _EMPTY)
561
564
  for lit in rule.body if lit.negated):
@@ -39,9 +39,10 @@ it and demand the semi-naive answer from each. `datalog.py --engine
39
39
  NAME -q ATOM` picks one from the command line.
40
40
 
41
41
  The extensions (semantics, semiring, incremental, defeasible, prolog,
42
- subsumption, containment) are deliberately not here: they ask other
43
- questions — other semantics, weighted answers, updates, a different
44
- logic — so "same answers as semi-naive" is not their contract.
42
+ subsumption, containment, chase) are deliberately not here: they ask
43
+ other questions — other semantics, weighted answers, updates, a
44
+ different logic, invented values — so "same answers as semi-naive" is
45
+ not their contract.
45
46
  """
46
47
 
47
48
  from __future__ import annotations
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: tiny-datalog
3
- Version: 0.4.0
3
+ Version: 0.5.0
4
4
  Summary: A Datalog engine whose core you can read in an afternoon, and a course built on it
5
5
  Author: Andrew Goodchild
6
6
  License-Expression: MIT
@@ -127,7 +127,7 @@ Nothing to install:
127
127
 
128
128
  ```sh
129
129
  git clone https://github.com/andrewgoodchild/tiny-datalog && cd tiny-datalog
130
- python3 tests.py # 303 tests, ~12s
130
+ python3 tests.py # 326 tests, ~12s
131
131
  ```
132
132
 
133
133
  ## Why the language choice decides what you can ask later
@@ -249,8 +249,12 @@ the current research threads. The field's own recent lecture notes
249
249
  observe that the literature advises people building Datalog engines
250
250
  better than people trying to *use* one; this course does both halves
251
251
  on purpose — sixteen lessons where the engine is the explanation, then
252
- a lesson on authoring rules that survive review, and one on a logic
253
- built for rules with exceptions. And it is built to be
252
+ a lesson on authoring rules that survive review, one on a logic
253
+ built for rules with exceptions, and three where the engine goes to
254
+ work: [authorization](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/20-authorization.md),
255
+ [program analysis](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/21-program-analysis.md),
256
+ and [knowledge graphs and the chase](https://github.com/andrewgoodchild/tiny-datalog/blob/main/lessons/22-knowledge-graphs.md).
257
+ And it is built to be
254
258
  inherited: `git clone`, no dependencies, no hosted anything, and every
255
259
  quoted transcript re-verified by CI — the exercises cannot rot. (For where each
256
260
  technique ships — CodeQL, RDFox, Feldera, SNOMED and the rest —
@@ -384,18 +388,20 @@ tiny_datalog/ the code you read, in three tiers:
384
388
  subsumption.py KL-ONE-style EL classifier, compiled to Datalog
385
389
  containment.py query containment and minimisation by homomorphism
386
390
  defeasible.py defeasible logic: exceptions, priorities, defeaters
391
+ chase.py existential rules: the chase, labelled nulls, weak
392
+ acyclicity
387
393
  *.py three-line launchers, so `python3 datalog.py ...` works
388
394
  straight from a checkout with nothing installed
389
395
  programs/ teaching programs, numbered by the lesson that uses
390
396
  them (00-* are the README's examples)
391
- lessons/ getting started, glossary, and lessons 0–19
397
+ lessons/ getting started, glossary, and lessons 0–22
392
398
  exercises/ worked answers, verified by the test suite
393
399
  cases/ golden test cases — add one without writing Python
394
400
  conformance/ the external datalog-conformance corpus (Soufflé, Nemo,
395
401
  Crepe, SPINdle), run against every evaluation strategy
396
402
  benchmarks/ scaled input generators (chain/tree/clique/grid, an EL
397
403
  ontology, and the N-queens puzzle)
398
- tests.py 303 tests: every shipped program and exercise answer is
404
+ tests.py 326 tests: every shipped program and exercise answer is
399
405
  executed, a conformance suite runs every query through
400
406
  every engine that handles it, and a seeded fuzzer checks
401
407
  the same property on random programs
@@ -412,7 +418,7 @@ This project started as an attempt at a tiny implementation of
412
418
  Datalog, small enough to read in an afternoon. It grew with the
413
419
  course. Each lesson that taught a new technique — magic sets, stable models,
414
420
  semirings, incremental maintenance, tabling, subsumption, containment,
415
- defeasible logic — brought a module to run it, and the repository is
421
+ defeasible logic, the chase — brought a module to run it, and the repository is
416
422
  now several thousand lines. The core idea stayed small, and the code
417
423
  is laid out so you can still read just that: the common core and the
418
424
  base engine, below, are the original tiny Datalog. Everything else is
@@ -424,19 +430,19 @@ Measured in lines, comments, docstrings and blank lines included:
424
430
  |---|---|
425
431
  | the common core (`core.py`: the language, evaluating nothing) | 785 |
426
432
  | the base engine's evaluator (`datalog.py` up to its CLI section: the module docstring, the import list that re-exports the core, `Program`, `Engine`, aggregation folding, `run_program`) | 395 |
427
- | `--explain`, why-not, printing and the CLI (the rest of `datalog.py`) | 535 |
433
+ | `--explain`, why-not, printing and the CLI (the rest of `datalog.py`) | 538 |
428
434
  | the other pluggable engines: magic sets (`magic.py`) and tabling (`tabling.py`), plus the `engines.py` registry (naive evaluation is a flag on the base engine) | 549 |
429
- | the seven extensions | 2,425 |
435
+ | the eight extensions | 2,822 |
430
436
 
431
437
  So the smallest complete Datalog system here — parse, check, stratify,
432
438
  evaluate semi-naively — is the core plus the evaluator, about 1,200
433
- lines. All of it is 4.7k lines of toolkit (every module but the
439
+ lines. All of it is 5.1k lines of toolkit (every module but the
434
440
  re-export list in `__init__.py`), roughly a quarter of it commentary,
435
- and 3.4k of tests.
441
+ and 3.6k of tests.
436
442
 
437
443
  "Tiny" is a claim about that core and evaluator, and about each module
438
- built on them singly: none of the ten (`magic.py`, `tabling.py`,
439
- `engines.py` and the seven extensions) exceeds 500 lines, which a test asserts. It is not a
444
+ built on them singly: none of the eleven (`magic.py`, `tabling.py`,
445
+ `engines.py` and the eight extensions) exceeds 500 lines, which a test asserts. It is not a
440
446
  claim about the repository, which is a dozen modules because it
441
447
  teaches a dozen things.
442
448
 
@@ -2,6 +2,7 @@ LICENSE
2
2
  README.md
3
3
  pyproject.toml
4
4
  tiny_datalog/__init__.py
5
+ tiny_datalog/chase.py
5
6
  tiny_datalog/containment.py
6
7
  tiny_datalog/core.py
7
8
  tiny_datalog/datalog.py
@@ -1,5 +1,6 @@
1
1
  [console_scripts]
2
2
  tiny-datalog = tiny_datalog.datalog:main
3
+ tiny-datalog-chase = tiny_datalog.chase:main
3
4
  tiny-datalog-containment = tiny_datalog.containment:main
4
5
  tiny-datalog-defeasible = tiny_datalog.defeasible:main
5
6
  tiny-datalog-incremental = tiny_datalog.incremental:main
File without changes
File without changes