@littlebigbrain/create 0.0.0-stage → 0.2.0

package/templates/ontology-fit/README.md

@@ -0,0 +1,81 @@
+# ontology-fit
+
+Reads the text of each new or changed instance of a class and proposes the
+ontology changes the text supports: a class `Mangel` for the defects a set
+of site reports follows, its properties `frist` and `status`, a relation
+`zustaendig` to `Firma`. Each change becomes a review item with its
+evolve operations and the quotes that support it; a person accepts it, and
+the ontology changes.
+
+The LBB-run version is the starter workflow `ontology.fit@v1`, the fit
+sources of `/v1/ontology/fit-sources` (`docs/reference/fit-sources.md`).
+
+## How it runs
+
+The triggered workflow watches the class and its text field. For each
+instance of a trigger message, one step:
+
+1. The reasoning model reads the text with the ontology
+   (`lbb.ontology()`, as classes, relations with their domain and range,
+   and properties) and the changes the earlier texts of the turn proposed,
+   and returns up to 12 proposals: `add_class`, `add_property`,
+   `add_relation` or `widen_relation`, each with a sentence of the text as
+   its quote. The prompt and schema are the LBB version's.
+2. Each proposal becomes a change (`shape`): its key, its title and its
+   evolve operations. A class the change names that the ontology lacks is
+   added first, and a new relation with the name of an existing one widens
+   it. The key follows the LBB version:
+   `fit:<workflow name>/add_class/mangel`,
+   `fit:<workflow name>/add_relation/mangel/zustaendig/firma`,
+   `fit:<workflow name>/widen_relation/betrifft-raum/mangel/x`.
+3. A change is kept when its quote is in the text with at least three
+   words, and a dry run (`lbb.evolve(change, { dryRun: true })`) says the
+   ontology does not have it yet.
+
+A last step files one review item per key, with the quotes of every text of
+the turn that supports it. The same key again updates the item. A turn reads
+at most 48 texts; the rest goes to a new turn.
+
+## Settings
+
+`defaults` in `workflow.ts`, which fit the samples:
+
+| Setting | Default | Meaning |
+| --- | --- | --- |
+| `className` | `Baustellenbericht` | The class whose instances hold the text |
+| `textField` | `text` | The property that holds the text |
+| `context` | the samples' | One sentence about what the text is |
+
+`trigger(settings)` names the triggered workflow `berichte`; the name starts
+every key. The workflow writes no graph facts, so it owns nothing.
+
+## What the LBB-run version adds
+
+- **Support scores.** The decision model scores how well each quote
+  supports its change; a change below 0.5 is dropped.
+- **Support across instances.** The instances that support each key are
+  counted over every run, and a dismissed change opens again when twice its
+  support and three more instances support it.
+- **The proposals of the source.** The open suggestions of the source and
+  the names of the dismissed ones go into every prompt, so the model reuses
+  names and does not propose a rejected change again.
+- **Name style.** A new relation or property name is written in the
+  ontology's style (`works_in` or `WORKS_IN`).
+- **Long texts.** A text past 40,000 characters is read in overlapping
+  chunks.
+- Its own monthly budget, a preview with the proposals of a sample, and
+  counters per outcome in the source's status.
+
+## Samples and test
+
+`samples/texts.json` holds five invented site reports and meeting minutes;
+`samples/expected.md` describes the graph before the run, the proposals a
+good run makes with their keys, those it may make, and those it does not
+make. `samples.ts` sets up that graph for `npm run try`.
+
+`workflow.test.ts` runs the workflow with a mock reasoning model whose rules
+are fixed (a sentence about a `Mangel` proposes the class, `Frist`, `Status`
+and `Zuständig` its facts, a room widens `betrifft_raum`). The run files
+the five keys of `expected.md` and no other: the relation `anwesend`, which
+the ontology has, is dropped by the dry run, and a proposal whose quote is
+not in its text is dropped by the quote check.

package/templates/ontology-fit/samples/expected.md

@@ -0,0 +1,88 @@
+# Fit from text samples: what a good run proposes
+
+The five texts in `texts.json` are invented site reports and meeting minutes
+of one project, Neubau Grundschule Musterstadt. They follow the defects found
+on site over four weeks: in which room, which company must fix each one, by
+when, and whether it is fixed. The graph has no class for a defect, so a fit
+run over these texts should propose one, with the facts the texts give about
+each defect. Texts are numbered 1 to 5 in the order of `texts.json`.
+
+## The graph before the run
+
+Each text is an instance of `Baustellenbericht`: its name is the label, its
+text is the property `text`. The ontology holds:
+
+| Class | Properties |
+| --- | --- |
+| `Baustellenbericht` | `text` (text), `datum` (datetime) |
+| `Raum` | `nummer` (keyword), `bezeichnung` (text) |
+| `Firma` | `name` (text), `gewerk` (keyword) |
+
+| Relation | From | To |
+| --- | --- | --- |
+| `betrifft_raum` | `Baustellenbericht` | `Raum` |
+| `anwesend` | `Baustellenbericht` | `Firma` |
+
+The fit source:
+
+```json
+{
+  "class": "Baustellenbericht",
+  "name": "berichte",
+  "from": ["text"],
+  "context": "Site reports and meeting minutes of a school building project, in German"
+}
+```
+
+## Proposals a good run makes
+
+The names follow the ontology: lower snake_case in ASCII (`zustaendig` for
+"zuständig"). A run can spell a name another way with the same meaning. The
+keys hold for the names in this table (a suggestion key keeps lowercase ASCII
+letters and digits, and writes every other run of characters as one dash; a
+missing class in a widen key is `x`).
+
+| Change | Key | Texts | Quote (one of them) |
+| --- | --- | --- | --- |
+| Add class `Mangel` | `fit:berichte/add_class/mangel` | 1 to 5 | "Der Mangel wurde der Fliesenbau Muster GmbH gemeldet" |
+| Add property `frist` (datetime) to `Mangel` | `fit:berichte/add_property/mangel/frist` | 1 to 4 | "die Frist zur Beseitigung endet am 16.09.2026" |
+| Add property `status` (keyword: `offen`, `behoben`) to `Mangel` | `fit:berichte/add_property/mangel/status` | 3 to 5 | "Frist 23.09.2026, Status offen" |
+| Add relation `zustaendig` from `Mangel` to `Firma` | `fit:berichte/add_relation/mangel/zustaendig/firma` | 1 to 4 | "Zuständig ist die Trockenbau Probe GmbH" |
+| Let `betrifft_raum` start at `Mangel` | `fit:berichte/widen_relation/betrifft-raum/mangel/x` | 1 to 5 | "In Raum 0.12 fehlt der Spiegel über dem Waschtisch" |
+
+The last row tests reuse. Every defect names its room, and the ontology
+already has a relation to `Raum`, so a good run widens `betrifft_raum`. A new
+relation from `Mangel` to `Raum` with another name (`liegt_in`,
+`befindet_sich_in`) is a synonym of it.
+
+## Proposals a good run may make
+
+- Add property `einstufung` (keyword: `wesentlich`, `unwesentlich`) to
+  `Mangel`. Only text 5 gives it: "die Bauleitung stuft diesen Mangel als
+  unwesentlich ein".
+- Add property `beschreibung` (text) to `Mangel`.
+- Add relation `stellt_fest` from `Baustellenbericht` to `Mangel`, for the
+  report that records a defect.
+- Add class `Bauteil`, for the part a defect concerns (Bodenfliesen, Spiegel,
+  Trennwand).
+- Add class `Abnahme` with a date. Only text 5 names it.
+- Add class `Lieferung` with a `liefertermin` (datetime). Texts 2 and 4 give
+  the delivery date of the ceiling lights.
+
+## Proposals a good run does not make
+
+- A relation from `Baustellenbericht` to `Firma` for the companies present
+  ("Anwesend: …", "Teilnehmer: …"). `anwesend` holds it; the check drops
+  such a change as `already_covered`.
+- A property for the date of a report. `datum` holds it.
+- A class for one company, one room or one defect (`Fliesenbau Muster GmbH`,
+  `Flur 0.10`, `Trennwand Kabine 2`). These are instances.
+- A second name for a change the run proposes: `Maengel` or `Defect` beside
+  `Mangel`, `deadline` beside `frist`, `verantwortlich` beside `zustaendig`.
+
+## How to check a run
+
+1. Every change in "Proposals a good run makes" is filed.
+2. No change in "Proposals a good run does not make" is filed.
+3. Every filed quote is in its text, with whitespace runs as one space, and
+   holds at least three words.

package/templates/ontology-fit/samples/texts.json

@@ -0,0 +1,22 @@
+[
+  {
+    "name": "Baustellenbericht 12",
+    "text": "Baustellenbericht 12 vom 02.09.2026. Begehung des Erdgeschosses mit der Bauleitung. Anwesend: Fliesenbau Muster GmbH, Bodenbau Beispiel GmbH. In Raum 0.14 (WC Mädchen) sind zwei Bodenfliesen gerissen. Der Mangel wurde der Fliesenbau Muster GmbH gemeldet, die Frist zur Beseitigung endet am 16.09.2026. In Raum 0.12 ist der Estrich trocken; die Bodenbau Beispiel GmbH verlegt das Linoleum ab dem 07.09.2026."
+  },
+  {
+    "name": "Baubesprechung 05.09.2026",
+    "text": "Protokoll der Baubesprechung vom 05.09.2026. Teilnehmer: Bauleitung, Elektro Beispiel GmbH, Haustechnik Exempel KG. Die Elektro Beispiel GmbH meldet, dass die Deckenleuchten erst am 21.09.2026 geliefert werden. Die Haustechnik Exempel KG hat die Waschtische im Erdgeschoss montiert. Mangel: In Raum 0.12 fehlt der Spiegel über dem Waschtisch. Zuständig ist die Haustechnik Exempel KG, Frist 12.09.2026."
+  },
+  {
+    "name": "Baustellenbericht 13",
+    "text": "Baustellenbericht 13 vom 09.09.2026. Mängelbegehung in Raum 0.14: Die gerissenen Bodenfliesen sind ersetzt, der Mangel ist behoben. Neu festgestellt: Die Trennwand der WC-Kabine 2 in Raum 0.14 steht schief. Zuständig ist die Trockenbau Probe GmbH, Frist 23.09.2026, Status offen."
+  },
+  {
+    "name": "Baustellenbericht 14",
+    "text": "Baustellenbericht 14 vom 16.09.2026. In Raum 0.12 ist der Spiegel montiert, der Mangel ist behoben. Im Flur 0.10 hat das Linoleum an zwei Stellen tiefe Kratzer. Zuständig ist die Bodenbau Beispiel GmbH, Frist 30.09.2026, Status offen. Die Deckenleuchten sind geliefert und werden bis zum 25.09.2026 montiert."
+  },
+  {
+    "name": "Vorbegehung zur Abnahme",
+    "text": "Vorbegehung zur Abnahme am 28.09.2026 mit Bauherr und Bauleitung. Die Trennwand der Kabine 2 in Raum 0.14 ist ausgerichtet, der Mangel ist behoben. Offen bleiben die Kratzer im Flur 0.10; die Bauleitung stuft diesen Mangel als unwesentlich ein. Wesentliche Mängel liegen nicht vor. Die Abnahme ist für den 02.10.2026 geplant."
+  }
+]

package/templates/ontology-fit/samples.ts

@@ -0,0 +1,40 @@
+// The samples of ontology-fit, for `npm run try`: the ontology of
+// samples/expected.md and five invented site reports (samples/texts.json),
+// committed as instances of the class the settings name.
+import { readFile } from "node:fs/promises";
+import { join } from "node:path";
+import type { LbbClient, Schemas } from "@littlebigbrain/client";
+import { defaults, ontology } from "./workflow.js";
+
+/** The graph before the run: the classes, properties and relations of expected.md. */
+export const sampleOntology: Schemas["OntologyEvolveOp"][] = [
+  ...ontology(),
+  { op: "add_entity_type", name: "Raum" },
+  { op: "add_entity_type", name: "Firma" },
+  { op: "add_property", name: "datum", value_type: "date_time" },
+  { op: "add_property", name: "nummer", value_type: "keyword" },
+  { op: "add_property", name: "bezeichnung", value_type: "text" },
+  { op: "add_property", name: "gewerk", value_type: "keyword" },
+  { op: "add_relation", name: "betrifft_raum", domain: [defaults.className], range: ["Raum"] },
+  { op: "add_relation", name: "anwesend", domain: [defaults.className], range: ["Firma"] },
+];
+
+/** Set up the sample ontology and commit the sample texts to the client's graph. */
+export async function loadSamples(client: LbbClient, dir: string): Promise<void> {
+  const texts = JSON.parse(await readFile(join(dir, "texts.json"), "utf8")) as {
+    name: string;
+    text: string;
+  }[];
+  await client.ontology.evolve({ ops: sampleOntology });
+  await client.commit(
+    {
+      entity_properties: texts.map((t) => ({
+        type: defaults.className,
+        name: t.name,
+        properties: { [defaults.textField]: t.text },
+      })),
+    },
+    { idempotencyKey: "ontology-fit-samples" },
+  );
+  console.log(`Committed ${texts.length} ${defaults.className} texts`);
+}

package/templates/ontology-fit/workflow.test.ts

@@ -0,0 +1,270 @@
+// ontology-fit on its samples, with a mock reasoning model whose rules are
+// fixed: a sentence about a defect (`Mangel`) proposes the class `Mangel`,
+// `Frist`, `Status` and `Zuständig` propose its facts, a room in the same
+// text widens `betrifft_raum`, a list of who was present proposes the
+// relation `anwesend` the ontology has, and a text about the acceptance
+// (`Abnahme`) gets a quote that is not in the text. The expected keys come
+// from samples/expected.md.
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import { join } from "node:path";
+import { ENTITY_IRI_NAMESPACE, type Schemas } from "@littlebigbrain/client";
+import {
+  MockActivities,
+  mockId,
+  phases,
+  readJson,
+  runMessage,
+  samplesDir,
+  triggerMessage,
+  type MockRules,
+} from "../_shared/testing.js";
+import { createWorkflow, keyPart, quoteInText } from "./workflow.js";
+import { sampleOntology } from "./samples.js";
+
+const SAMPLES = samplesDir(import.meta.url, "ontology-fit");
+const TEXTS = readJson<{ name: string; text: string }[]>(join(SAMPLES, "texts.json"));
+const EXPECTED_MD = readFileSync(join(SAMPLES, "expected.md"), "utf8");
+/** The keys of the table "Proposals a good run makes". */
+const EXPECTED_KEYS = [
+  ...EXPECTED_MD.split("## Proposals a good run makes")[1]!
+    .split("## Proposals a good run may make")[0]!
+    .matchAll(/`(fit:[^`]+)`/g),
+].map((m) => m[1]!);
+
+const iriOf = (name: string) => ENTITY_IRI_NAMESPACE + mockId(`Baustellenbericht:${name}`);
+
+/** The ontology view of the graph before the run, built from the sample ontology. */
+function view(): Schemas["OntologyView"] {
+  const ops = sampleOntology;
+  const types = ops.flatMap((op) => (op.op === "add_entity_type" ? [op.name] : []));
+  return {
+    graph: { tenant_id: "t", graph_id: "samples", branch_id: "main" },
+    ontology_version: 3,
+    entity_types: [],
+    relations: [],
+    entity_type_defs: types.map((name) => ({
+      name,
+      stable_id: `type:${name}`,
+      properties: [],
+      resolvable: true,
+      since_version: 1,
+    })),
+    relation_defs: ops.flatMap((op) =>
+      op.op === "add_relation"
+        ? [
+            {
+              name: op.name,
+              src_types: op.domain ?? [],
+              dst_types: op.range ?? [],
+              cardinality: "many_to_many",
+              since_version: 1,
+              symmetric: false,
+              temporal_semantics: "bitemporal",
+              transitive: false,
+            },
+          ]
+        : [],
+    ),
+    property_defs: ops.flatMap((op) =>
+      op.op === "add_property"
+        ? [{ name: op.name, value_type: op.value_type ?? "text", required: false }]
+        : [],
+    ),
+  } as Schemas["OntologyView"];
+}
+
+/** Whether the ontology has a change already: the dry run's no-op. */
+function covered(op: Schemas["OntologyEvolveOp"], v: Schemas["OntologyView"]): boolean {
+  const has = (names: { name: string }[] | undefined, name: string) =>
+    (names ?? []).some((n) => n.name.toLowerCase() === name.toLowerCase());
+  if (op.op === "add_entity_type") return has(v.entity_type_defs, op.name);
+  if (op.op === "add_property") return has(v.property_defs, op.name);
+  if (op.op === "add_relation") return has(v.relation_defs, op.name);
+  if (op.op === "widen_relation") {
+    const r = v.relation_defs?.find((d) => d.name.toLowerCase() === op.relation.toLowerCase());
+    return Boolean(
+      r &&
+      (op.add_domain ?? []).every((d) => r.src_types.includes(d)) &&
+      (op.add_range ?? []).every((d) => r.dst_types.includes(d)),
+    );
+  }
+  return false;
+}
+
+const proposal = (fields: Partial<Record<string, string>>) => ({
+  kind: "",
+  name: "",
+  parent: "",
+  class: "",
+  value_type: "",
+  domain: "",
+  range: "",
+  quote: "",
+  reason: "",
+  ...fields,
+});
+
+function propose(text: string) {
+  const sentences = text.split(/(?<=\.)\s+/);
+  const find = (pattern: RegExp) => sentences.find((s) => pattern.test(s));
+  const out = [];
+  const defect = find(/Mangel|Mängel/);
+  if (defect)
+    out.push(
+      proposal({
+        kind: "add_class",
+        name: "Mangel",
+        quote: defect,
+        reason: "The texts follow defects.",
+      }),
+    );
+  const deadline = find(/Frist/);
+  if (defect && deadline)
+    out.push(
+      proposal({
+        kind: "add_property",
+        name: "frist",
+        class: "Mangel",
+        value_type: "date_time",
+        quote: deadline,
+      }),
+    );
+  const status = find(/Status/);
+  if (defect && status)
+    out.push(
+      proposal({
+        kind: "add_property",
+        name: "status",
+        class: "Mangel",
+        value_type: "keyword",
+        quote: status,
+      }),
+    );
+  const owner = find(/Zuständig/);
+  if (defect && owner)
+    out.push(
+      proposal({
+        kind: "add_relation",
+        name: "zustaendig",
+        domain: "Mangel",
+        range: "Firma",
+        quote: owner,
+      }),
+    );
+  const room = find(/Raum 0\.\d+/);
+  if (defect && room)
+    out.push(
+      proposal({ kind: "widen_relation", name: "betrifft_raum", domain: "Mangel", quote: room }),
+    );
+  const present = find(/Anwesend:|Teilnehmer:/);
+  if (present)
+    out.push(
+      proposal({
+        kind: "add_relation",
+        name: "anwesend",
+        domain: "Baustellenbericht",
+        range: "Firma",
+        quote: present,
+      }),
+    );
+  if (find(/Abnahme/))
+    out.push(
+      proposal({ kind: "add_class", name: "Abnahme", quote: "Die Abnahme findet morgen statt." }),
+    );
+  return out;
+}
+
+function rules(more: MockRules = {}): MockRules {
+  const v = view();
+  return {
+    ontology: v,
+    reason: (request) => ({
+      proposals: propose((JSON.parse(request.prompt) as { text: string }).text),
+    }),
+    evolve: (ops) => ({ no_op: ops.every((op) => covered(op, v)) }),
+    sparql: (query) =>
+      TEXTS.filter((t) => query.includes(iriOf(t.name))).map((t) => ({
+        s: iriOf(t.name),
+        label: t.name,
+        text: t.text,
+      })),
+    ...more,
+  };
+}
+
+test("the samples give the changes of expected.md, one review item per key", async () => {
+  const lbb = new MockActivities(rules());
+  const { state, turns } = await runMessage(
+    createWorkflow(lbb),
+    triggerMessage(
+      "berichte",
+      TEXTS.map((t) => iriOf(t.name)),
+    ),
+  );
+
+  assert.deepEqual(phases(turns), ["reading texts", "filing review items"]);
+  assert.deepEqual(
+    turns[0]!.progress.filter((p) => p.phase === "filing review items").map((p) => p.done),
+    [0, 1, 2, 3, 4],
+  );
+  assert.equal(EXPECTED_KEYS.length, 5);
+  assert.deepEqual([...lbb.reviews.keys()].sort(), [...EXPECTED_KEYS].sort());
+  assert.deepEqual(state, { texts: 5, proposals: 3 + 4 + 5 + 5 + 2, filed: 5 });
+  const mangel = lbb.reviews.get("fit:berichte/add_class/mangel")!;
+  assert.equal(mangel.workflow, "berichte");
+  assert.deepEqual(mangel.change, [{ op: "add_entity_type", name: "Mangel" }]);
+  assert.equal(mangel.evidence!.records, 5);
+});
+
+test("a change names the classes it needs, and a widening keeps the relation's name", async () => {
+  const lbb = new MockActivities(rules());
+  await runMessage(
+    createWorkflow(lbb),
+    triggerMessage(
+      "berichte",
+      TEXTS.map((t) => iriOf(t.name)),
+    ),
+  );
+
+  assert.deepEqual(lbb.reviews.get("fit:berichte/add_relation/mangel/zustaendig/firma")!.change, [
+    { op: "add_entity_type", name: "Mangel" },
+    { op: "add_relation", name: "zustaendig", domain: ["Mangel"], range: ["Firma"] },
+  ]);
+  assert.deepEqual(lbb.reviews.get("fit:berichte/widen_relation/betrifft-raum/mangel/x")!.change, [
+    { op: "add_entity_type", name: "Mangel" },
+    { op: "widen_relation", relation: "betrifft_raum", add_domain: ["Mangel"], add_range: [] },
+  ]);
+  assert.equal(
+    lbb.reviews.get("fit:berichte/add_property/mangel/frist")!.title,
+    "Add property frist to Mangel",
+  );
+});
+
+test("a change the ontology has, and a quote that is not in the text, are not filed", async () => {
+  const lbb = new MockActivities(rules());
+  await runMessage(
+    createWorkflow(lbb),
+    triggerMessage(
+      "berichte",
+      TEXTS.map((t) => iriOf(t.name)),
+    ),
+  );
+  const keys = [...lbb.reviews.keys()];
+
+  assert.ok(!keys.some((k) => k.includes("anwesend")), "anwesend is already in the ontology");
+  assert.ok(!keys.some((k) => k.includes("abnahme")), "its quote is not in the text");
+  for (const item of lbb.reviews.values())
+    for (const sample of item.evidence!.samples! as { entity: string; quote: string }[]) {
+      const text = TEXTS.find((t) => iriOf(t.name) === sample.entity)!.text;
+      assert.ok(quoteInText(sample.quote, text), sample.quote);
+    }
+});
+
+test("keys keep lowercase ASCII letters and digits, other runs as one dash", () => {
+  assert.equal(keyPart("Mangel"), "mangel");
+  assert.equal(keyPart("betrifft_raum"), "betrifft-raum");
+  assert.equal(keyPart("-"), "x");
+  assert.equal(keyPart("zuständig"), "zust-ndig");
+});