strom-research 1.19.0-beta.5 → 1.19.0-beta.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.cs.md CHANGED
@@ -7,7 +7,7 @@ Změny každé verze ve zkratce. Úplné poznámky a instalační řádky jsou n
7
7
 
8
8
  ## 1.19.0
9
9
 
10
- *Připravuje se: bety do 1.19.0-beta.5. Oddíl bude úplný s vydáním.*
10
+ *Připravuje se: bety do 1.19.0-beta.6. Oddíl bude úplný s vydáním.*
11
11
 
12
12
  - **Na archiv, který zmlkl, se čeká, místo opakovaných dotazů.** Když archiv přestane odpovídat, strom ho nechá na
13
13
  pokoji hodinu — a když týž den mlčí dál, dvě a pak čtyři hodiny — a agent se dozví, do kdy: mezitím dělá
@@ -30,15 +30,16 @@ Změny každé verze ve zkratce. Úplné poznámky a instalační řádky jsou n
30
30
  udělala špatně, `strom fixes` a Nastavení → Údržba → Opravy řeknou, o co jde, co se našlo a co by napravení udělalo,
31
31
  i jak je náročné; bez souhlasu se nestane nic a odmítnutí platí. První: slepá čtení ze zmenšeného snímku tam, kde
32
32
  byla ostřejší část — přečtou se znovu vedle starých, nejdřív na několika, nic se nepřepíše, rozdíl se stane úkolem.
33
- Druhá: po změně modelu nebo agenta pro čtení dřívější čtení, která stojí za nové přečtení — s otevřenými místy,
34
- s neshodou čtenářů, s otevřeným rozporem, nepotvrzená u pramene; řečeno jako „četl model X, teď se čte modelem Y“,
35
- nabídnuto jednou na agenta a model, každé se přečte znovu tak, jak se četlo (snímky jednoho čtení spolu, zkouška
36
- i část vždy z celých čtení). Počítá se v čteních s jejich místy a náročnost ve voláních čtenáře; nabízí se po
37
- stupních — jen sporná (rozchod čtenářů, otevřený rozpor, nepotvrzené u pramene), s přímou linií, nebo všechna —
38
- každý s tím, co stojí, nejužší první; data a svědci jen řadí. Napravit může i jiný připravený agent nebo model
39
- (`strom fixes run <id> --agent <a> --model <m>`), s tím, kdo četl dřív; ten, kdo tu číst nemůže, se odmítne hned
40
- i s těmi, kdo mohou. Čas napravení se zaokrouhluje nahoru a po zkoušce se přepočítá podle čtení, jak běžela
41
- souběžně.
33
+ Čtení se porovnají podle přečteného — jmen, dat a čísel přepisu, ne slov komentáře čtenáře — a úkol jednou větou
34
+ řekne, který údaj se liší. Druhá: po změně modelu nebo agenta pro čtení dřívější čtení, která stojí za nové přečtení
35
+ — s otevřenými místy, s neshodou čtenářů, s otevřeným rozporem, nepotvrzená u pramene; řečeno jako „četl model X,
36
+ teď se čte modelem Y“, nabídnuto jednou na agenta a model, každé se přečte znovu tak, jak se četlo (snímky jednoho
37
+ čtení spolu, zkouška i část vždy z celých čtení). Počítá se v čteních s jejich místy a náročnost ve voláních
38
+ čtenáře; nabízí se po stupních — jen sporná (rozchod čtenářů, otevřený rozpor, nepotvrzené u pramene), s přímou
39
+ linií, nebo všechna — každý s tím, co stojí, nejužší první; data a svědci jen řadí. Napravit může i jiný připravený
40
+ agent nebo model (`strom fixes run <id> --agent <a> --model <m>`), s tím, kdo četl dřív; ten, kdo tu číst nemůže, se
41
+ odmítne hned i s těmi, kdo mohou. Čas napravení se zaokrouhluje nahoru a po zkoušce se přepočítá podle čtení, jak
42
+ běžela souběžně.
42
43
 
43
44
  ## 1.18.0
44
45
 
package/CHANGELOG.de.md CHANGED
@@ -7,7 +7,7 @@ Die Änderungen jeder Version in Kürze. Die vollständigen Hinweise und die Ins
7
7
 
8
8
  ## 1.19.0
9
9
 
10
- *In Vorbereitung: die Betas bis 1.19.0-beta.5. Der Abschnitt ist mit dem Release vollständig.*
10
+ *In Vorbereitung: die Betas bis 1.19.0-beta.6. Der Abschnitt ist mit dem Release vollständig.*
11
11
 
12
12
  - **Auf ein verstummtes Archiv wird gewartet, statt es immer wieder zu fragen.** Antwortet ein Archiv nicht mehr,
13
13
  lässt strom es eine Stunde in Ruhe — zwei, dann vier, wenn es am selben Tag still bleibt — und der Agent erfährt, bis
@@ -31,20 +31,21 @@ Die Änderungen jeder Version in Kürze. Die vollständigen Hinweise und die Ins
31
31
  Neustart des Computers startet, sagen `strom phone` und die Einstellungen, dass die Verbindung an ist und ihr Server
32
32
  mit der Forschung wieder startet.
33
33
  - **Korrekturen eigener Fehler von strom zur Entscheidung.** Findet strom, dass eine frühere Version von ihm in einer
34
- Forschung etwas falsch gemacht hat, sagen `strom fixes` und Einstellungen → Wartung → Korrekturen, worum es geht, was
35
- gefunden wurde und was eine Behebung täte, auch wie aufwendig sie ist; ohne Ja geschieht nichts, ein Nein bleibt.
36
- Die erste: blinde Lesungen aus dem verkleinerten Scan, wo es einen schärferen Teil gab — neben den alten noch einmal
37
- gelesen, zuerst an einigen, nichts überschrieben, ein Unterschied wird zur Aufgabe. Die zweite: nach einem Wechsel
38
- des Lesemodells oder Agenten die früheren Lesungen, die ein neues Lesen lohnen — mit offenen Stellen, uneinigen
39
- Lesern, einem offenen Widerspruch, an der Quelle nicht bestätigt; gesagt als „gelesen von Modell X, jetzt liest
40
- Modell Y“, einmal je Agent und Modell gefragt, jede so neu gelesen, wie sie gelesen wurde (die Bilder einer Lesung
41
- zusammen, ein Versuch oder ein Teil immer aus ganzen Lesungen). Gezählt in Lesungen mit ihren Stellen, der Aufwand
42
- in Leseaufrufen; angeboten nach Stufen — nur die strittigen (uneinige Leser, ein offener Widerspruch, an der Quelle
43
- nicht bestätigt), mit der direkten Linie oder alle —, jede mit dem, was sie kostet, die engste zuerst; Daten und
44
- Zeugen ordnen nur. Beheben kann auch ein anderer bereiter Agent oder ein anderes Modell
45
- (`strom fixes run <id> --agent <a> --model <m>`), mit dem, der vorher las; einer, der hier nicht lesen kann, wird
46
- sofort abgelehnt, mit denen, die es können. Die Zeit einer Behebung wird aufgerundet und nach dem Versuch nach den
47
- nebeneinander gelaufenen Lesungen neu gemessen.
34
+ Forschung etwas falsch gemacht hat, sagen `strom fixes` und Einstellungen → Wartung → Korrekturen, worum es geht,
35
+ was gefunden wurde und was eine Behebung täte, auch wie aufwendig sie ist; ohne Ja geschieht nichts, ein Nein
36
+ bleibt. Die erste: blinde Lesungen aus dem verkleinerten Scan, wo es einen schärferen Teil gab — neben den alten
37
+ noch einmal gelesen, zuerst an einigen, nichts überschrieben, ein Unterschied wird zur Aufgabe. Verglichen wird, was
38
+ gelesen wurde — die Namen, Daten und Zahlen der Abschrift, nicht die Worte des Kommentars —, und die Aufgabe sagt in
39
+ einem Satz, welche Angabe abweicht. Die zweite: nach einem Wechsel des Lesemodells oder Agenten die früheren
40
+ Lesungen, die ein neues Lesen lohnen — mit offenen Stellen, uneinigen Lesern, einem offenen Widerspruch, an der
41
+ Quelle nicht bestätigt; gesagt als „gelesen von Modell X, jetzt liest Modell Y“, einmal je Agent und Modell gefragt,
42
+ jede so neu gelesen, wie sie gelesen wurde (die Bilder einer Lesung zusammen, ein Versuch oder ein Teil immer aus
43
+ ganzen Lesungen). Gezählt in Lesungen mit ihren Stellen, der Aufwand in Leseaufrufen; angeboten nach Stufen — nur
44
+ die strittigen (uneinige Leser, ein offener Widerspruch, an der Quelle nicht bestätigt), mit der direkten Linie oder
45
+ alle —, jede mit dem, was sie kostet, die engste zuerst; Daten und Zeugen ordnen nur. Beheben kann auch ein anderer
46
+ bereiter Agent oder ein anderes Modell (`strom fixes run <id> --agent <a> --model <m>`), mit dem, der vorher las;
47
+ einer, der hier nicht lesen kann, wird sofort abgelehnt, mit denen, die es können. Die Zeit einer Behebung wird
48
+ aufgerundet und nach dem Versuch nach den nebeneinander gelaufenen Lesungen neu gemessen.
48
49
 
49
50
  ## 1.18.0
50
51
 
package/CHANGELOG.md CHANGED
@@ -7,7 +7,7 @@ The changes of each version in short. The full notes and the install lines are o
7
7
 
8
8
  ## 1.19.0
9
9
 
10
- *In preparation: the betas up to 1.19.0-beta.5. The section is complete with the release.*
10
+ *In preparation: the betas up to 1.19.0-beta.6. The section is complete with the release.*
11
11
 
12
12
  - **An archive that went silent is waited for, not asked again and again.** When an archive stops answering, strom
13
13
  leaves it alone for an hour — two, then four when it stays silent the same day — and the agent hears until when:
@@ -32,16 +32,17 @@ The changes of each version in short. The full notes and the install lines are o
32
32
  something wrong in a research, `strom fixes` and Settings → Maintenance → Fixes say what it is about, what was found
33
33
  and what a remedy would do, with how demanding it is; nothing happens without a yes, and a no is kept. The first:
34
34
  blind readings made from the reduced scan where a sharper part existed — read again beside the old ones, first on a
35
- few, nothing overwritten, a difference becomes a task. The second: after the reading model or agent changed, the
36
- earlier readings worth reading again — with open places, readers who disagreed, an open conflict, a reading not
37
- confirmed at the source; said as "read by model X, now read by model Y", asked once per agent and model, each read
38
- again as it was read (the images of one reading together, a trial or a part always of whole readings). Counted in
39
- readings with their places and what a remedy takes in reader calls; offered by stage — the disputed alone (readers
40
- who differed, an open conflict, not confirmed at the source), with the direct line, or all — each with what it
41
- takes, the narrowest first, dates and witnesses only ordering them. The remedy may be read by another agent or model
42
- ready here (`strom fixes run <id> --agent <a> --model <m>`), with which one read before said; one that cannot read
43
- here is refused at once, with those that can. The time of a remedy is rounded up and, after the trial, measured
44
- again by the readings as they ran side by side.
35
+ few, nothing overwritten, a difference becomes a task. Held against each other by what was read — the names, dates
36
+ and numbers of the transcript, not the words of the reader's comment — the task says in one sentence which of them
37
+ differs. The second: after the reading model or agent changed, the earlier readings worth reading again — with open
38
+ places, readers who disagreed, an open conflict, a reading not confirmed at the source; said as "read by model X,
39
+ now read by model Y", asked once per agent and model, each read again as it was read (the images of one reading
40
+ together, a trial or a part always of whole readings). Counted in readings with their places and what a remedy takes
41
+ in reader calls; offered by stage — the disputed alone (readers who differed, an open conflict, not confirmed at the
42
+ source), with the direct line, or all — each with what it takes, the narrowest first, dates and witnesses only
43
+ ordering them. The remedy may be read by another agent or model ready here (`strom fixes run <id> --agent <a>
44
+ --model <m>`), with which one read before said; one that cannot read here is refused at once, with those that can.
45
+ The time of a remedy is rounded up and, after the trial, measured again by the readings as they ran side by side.
45
46
 
46
47
  ## 1.18.0
47
48
 
@@ -2656,6 +2656,8 @@
2656
2656
  "fix.model.why.differ": "Přečteno znovu modelem {now} (dříve: {before}) tam, kde něco zůstalo otevřené, a čtení se liší.",
2657
2657
  "fix.model.why.refined": "Přečteno znovu modelem {now} (dříve: {before}) tam, kde něco zůstalo otevřené, a nové čtení přečetlo víc nebo jistěji.",
2658
2658
  "fix.model.done": "obě čtení porovnaná mezi sebou ({show} {old} · {show} {now}) a s {sources}: co platí, zapsané u pramene, údaje na něm ověřené; nic odebrané bez důvodu",
2659
+ "fix.differs": "Liší se: dříve {before}, nově {now}.",
2660
+ "fix.more": "Nově přečteno navíc: {now}.",
2659
2661
  "fix.model.unknown": "nezaznamenaný model",
2660
2662
  "fix.model.own": "vlastní model"
2661
2663
  }
@@ -2656,6 +2656,8 @@
2656
2656
  "fix.model.why.differ": "Von {now} erneut gelesen (zuvor: {before}), wo etwas offen blieb, und die Lesungen weichen ab.",
2657
2657
  "fix.model.why.refined": "Von {now} erneut gelesen (zuvor: {before}), wo etwas offen blieb, und die neue Lesung liest mehr oder sicherer.",
2658
2658
  "fix.model.done": "beide Lesungen miteinander verglichen ({show} {old} · {show} {now}) und mit {sources}: was gilt, an der Quelle vermerkt, die Angaben darauf geprüft; nichts ohne Grund entfernt",
2659
+ "fix.differs": "Abweichend: früher {before}, jetzt {now}.",
2660
+ "fix.more": "Jetzt zusätzlich gelesen: {now}.",
2659
2661
  "fix.model.unknown": "ein nicht vermerktes Modell",
2660
2662
  "fix.model.own": "eigenes Modell"
2661
2663
  }
@@ -39,6 +39,8 @@ export const BLIND_REDUCED_FIXED_AT = "2026-10-21T13:42:49Z";
39
39
  /** A view made for a reader this long before its reader began belongs to its reading. */
40
40
  const VIEW_BEFORE_MS = 30 * 60_000;
41
41
  const VIEW_AFTER_MS = 2 * 60_000;
42
+ /** strom makes a reader's views before it begins it: of a reader whose own start is known, only those (ms rounding). */
43
+ const VIEW_EXACT_MS = 100;
42
44
  /** Views of one reading are made within moments of each other. */
43
45
  const VIEW_CLUSTER_MS = 10_000;
44
46
  /** A later reading from a sharper copy that covers this much of a place has read it again. */
@@ -96,6 +98,9 @@ function blindReducedDetect(tree, env) {
96
98
  // when each reading began (its readers, this computer)
97
99
  const began = new Map();
98
100
  const capOf = new Map();
101
+ // when the reader of each image of a reading began, and every reader's start of an image (any reading)
102
+ const beganImage = new Map();
103
+ const startsOf = new Map();
99
104
  for (const r of readJournal(path.join(metricsDir(tree.root), "readers.jsonl"))) {
100
105
  if (r.kind !== "read" || typeof r.reader !== "string" || typeof r.at !== "string")
101
106
  continue;
@@ -105,6 +110,17 @@ function blindReducedDetect(tree, env) {
105
110
  began.set(stem, t);
106
111
  if (num(r.cap))
107
112
  capOf.set(stem, num(r.cap));
113
+ if (!Number.isFinite(t))
114
+ continue;
115
+ // the images it read (an older strom's record: those its reading is named after)
116
+ for (const id of Array.isArray(r.imageIds) ? r.imageIds : (stem.match(/M\d+/gu) ?? [])) {
117
+ if (typeof id !== "string")
118
+ continue;
119
+ const k = `${stem}|${id}`;
120
+ if (t < (beganImage.get(k) ?? Infinity))
121
+ beganImage.set(k, t);
122
+ startsOf.set(id, [...(startsOf.get(id) ?? []), t]);
123
+ }
108
124
  }
109
125
  // the views made for readers (this computer): a crop, the image it is of, when
110
126
  const views = new Map();
@@ -132,13 +148,23 @@ function blindReducedDetect(tree, env) {
132
148
  const times = r.reports.map((n) => added.get(n)).filter((t) => t !== undefined);
133
149
  return times.length ? Math.min(...times) : undefined;
134
150
  };
135
- /** The views of one reading of an image: those made last before its reader began, within moments of each other. */
136
- const viewsOf = (key, t) => {
137
- const near = (views.get(key) ?? []).filter((v) => v.t >= t - VIEW_BEFORE_MS && v.t <= t + VIEW_AFTER_MS);
151
+ /**
152
+ * The views of one reading of an image: those made last before its reader began, within moments of each other —
153
+ * never those of another reader of the same image (made before it began). Its reader's own start unknown here and
154
+ * views of more than one moment near: which it read is not sure (an estimate).
155
+ */
156
+ const viewsOf = (stem, key, t) => {
157
+ const own = beganImage.get(`${stem}|${key}`);
158
+ const at = own ?? t;
159
+ const others = (startsOf.get(key) ?? []).filter((s) => s !== own);
160
+ const after = Math.min(at + (own !== undefined ? VIEW_EXACT_MS : VIEW_AFTER_MS), ...others.filter((s) => s > at));
161
+ const before = Math.max(at - VIEW_BEFORE_MS, ...others.filter((s) => s < at));
162
+ const near = (views.get(key) ?? []).filter((v) => v.t > before && v.t <= after);
138
163
  if (!near.length)
139
- return [];
164
+ return { views: [], estimated: false };
140
165
  const last = Math.max(...near.map((v) => v.t));
141
- return near.filter((v) => v.t >= last - VIEW_CLUSTER_MS);
166
+ const cluster = near.filter((v) => v.t >= last - VIEW_CLUSTER_MS);
167
+ return { views: cluster, estimated: own === undefined && cluster.length < near.length };
142
168
  };
143
169
  // places read again from a sharper copy since (a later reading): by whole image, fractions as the file lies
144
170
  const readSharper = new Map();
@@ -167,7 +193,7 @@ function blindReducedDetect(tree, env) {
167
193
  const m = byId.get(id);
168
194
  if (!m || m.retracted)
169
195
  continue;
170
- const own = viewsOf(id, t);
196
+ const { views: own, estimated: unsure } = viewsOf(r.stem, id, t);
171
197
  if (m.part) {
172
198
  // a part read: the place in the whole image read sharper (when its views say where)
173
199
  if (m.recordset !== undefined && m.image !== undefined) {
@@ -189,7 +215,7 @@ function blindReducedDetect(tree, env) {
189
215
  const x1 = Math.max(...own.map((v) => v.region.x + v.region.w));
190
216
  const y1 = Math.max(...own.map((v) => v.region.y + v.region.h));
191
217
  const v = own[0];
192
- places = [{ file: shownToFile({ x: x0 / v.W, y: y0 / v.H, w: (x1 - x0) / v.W, h: (y1 - y0) / v.H }, v.orient), estimated: false }];
218
+ places = [{ file: shownToFile({ x: x0 / v.W, y: y0 / v.H, w: (x1 - x0) / v.W, h: (y1 - y0) / v.H }, v.orient), estimated: unsure }];
193
219
  }
194
220
  else if ((views.size === 0 || !views.has(id)) && r.blind && few && t < Date.parse(BLIND_REDUCED_FIXED_AT)) {
195
221
  // no record of its views here (another computer): the places of the sources' entries on the image, an estimate
@@ -504,7 +530,7 @@ function newModelDetect(tree, env) {
504
530
  const unclear = blocks.reduce((n, b) => n + Math.max(unclearOf(b), b.result === "unclear" ? 1 : 0), 0);
505
531
  // a blind reading that reads otherwise than the researcher's reading of the image before it
506
532
  const theirs = r.blind ? list.filter((x) => !x.r.blind && x.t <= t && x.r !== r).sort((a, b) => b.t - a.t)[0] : undefined;
507
- const differ = !!theirs && compareReadings(theirs.blocks, blocks) === "differ";
533
+ const differ = !!theirs && compareReadings(theirs.blocks, blocks).result === "differ";
508
534
  const reg = cropRegion(crop);
509
535
  const src = reg && !m.part ? sourcesOf(sources, media, m, shownToFile(reg, mediaOrient(env.shared, m))) : sources.filter((s) => (s.media ?? []).includes(id) || (s.clips ?? []).some((c) => c.media === id)).map((s) => s.id).sort();
510
536
  const reasons = [...(unclear ? ["unclear"] : []), ...(differ ? ["differ"] : []), ...(src.some((s) => conflicted.has(s)) ? ["conflict"] : [])];
@@ -671,13 +697,16 @@ export function decide(tree, id, decision) {
671
697
  delete c.fixes;
672
698
  }, { op: "fix.decide", summary: decision ? `fix ${id}: ${decision}` : `fix ${id}: asked again` });
673
699
  }
674
- /** Each check with what it found here and the decision about it. */
700
+ /**
701
+ * Each check with what it found here and the decision about it. Set right holds for what was read again: what a check
702
+ * still finds after (never read again — a few first, or a place strom took wrongly then) waits again, it alone.
703
+ */
675
704
  export function fixStates(tree, env) {
676
705
  return FIX_CHECKS.map((check) => {
677
706
  const found = check.detect(tree, env);
678
707
  const key = decisionKeyOf(tree, check, env);
679
708
  const decision = decisionOf(tree.config, key);
680
- const state = decision ? decision.decision : found.items.length ? "pending" : "nothing";
709
+ const state = decision?.decision === "declined" ? "declined" : found.items.length ? "pending" : decision ? "fixed" : "nothing";
681
710
  return { check, key, found, ...(decision ? { decision } : {}), state };
682
711
  });
683
712
  }
@@ -719,39 +748,94 @@ export function estimateFix(root, o) {
719
748
  }
720
749
  return { level: n >= COST_COUNT.expensive ? "expensive" : n >= COST_COUNT.medium ? "medium" : "cheap", calls: n, minutes, basis: "count", earlier: 0 };
721
750
  }
722
- /** The words of a text, folded, without the marks of uncertainty. */
723
- function words(text) {
724
- return new Set(foldText(text.replace(/\[\?\]/gu, " ")).split(/[^\p{L}\p{M}\p{N}]+/u).filter((w) => w.length > 1));
751
+ // a reader's quotation marks: what it quotes of the record is its transcript, the rest its comment
752
+ const QUOTED = /[„“”"«»「」『』]([^„“”"«»「」『』]*)[„“”"«»「」『』]/gu;
753
+ /** A word as compared: folded, the long s and the ezh of old hands as the letters they are. */
754
+ const wordKey = (w) => foldText(w).replace(/ſ/gu, "s").replace(/ʒ/gu, "z");
755
+ /**
756
+ * The facts of a transcript: the names (a word that begins with a capital, or any word of a script without capitals)
757
+ * and the numbers; a word broken over a line ("Wac- / lawa", "In= / mann", "Wac|lawa") whole, the marks of doubt and an editor's
758
+ * notes in brackets left out (a letter supplied in brackets, "Mi[cha]la", kept). Of a reader's own words (prose: nothing
759
+ * quoted), a capital that only begins a sentence ("The crop shows …", "Výřez ukazuje …") is no name.
760
+ */
761
+ function readOf(text, unclear, found, prose = false) {
762
+ const clean = text
763
+ .replace(/\[\?\]|\(\?\)/gu, "")
764
+ .replace(/\[[^\]]*\s[^\]]*\]/gu, " ")
765
+ .replace(/[[\]]/gu, "")
766
+ .replace(/([\p{L}\p{M}])(?:[-=¬]\s*[/|]?\s*|\|)(?=[\p{L}\p{M}])/gu, "$1");
767
+ const facts = new Map();
768
+ const words = new Set();
769
+ for (const m of clean.matchAll(/[\p{L}\p{M}\p{N}]+/gu)) {
770
+ const w = m[0];
771
+ const k = wordKey(w);
772
+ words.add(k);
773
+ const digits = /\p{N}/u.test(w);
774
+ const cased = /[\p{Lu}\p{Ll}]/u.test(w);
775
+ if (!digits && cased && prose && /^\p{Lu}/u.test(w) && opensSentence(clean, m.index, w.length))
776
+ continue;
777
+ if (digits || (w.length > 1 && (!cased || /^\p{Lu}/u.test(w))))
778
+ if (!facts.has(k))
779
+ facts.set(k, w);
780
+ }
781
+ return { facts, words, unclear, found };
782
+ }
783
+ /**
784
+ * A word that begins a sentence of prose: at a line's start or after a sentence's end (never after a number's dot, "5. Mai"),
785
+ * the next word of its sentence in small letters ("Jan Novak", "Jan, son of" stay names).
786
+ */
787
+ function opensSentence(text, at, length) {
788
+ let i = at - 1;
789
+ while (i >= 0 && /\s/u.test(text[i]) && text[i] !== "\n")
790
+ i--;
791
+ const start = i < 0 || text[i] === "\n" || (/[.!?…]/u.test(text[i]) && i > 0 && !/\p{N}/u.test(text[i - 1]));
792
+ return start && /^\s+\p{Ll}/u.test(text.slice(at + length));
793
+ }
794
+ /**
795
+ * What a reader read of its blocks: the parts of its entries it quotes from the record (when it quotes any), else its
796
+ * entries — never a line of possible readings with their chances or of letters with theirs (a "%" in it), nor how it
797
+ * describes the hand or how sure it is (hand, certainty); what it could not read only counts as unclear.
798
+ */
799
+ function readOfBlocks(blocks) {
800
+ const lines = blocks.flatMap((b) => b.entries ?? []).filter((l) => !/%/u.test(l));
801
+ const quoted = lines.flatMap((l) => [...l.matchAll(QUOTED)].map((m) => m[1]));
802
+ const unclear = blocks.reduce((n, x) => n + unclearOf(x), 0);
803
+ const found = blocks.some((b) => b.result === "found");
804
+ return quoted.length ? readOf(quoted.join(" "), unclear, found) : readOf(lines.join("\n"), unclear, found, true);
805
+ }
806
+ /** What a reading of a document read: the people's names and ages, the events' dates and places (the reader's own fields). */
807
+ function readOfDocument(r) {
808
+ const { unclear } = docText(r);
809
+ const parts = r.entries.flatMap((e) => [...e.persons.flatMap((p) => [p.given, p.surname, p.age]), ...e.events.flatMap((v) => [v.date?.gedcom ?? v.date?.text, v.place?.text])]);
810
+ // every word of its fields counts: a document's reader says what each one is
811
+ const read = readOf(parts.filter(Boolean).join(" "), unclear, r.entries.length > 0);
812
+ for (const w of read.words)
813
+ if (!read.facts.has(w))
814
+ read.facts.set(w, w);
815
+ return read;
725
816
  }
726
- /** A reader's entries as one text, and its uncertain places. */
727
- const readText = (blocks) => ({ text: blocks.map((b) => (b.entries?.length ? b.entries.join(" ") : b.text)).join(" "), unclear: blocks.reduce((n, x) => n + unclearOf(x), 0) });
728
817
  /**
729
- * The new reading of a place against the earlier one: the same words (shoda), the same with fewer places unclear or
730
- * more read (upřesněno), or other words (rozchod) — nothing read: failed.
818
+ * The new reading of a place against the earlier one, by what each read — the names and numbers of its transcript:
819
+ * the same (shoda); all the earlier one read with more beside it, fewer places unclear, or found where it was unclear
820
+ * (upřesněno); something the earlier one read that the new one does not — another name, date, number or place (rozchod).
821
+ * Nothing read: failed.
731
822
  */
732
823
  export function compareReadings(before, now) {
733
- return now.length ? compareTexts(readText(before), readText(now)) : "failed";
824
+ return now.length ? compareRead(readOfBlocks(before), readOfBlocks(now)) : { result: "failed", before: [], now: [] };
734
825
  }
735
- /** A reading of a document again against the earlier one, as compareReadings (its entries' transcripts). */
826
+ /** A reading of a document again against the earlier one, as compareReadings (its people and events). */
736
827
  export function compareDocuments(before, now) {
737
828
  if (!now || !now.entries.length)
738
- return "failed";
739
- return compareTexts(docText(before), docText(now));
740
- }
741
- function compareTexts(before, now) {
742
- const a = words(before.text);
743
- const b = words(now.text);
744
- const both = [...a].filter((w) => b.has(w)).length;
745
- const all = new Set([...a, ...b]).size;
746
- const alike = all ? both / all : 1;
747
- const fewer = now.unclear < before.unclear;
748
- // what the earlier reading had, the new one has too, with more beside it: read further
749
- const more = a.size > 0 && both / a.size >= 0.85 && b.size > a.size;
750
- if (alike >= 0.9)
751
- return fewer ? "refined" : "same";
752
- if ((alike >= 0.6 && fewer) || more)
753
- return "refined";
754
- return "differ";
829
+ return { result: "failed", before: [], now: [] };
830
+ return compareRead(readOfDocument(before), readOfDocument(now));
831
+ }
832
+ function compareRead(a, b) {
833
+ const before = [...a.facts].filter(([k]) => !b.words.has(k)).map(([, w]) => w);
834
+ const now = [...b.facts].filter(([k]) => !a.words.has(k)).map(([, w]) => w);
835
+ if (before.length)
836
+ return { result: "differ", before, now };
837
+ const surer = b.unclear < a.unclear || (b.found && !a.found);
838
+ return { result: now.length || surer ? "refined" : "same", before, now };
755
839
  }
756
840
  // ── what waits ──────────────────────────────────────────────────────────────────────────────────────────────────
757
841
  /** The checks that wait for the person's decision here (none in an archive: nobody reads there). */
@@ -202,6 +202,12 @@ function blocksOf(tree, stem, media) {
202
202
  const r = loadReadings(tree).find((x) => x.stem === stem);
203
203
  return (r?.blocks ?? []).filter((b) => b.media === media.id || (b.recordset === media.recordset && b.num !== undefined && b.num === media.image));
204
204
  }
205
+ /** A comparison as a reading again keeps it: its result, and what differs where something does. */
206
+ function compared(c) {
207
+ if (!c)
208
+ return { result: "failed" };
209
+ return { result: c.result, ...(c.before.length || c.now.length ? { differs: { before: c.before, now: c.now } } : {}) };
210
+ }
205
211
  /** Each place read again, blind with the earlier question, from the sharper copy; the readings held against each other. */
206
212
  async function readAgain(ctx, tree, t, items, model) {
207
213
  const { def } = resolveCommand(["read"]);
@@ -231,7 +237,8 @@ async function readAgain(ctx, tree, t, items, model) {
231
237
  }
232
238
  const before = loadDocResult(tree.root, item.reading);
233
239
  const secs = (Date.now() - began) / 1000;
234
- return [{ item, result: before ? compareDocuments(before, now) : "failed", ...(now?.id ? { reading: now.id } : {}), secs, ...(error ? { error } : {}) }];
240
+ const c = before ? compareDocuments(before, now) : undefined;
241
+ return [{ item, ...compared(c), ...(now?.id ? { reading: now.id } : {}), secs, ...(error ? { error } : {}) }];
235
242
  }
236
243
  const opts = { question: item.question, ...(model ? { model } : {}), ...(item.crop ? { crop: item.crop } : {}), parallel: "1", ...(item.blind ? { blind: true } : {}), ...(item.context ? { context: item.context } : {}) };
237
244
  let reading;
@@ -249,7 +256,7 @@ async function readAgain(ctx, tree, t, items, model) {
249
256
  const m = byId.get(i.media);
250
257
  const now = reading && m ? blocksOf(tree, reading, m) : [];
251
258
  const before = m ? blocksOf(tree, i.reading, m) : [];
252
- return { item: i, result: compareReadings(before, now), ...(reading ? { reading } : {}), secs, ...(error ? { error } : {}) };
259
+ return { item: i, ...compared(compareReadings(before, now)), ...(reading ? { reading } : {}), secs, ...(error ? { error } : {}) };
253
260
  });
254
261
  })).flat();
255
262
  }
@@ -267,7 +274,7 @@ function tasksFor(tree, check, done, now) {
267
274
  priority: d.result === "differ" ? 2 : 3,
268
275
  what: phrase(lang, `${p}.what`, { media: i.document ? i.document.files.join(", ") : i.media, old: i.reading, now: d.reading }),
269
276
  where: [...(i.document ? i.document.files : [i.media]), ...i.sources],
270
- why: phrase(lang, `${p}.why.${d.result}`, { since: check.since, from: i.from ?? "", before: i.before ? label(i.before) : "", now: now ? label(now) : "" }),
277
+ why: [phrase(lang, `${p}.why.${d.result}`, { since: check.since, from: i.from ?? "", before: i.before ? label(i.before) : "", now: now ? label(now) : "" }), differsText(lang, d)].filter(Boolean).join(" "),
271
278
  doneWhen: phrase(lang, `${p}.done`, { old: i.reading, now: d.reading, sources: i.sources.join(", ") || i.media, show: i.document ? "strom document show" : "strom readings" }),
272
279
  subject: [...i.sources],
273
280
  state: "open",
@@ -276,6 +283,15 @@ function tasksFor(tree, check, done, now) {
276
283
  d.task = task.id;
277
284
  }
278
285
  }
286
+ /** One sentence of what differs (the facts each read that the other does not), or what the new reading read beside. */
287
+ function differsText(lang, d) {
288
+ const list = (ws) => (ws.length ? ws.slice(0, 6).join(", ") + (ws.length > 6 ? ", …" : "") : "—");
289
+ if (!d.differs)
290
+ return "";
291
+ if (d.result === "differ")
292
+ return phrase(lang, "fix.differs", { before: list(d.differs.before), now: list(d.differs.now) });
293
+ return d.differs.now.length ? phrase(lang, "fix.more", { now: list(d.differs.now) }) : "";
294
+ }
279
295
  /** `secs`: the real time the readings took, readers at once and all. Its places: the same, refined, different. */
280
296
  function summaryText(t, done, secs) {
281
297
  const n = (r) => done.filter((d) => d.result === r).length;
@@ -111,6 +111,8 @@ export const PHRASES = {
111
111
  "fix.model.why.differ": "Read again by {now} (earlier: {before}) where something stayed open, and the readings differ.",
112
112
  "fix.model.why.refined": "Read again by {now} (earlier: {before}) where something stayed open, and the new reading reads more or is surer.",
113
113
  "fix.model.done": "both readings held against each other ({show} {old} · {show} {now}) and against {sources}: what holds kept on the source, the facts on it checked; nothing taken away without a reason",
114
+ "fix.differs": "What differs: earlier {before}, now {now}.",
115
+ "fix.more": "Read now beside the earlier: {now}.",
114
116
  "fix.model.unknown": "a model not recorded",
115
117
  "fix.model.own": "its own model",
116
118
  "batch.unnamed": "material from the Strom app",
package/dist/core/tree.js CHANGED
@@ -27,7 +27,7 @@ import { lastBackup } from "./backup.js";
27
27
  export const TREE_FILE = "strom.json";
28
28
  // between releases the candidate being made (a bridge run from the sources says what it is: the Strom app goes by it);
29
29
  // at a release the release's own (package.json)
30
- export const VERSION = "1.19.0-beta.5";
30
+ export const VERSION = "1.19.0-beta.6";
31
31
  const GITIGNORE = `# Strom Research — local state and large files are not versioned
32
32
  .strom/
33
33
  /media/
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "strom-research",
3
- "version": "1.19.0-beta.5",
3
+ "version": "1.19.0-beta.6",
4
4
  "description": "Genealogical research toolkit for AI coding agents",
5
5
  "type": "module",
6
6
  "bin": {