strom-research 1.19.0-beta.5 → 1.19.0-beta.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.cs.md +11 -10
- package/CHANGELOG.de.md +16 -15
- package/CHANGELOG.md +12 -11
- package/assets/lang/cs.json +2 -0
- package/assets/lang/de.json +2 -0
- package/dist/commands/fixchecks.js +120 -36
- package/dist/commands/fixes.js +19 -3
- package/dist/core/phrases.js +2 -0
- package/dist/core/tree.js +1 -1
- package/package.json +1 -1
package/CHANGELOG.cs.md
CHANGED
|
@@ -7,7 +7,7 @@ Změny každé verze ve zkratce. Úplné poznámky a instalační řádky jsou n
|
|
|
7
7
|
|
|
8
8
|
## 1.19.0
|
|
9
9
|
|
|
10
|
-
*Připravuje se: bety do 1.19.0-beta.
|
|
10
|
+
*Připravuje se: bety do 1.19.0-beta.6. Oddíl bude úplný s vydáním.*
|
|
11
11
|
|
|
12
12
|
- **Na archiv, který zmlkl, se čeká, místo opakovaných dotazů.** Když archiv přestane odpovídat, strom ho nechá na
|
|
13
13
|
pokoji hodinu — a když týž den mlčí dál, dvě a pak čtyři hodiny — a agent se dozví, do kdy: mezitím dělá
|
|
@@ -30,15 +30,16 @@ Změny každé verze ve zkratce. Úplné poznámky a instalační řádky jsou n
|
|
|
30
30
|
udělala špatně, `strom fixes` a Nastavení → Údržba → Opravy řeknou, o co jde, co se našlo a co by napravení udělalo,
|
|
31
31
|
i jak je náročné; bez souhlasu se nestane nic a odmítnutí platí. První: slepá čtení ze zmenšeného snímku tam, kde
|
|
32
32
|
byla ostřejší část — přečtou se znovu vedle starých, nejdřív na několika, nic se nepřepíše, rozdíl se stane úkolem.
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
33
|
+
Čtení se porovnají podle přečteného — jmen, dat a čísel přepisu, ne slov komentáře čtenáře — a úkol jednou větou
|
|
34
|
+
řekne, který údaj se liší. Druhá: po změně modelu nebo agenta pro čtení dřívější čtení, která stojí za nové přečtení
|
|
35
|
+
— s otevřenými místy, s neshodou čtenářů, s otevřeným rozporem, nepotvrzená u pramene; řečeno jako „četl model X,
|
|
36
|
+
teď se čte modelem Y“, nabídnuto jednou na agenta a model, každé se přečte znovu tak, jak se četlo (snímky jednoho
|
|
37
|
+
čtení spolu, zkouška i část vždy z celých čtení). Počítá se v čteních s jejich místy a náročnost ve voláních
|
|
38
|
+
čtenáře; nabízí se po stupních — jen sporná (rozchod čtenářů, otevřený rozpor, nepotvrzené u pramene), s přímou
|
|
39
|
+
linií, nebo všechna — každý s tím, co stojí, nejužší první; data a svědci jen řadí. Napravit může i jiný připravený
|
|
40
|
+
agent nebo model (`strom fixes run <id> --agent <a> --model <m>`), s tím, kdo četl dřív; ten, kdo tu číst nemůže, se
|
|
41
|
+
odmítne hned i s těmi, kdo mohou. Čas napravení se zaokrouhluje nahoru a po zkoušce se přepočítá podle čtení, jak
|
|
42
|
+
běžela souběžně.
|
|
42
43
|
|
|
43
44
|
## 1.18.0
|
|
44
45
|
|
package/CHANGELOG.de.md
CHANGED
|
@@ -7,7 +7,7 @@ Die Änderungen jeder Version in Kürze. Die vollständigen Hinweise und die Ins
|
|
|
7
7
|
|
|
8
8
|
## 1.19.0
|
|
9
9
|
|
|
10
|
-
*In Vorbereitung: die Betas bis 1.19.0-beta.
|
|
10
|
+
*In Vorbereitung: die Betas bis 1.19.0-beta.6. Der Abschnitt ist mit dem Release vollständig.*
|
|
11
11
|
|
|
12
12
|
- **Auf ein verstummtes Archiv wird gewartet, statt es immer wieder zu fragen.** Antwortet ein Archiv nicht mehr,
|
|
13
13
|
lässt strom es eine Stunde in Ruhe — zwei, dann vier, wenn es am selben Tag still bleibt — und der Agent erfährt, bis
|
|
@@ -31,20 +31,21 @@ Die Änderungen jeder Version in Kürze. Die vollständigen Hinweise und die Ins
|
|
|
31
31
|
Neustart des Computers startet, sagen `strom phone` und die Einstellungen, dass die Verbindung an ist und ihr Server
|
|
32
32
|
mit der Forschung wieder startet.
|
|
33
33
|
- **Korrekturen eigener Fehler von strom zur Entscheidung.** Findet strom, dass eine frühere Version von ihm in einer
|
|
34
|
-
Forschung etwas falsch gemacht hat, sagen `strom fixes` und Einstellungen → Wartung → Korrekturen, worum es geht,
|
|
35
|
-
gefunden wurde und was eine Behebung täte, auch wie aufwendig sie ist; ohne Ja geschieht nichts, ein Nein
|
|
36
|
-
Die erste: blinde Lesungen aus dem verkleinerten Scan, wo es einen schärferen Teil gab — neben den alten
|
|
37
|
-
gelesen, zuerst an einigen, nichts überschrieben, ein Unterschied wird zur Aufgabe.
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
34
|
+
Forschung etwas falsch gemacht hat, sagen `strom fixes` und Einstellungen → Wartung → Korrekturen, worum es geht,
|
|
35
|
+
was gefunden wurde und was eine Behebung täte, auch wie aufwendig sie ist; ohne Ja geschieht nichts, ein Nein
|
|
36
|
+
bleibt. Die erste: blinde Lesungen aus dem verkleinerten Scan, wo es einen schärferen Teil gab — neben den alten
|
|
37
|
+
noch einmal gelesen, zuerst an einigen, nichts überschrieben, ein Unterschied wird zur Aufgabe. Verglichen wird, was
|
|
38
|
+
gelesen wurde — die Namen, Daten und Zahlen der Abschrift, nicht die Worte des Kommentars —, und die Aufgabe sagt in
|
|
39
|
+
einem Satz, welche Angabe abweicht. Die zweite: nach einem Wechsel des Lesemodells oder Agenten die früheren
|
|
40
|
+
Lesungen, die ein neues Lesen lohnen — mit offenen Stellen, uneinigen Lesern, einem offenen Widerspruch, an der
|
|
41
|
+
Quelle nicht bestätigt; gesagt als „gelesen von Modell X, jetzt liest Modell Y“, einmal je Agent und Modell gefragt,
|
|
42
|
+
jede so neu gelesen, wie sie gelesen wurde (die Bilder einer Lesung zusammen, ein Versuch oder ein Teil immer aus
|
|
43
|
+
ganzen Lesungen). Gezählt in Lesungen mit ihren Stellen, der Aufwand in Leseaufrufen; angeboten nach Stufen — nur
|
|
44
|
+
die strittigen (uneinige Leser, ein offener Widerspruch, an der Quelle nicht bestätigt), mit der direkten Linie oder
|
|
45
|
+
alle —, jede mit dem, was sie kostet, die engste zuerst; Daten und Zeugen ordnen nur. Beheben kann auch ein anderer
|
|
46
|
+
bereiter Agent oder ein anderes Modell (`strom fixes run <id> --agent <a> --model <m>`), mit dem, der vorher las;
|
|
47
|
+
einer, der hier nicht lesen kann, wird sofort abgelehnt, mit denen, die es können. Die Zeit einer Behebung wird
|
|
48
|
+
aufgerundet und nach dem Versuch nach den nebeneinander gelaufenen Lesungen neu gemessen.
|
|
48
49
|
|
|
49
50
|
## 1.18.0
|
|
50
51
|
|
package/CHANGELOG.md
CHANGED
|
@@ -7,7 +7,7 @@ The changes of each version in short. The full notes and the install lines are o
|
|
|
7
7
|
|
|
8
8
|
## 1.19.0
|
|
9
9
|
|
|
10
|
-
*In preparation: the betas up to 1.19.0-beta.
|
|
10
|
+
*In preparation: the betas up to 1.19.0-beta.6. The section is complete with the release.*
|
|
11
11
|
|
|
12
12
|
- **An archive that went silent is waited for, not asked again and again.** When an archive stops answering, strom
|
|
13
13
|
leaves it alone for an hour — two, then four when it stays silent the same day — and the agent hears until when:
|
|
@@ -32,16 +32,17 @@ The changes of each version in short. The full notes and the install lines are o
|
|
|
32
32
|
something wrong in a research, `strom fixes` and Settings → Maintenance → Fixes say what it is about, what was found
|
|
33
33
|
and what a remedy would do, with how demanding it is; nothing happens without a yes, and a no is kept. The first:
|
|
34
34
|
blind readings made from the reduced scan where a sharper part existed — read again beside the old ones, first on a
|
|
35
|
-
few, nothing overwritten, a difference becomes a task.
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
35
|
+
few, nothing overwritten, a difference becomes a task. Held against each other by what was read — the names, dates
|
|
36
|
+
and numbers of the transcript, not the words of the reader's comment — the task says in one sentence which of them
|
|
37
|
+
differs. The second: after the reading model or agent changed, the earlier readings worth reading again — with open
|
|
38
|
+
places, readers who disagreed, an open conflict, a reading not confirmed at the source; said as "read by model X,
|
|
39
|
+
now read by model Y", asked once per agent and model, each read again as it was read (the images of one reading
|
|
40
|
+
together, a trial or a part always of whole readings). Counted in readings with their places and what a remedy takes
|
|
41
|
+
in reader calls; offered by stage — the disputed alone (readers who differed, an open conflict, not confirmed at the
|
|
42
|
+
source), with the direct line, or all — each with what it takes, the narrowest first, dates and witnesses only
|
|
43
|
+
ordering them. The remedy may be read by another agent or model ready here (`strom fixes run <id> --agent <a>
|
|
44
|
+
--model <m>`), with which one read before said; one that cannot read here is refused at once, with those that can.
|
|
45
|
+
The time of a remedy is rounded up and, after the trial, measured again by the readings as they ran side by side.
|
|
45
46
|
|
|
46
47
|
## 1.18.0
|
|
47
48
|
|
package/assets/lang/cs.json
CHANGED
|
@@ -2656,6 +2656,8 @@
|
|
|
2656
2656
|
"fix.model.why.differ": "Přečteno znovu modelem {now} (dříve: {before}) tam, kde něco zůstalo otevřené, a čtení se liší.",
|
|
2657
2657
|
"fix.model.why.refined": "Přečteno znovu modelem {now} (dříve: {before}) tam, kde něco zůstalo otevřené, a nové čtení přečetlo víc nebo jistěji.",
|
|
2658
2658
|
"fix.model.done": "obě čtení porovnaná mezi sebou ({show} {old} · {show} {now}) a s {sources}: co platí, zapsané u pramene, údaje na něm ověřené; nic odebrané bez důvodu",
|
|
2659
|
+
"fix.differs": "Liší se: dříve {before}, nově {now}.",
|
|
2660
|
+
"fix.more": "Nově přečteno navíc: {now}.",
|
|
2659
2661
|
"fix.model.unknown": "nezaznamenaný model",
|
|
2660
2662
|
"fix.model.own": "vlastní model"
|
|
2661
2663
|
}
|
package/assets/lang/de.json
CHANGED
|
@@ -2656,6 +2656,8 @@
|
|
|
2656
2656
|
"fix.model.why.differ": "Von {now} erneut gelesen (zuvor: {before}), wo etwas offen blieb, und die Lesungen weichen ab.",
|
|
2657
2657
|
"fix.model.why.refined": "Von {now} erneut gelesen (zuvor: {before}), wo etwas offen blieb, und die neue Lesung liest mehr oder sicherer.",
|
|
2658
2658
|
"fix.model.done": "beide Lesungen miteinander verglichen ({show} {old} · {show} {now}) und mit {sources}: was gilt, an der Quelle vermerkt, die Angaben darauf geprüft; nichts ohne Grund entfernt",
|
|
2659
|
+
"fix.differs": "Abweichend: früher {before}, jetzt {now}.",
|
|
2660
|
+
"fix.more": "Jetzt zusätzlich gelesen: {now}.",
|
|
2659
2661
|
"fix.model.unknown": "ein nicht vermerktes Modell",
|
|
2660
2662
|
"fix.model.own": "eigenes Modell"
|
|
2661
2663
|
}
|
|
@@ -39,6 +39,8 @@ export const BLIND_REDUCED_FIXED_AT = "2026-10-21T13:42:49Z";
|
|
|
39
39
|
/** A view made for a reader this long before its reader began belongs to its reading. */
|
|
40
40
|
const VIEW_BEFORE_MS = 30 * 60_000;
|
|
41
41
|
const VIEW_AFTER_MS = 2 * 60_000;
|
|
42
|
+
/** strom makes a reader's views before it begins it: of a reader whose own start is known, only those (ms rounding). */
|
|
43
|
+
const VIEW_EXACT_MS = 100;
|
|
42
44
|
/** Views of one reading are made within moments of each other. */
|
|
43
45
|
const VIEW_CLUSTER_MS = 10_000;
|
|
44
46
|
/** A later reading from a sharper copy that covers this much of a place has read it again. */
|
|
@@ -96,6 +98,9 @@ function blindReducedDetect(tree, env) {
|
|
|
96
98
|
// when each reading began (its readers, this computer)
|
|
97
99
|
const began = new Map();
|
|
98
100
|
const capOf = new Map();
|
|
101
|
+
// when the reader of each image of a reading began, and every reader's start of an image (any reading)
|
|
102
|
+
const beganImage = new Map();
|
|
103
|
+
const startsOf = new Map();
|
|
99
104
|
for (const r of readJournal(path.join(metricsDir(tree.root), "readers.jsonl"))) {
|
|
100
105
|
if (r.kind !== "read" || typeof r.reader !== "string" || typeof r.at !== "string")
|
|
101
106
|
continue;
|
|
@@ -105,6 +110,17 @@ function blindReducedDetect(tree, env) {
|
|
|
105
110
|
began.set(stem, t);
|
|
106
111
|
if (num(r.cap))
|
|
107
112
|
capOf.set(stem, num(r.cap));
|
|
113
|
+
if (!Number.isFinite(t))
|
|
114
|
+
continue;
|
|
115
|
+
// the images it read (an older strom's record: those its reading is named after)
|
|
116
|
+
for (const id of Array.isArray(r.imageIds) ? r.imageIds : (stem.match(/M\d+/gu) ?? [])) {
|
|
117
|
+
if (typeof id !== "string")
|
|
118
|
+
continue;
|
|
119
|
+
const k = `${stem}|${id}`;
|
|
120
|
+
if (t < (beganImage.get(k) ?? Infinity))
|
|
121
|
+
beganImage.set(k, t);
|
|
122
|
+
startsOf.set(id, [...(startsOf.get(id) ?? []), t]);
|
|
123
|
+
}
|
|
108
124
|
}
|
|
109
125
|
// the views made for readers (this computer): a crop, the image it is of, when
|
|
110
126
|
const views = new Map();
|
|
@@ -132,13 +148,23 @@ function blindReducedDetect(tree, env) {
|
|
|
132
148
|
const times = r.reports.map((n) => added.get(n)).filter((t) => t !== undefined);
|
|
133
149
|
return times.length ? Math.min(...times) : undefined;
|
|
134
150
|
};
|
|
135
|
-
/**
|
|
136
|
-
|
|
137
|
-
|
|
151
|
+
/**
|
|
152
|
+
* The views of one reading of an image: those made last before its reader began, within moments of each other —
|
|
153
|
+
* never those of another reader of the same image (made before it began). Its reader's own start unknown here and
|
|
154
|
+
* views of more than one moment near: which it read is not sure (an estimate).
|
|
155
|
+
*/
|
|
156
|
+
const viewsOf = (stem, key, t) => {
|
|
157
|
+
const own = beganImage.get(`${stem}|${key}`);
|
|
158
|
+
const at = own ?? t;
|
|
159
|
+
const others = (startsOf.get(key) ?? []).filter((s) => s !== own);
|
|
160
|
+
const after = Math.min(at + (own !== undefined ? VIEW_EXACT_MS : VIEW_AFTER_MS), ...others.filter((s) => s > at));
|
|
161
|
+
const before = Math.max(at - VIEW_BEFORE_MS, ...others.filter((s) => s < at));
|
|
162
|
+
const near = (views.get(key) ?? []).filter((v) => v.t > before && v.t <= after);
|
|
138
163
|
if (!near.length)
|
|
139
|
-
return [];
|
|
164
|
+
return { views: [], estimated: false };
|
|
140
165
|
const last = Math.max(...near.map((v) => v.t));
|
|
141
|
-
|
|
166
|
+
const cluster = near.filter((v) => v.t >= last - VIEW_CLUSTER_MS);
|
|
167
|
+
return { views: cluster, estimated: own === undefined && cluster.length < near.length };
|
|
142
168
|
};
|
|
143
169
|
// places read again from a sharper copy since (a later reading): by whole image, fractions as the file lies
|
|
144
170
|
const readSharper = new Map();
|
|
@@ -167,7 +193,7 @@ function blindReducedDetect(tree, env) {
|
|
|
167
193
|
const m = byId.get(id);
|
|
168
194
|
if (!m || m.retracted)
|
|
169
195
|
continue;
|
|
170
|
-
const own = viewsOf(id, t);
|
|
196
|
+
const { views: own, estimated: unsure } = viewsOf(r.stem, id, t);
|
|
171
197
|
if (m.part) {
|
|
172
198
|
// a part read: the place in the whole image read sharper (when its views say where)
|
|
173
199
|
if (m.recordset !== undefined && m.image !== undefined) {
|
|
@@ -189,7 +215,7 @@ function blindReducedDetect(tree, env) {
|
|
|
189
215
|
const x1 = Math.max(...own.map((v) => v.region.x + v.region.w));
|
|
190
216
|
const y1 = Math.max(...own.map((v) => v.region.y + v.region.h));
|
|
191
217
|
const v = own[0];
|
|
192
|
-
places = [{ file: shownToFile({ x: x0 / v.W, y: y0 / v.H, w: (x1 - x0) / v.W, h: (y1 - y0) / v.H }, v.orient), estimated:
|
|
218
|
+
places = [{ file: shownToFile({ x: x0 / v.W, y: y0 / v.H, w: (x1 - x0) / v.W, h: (y1 - y0) / v.H }, v.orient), estimated: unsure }];
|
|
193
219
|
}
|
|
194
220
|
else if ((views.size === 0 || !views.has(id)) && r.blind && few && t < Date.parse(BLIND_REDUCED_FIXED_AT)) {
|
|
195
221
|
// no record of its views here (another computer): the places of the sources' entries on the image, an estimate
|
|
@@ -504,7 +530,7 @@ function newModelDetect(tree, env) {
|
|
|
504
530
|
const unclear = blocks.reduce((n, b) => n + Math.max(unclearOf(b), b.result === "unclear" ? 1 : 0), 0);
|
|
505
531
|
// a blind reading that reads otherwise than the researcher's reading of the image before it
|
|
506
532
|
const theirs = r.blind ? list.filter((x) => !x.r.blind && x.t <= t && x.r !== r).sort((a, b) => b.t - a.t)[0] : undefined;
|
|
507
|
-
const differ = !!theirs && compareReadings(theirs.blocks, blocks) === "differ";
|
|
533
|
+
const differ = !!theirs && compareReadings(theirs.blocks, blocks).result === "differ";
|
|
508
534
|
const reg = cropRegion(crop);
|
|
509
535
|
const src = reg && !m.part ? sourcesOf(sources, media, m, shownToFile(reg, mediaOrient(env.shared, m))) : sources.filter((s) => (s.media ?? []).includes(id) || (s.clips ?? []).some((c) => c.media === id)).map((s) => s.id).sort();
|
|
510
536
|
const reasons = [...(unclear ? ["unclear"] : []), ...(differ ? ["differ"] : []), ...(src.some((s) => conflicted.has(s)) ? ["conflict"] : [])];
|
|
@@ -671,13 +697,16 @@ export function decide(tree, id, decision) {
|
|
|
671
697
|
delete c.fixes;
|
|
672
698
|
}, { op: "fix.decide", summary: decision ? `fix ${id}: ${decision}` : `fix ${id}: asked again` });
|
|
673
699
|
}
|
|
674
|
-
/**
|
|
700
|
+
/**
|
|
701
|
+
* Each check with what it found here and the decision about it. Set right holds for what was read again: what a check
|
|
702
|
+
* still finds after (never read again — a few first, or a place strom took wrongly then) waits again, it alone.
|
|
703
|
+
*/
|
|
675
704
|
export function fixStates(tree, env) {
|
|
676
705
|
return FIX_CHECKS.map((check) => {
|
|
677
706
|
const found = check.detect(tree, env);
|
|
678
707
|
const key = decisionKeyOf(tree, check, env);
|
|
679
708
|
const decision = decisionOf(tree.config, key);
|
|
680
|
-
const state = decision ?
|
|
709
|
+
const state = decision?.decision === "declined" ? "declined" : found.items.length ? "pending" : decision ? "fixed" : "nothing";
|
|
681
710
|
return { check, key, found, ...(decision ? { decision } : {}), state };
|
|
682
711
|
});
|
|
683
712
|
}
|
|
@@ -719,39 +748,94 @@ export function estimateFix(root, o) {
|
|
|
719
748
|
}
|
|
720
749
|
return { level: n >= COST_COUNT.expensive ? "expensive" : n >= COST_COUNT.medium ? "medium" : "cheap", calls: n, minutes, basis: "count", earlier: 0 };
|
|
721
750
|
}
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
751
|
+
// a reader's quotation marks: what it quotes of the record is its transcript, the rest its comment
|
|
752
|
+
const QUOTED = /[„“”"«»「」『』]([^„“”"«»「」『』]*)[„“”"«»「」『』]/gu;
|
|
753
|
+
/** A word as compared: folded, the long s and the ezh of old hands as the letters they are. */
|
|
754
|
+
const wordKey = (w) => foldText(w).replace(/ſ/gu, "s").replace(/ʒ/gu, "z");
|
|
755
|
+
/**
|
|
756
|
+
* The facts of a transcript: the names (a word that begins with a capital, or any word of a script without capitals)
|
|
757
|
+
* and the numbers; a word broken over a line ("Wac- / lawa", "In= / mann", "Wac|lawa") whole, the marks of doubt and an editor's
|
|
758
|
+
* notes in brackets left out (a letter supplied in brackets, "Mi[cha]la", kept). Of a reader's own words (prose: nothing
|
|
759
|
+
* quoted), a capital that only begins a sentence ("The crop shows …", "Výřez ukazuje …") is no name.
|
|
760
|
+
*/
|
|
761
|
+
function readOf(text, unclear, found, prose = false) {
|
|
762
|
+
const clean = text
|
|
763
|
+
.replace(/\[\?\]|\(\?\)/gu, "")
|
|
764
|
+
.replace(/\[[^\]]*\s[^\]]*\]/gu, " ")
|
|
765
|
+
.replace(/[[\]]/gu, "")
|
|
766
|
+
.replace(/([\p{L}\p{M}])(?:[-=¬]\s*[/|]?\s*|\|)(?=[\p{L}\p{M}])/gu, "$1");
|
|
767
|
+
const facts = new Map();
|
|
768
|
+
const words = new Set();
|
|
769
|
+
for (const m of clean.matchAll(/[\p{L}\p{M}\p{N}]+/gu)) {
|
|
770
|
+
const w = m[0];
|
|
771
|
+
const k = wordKey(w);
|
|
772
|
+
words.add(k);
|
|
773
|
+
const digits = /\p{N}/u.test(w);
|
|
774
|
+
const cased = /[\p{Lu}\p{Ll}]/u.test(w);
|
|
775
|
+
if (!digits && cased && prose && /^\p{Lu}/u.test(w) && opensSentence(clean, m.index, w.length))
|
|
776
|
+
continue;
|
|
777
|
+
if (digits || (w.length > 1 && (!cased || /^\p{Lu}/u.test(w))))
|
|
778
|
+
if (!facts.has(k))
|
|
779
|
+
facts.set(k, w);
|
|
780
|
+
}
|
|
781
|
+
return { facts, words, unclear, found };
|
|
782
|
+
}
|
|
783
|
+
/**
|
|
784
|
+
* A word that begins a sentence of prose: at a line's start or after a sentence's end (never after a number's dot, "5. Mai"),
|
|
785
|
+
* the next word of its sentence in small letters ("Jan Novak", "Jan, son of" stay names).
|
|
786
|
+
*/
|
|
787
|
+
function opensSentence(text, at, length) {
|
|
788
|
+
let i = at - 1;
|
|
789
|
+
while (i >= 0 && /\s/u.test(text[i]) && text[i] !== "\n")
|
|
790
|
+
i--;
|
|
791
|
+
const start = i < 0 || text[i] === "\n" || (/[.!?…]/u.test(text[i]) && i > 0 && !/\p{N}/u.test(text[i - 1]));
|
|
792
|
+
return start && /^\s+\p{Ll}/u.test(text.slice(at + length));
|
|
793
|
+
}
|
|
794
|
+
/**
|
|
795
|
+
* What a reader read of its blocks: the parts of its entries it quotes from the record (when it quotes any), else its
|
|
796
|
+
* entries — never a line of possible readings with their chances or of letters with theirs (a "%" in it), nor how it
|
|
797
|
+
* describes the hand or how sure it is (hand, certainty); what it could not read only counts as unclear.
|
|
798
|
+
*/
|
|
799
|
+
function readOfBlocks(blocks) {
|
|
800
|
+
const lines = blocks.flatMap((b) => b.entries ?? []).filter((l) => !/%/u.test(l));
|
|
801
|
+
const quoted = lines.flatMap((l) => [...l.matchAll(QUOTED)].map((m) => m[1]));
|
|
802
|
+
const unclear = blocks.reduce((n, x) => n + unclearOf(x), 0);
|
|
803
|
+
const found = blocks.some((b) => b.result === "found");
|
|
804
|
+
return quoted.length ? readOf(quoted.join(" "), unclear, found) : readOf(lines.join("\n"), unclear, found, true);
|
|
805
|
+
}
|
|
806
|
+
/** What a reading of a document read: the people's names and ages, the events' dates and places (the reader's own fields). */
|
|
807
|
+
function readOfDocument(r) {
|
|
808
|
+
const { unclear } = docText(r);
|
|
809
|
+
const parts = r.entries.flatMap((e) => [...e.persons.flatMap((p) => [p.given, p.surname, p.age]), ...e.events.flatMap((v) => [v.date?.gedcom ?? v.date?.text, v.place?.text])]);
|
|
810
|
+
// every word of its fields counts: a document's reader says what each one is
|
|
811
|
+
const read = readOf(parts.filter(Boolean).join(" "), unclear, r.entries.length > 0);
|
|
812
|
+
for (const w of read.words)
|
|
813
|
+
if (!read.facts.has(w))
|
|
814
|
+
read.facts.set(w, w);
|
|
815
|
+
return read;
|
|
725
816
|
}
|
|
726
|
-
/** A reader's entries as one text, and its uncertain places. */
|
|
727
|
-
const readText = (blocks) => ({ text: blocks.map((b) => (b.entries?.length ? b.entries.join(" ") : b.text)).join(" "), unclear: blocks.reduce((n, x) => n + unclearOf(x), 0) });
|
|
728
817
|
/**
|
|
729
|
-
* The new reading of a place against the earlier one
|
|
730
|
-
*
|
|
818
|
+
* The new reading of a place against the earlier one, by what each read — the names and numbers of its transcript:
|
|
819
|
+
* the same (shoda); all the earlier one read with more beside it, fewer places unclear, or found where it was unclear
|
|
820
|
+
* (upřesněno); something the earlier one read that the new one does not — another name, date, number or place (rozchod).
|
|
821
|
+
* Nothing read: failed.
|
|
731
822
|
*/
|
|
732
823
|
export function compareReadings(before, now) {
|
|
733
|
-
return now.length ?
|
|
824
|
+
return now.length ? compareRead(readOfBlocks(before), readOfBlocks(now)) : { result: "failed", before: [], now: [] };
|
|
734
825
|
}
|
|
735
|
-
/** A reading of a document again against the earlier one, as compareReadings (its
|
|
826
|
+
/** A reading of a document again against the earlier one, as compareReadings (its people and events). */
|
|
736
827
|
export function compareDocuments(before, now) {
|
|
737
828
|
if (!now || !now.entries.length)
|
|
738
|
-
return "failed";
|
|
739
|
-
return
|
|
740
|
-
}
|
|
741
|
-
function
|
|
742
|
-
const
|
|
743
|
-
const
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
const
|
|
747
|
-
|
|
748
|
-
// what the earlier reading had, the new one has too, with more beside it: read further
|
|
749
|
-
const more = a.size > 0 && both / a.size >= 0.85 && b.size > a.size;
|
|
750
|
-
if (alike >= 0.9)
|
|
751
|
-
return fewer ? "refined" : "same";
|
|
752
|
-
if ((alike >= 0.6 && fewer) || more)
|
|
753
|
-
return "refined";
|
|
754
|
-
return "differ";
|
|
829
|
+
return { result: "failed", before: [], now: [] };
|
|
830
|
+
return compareRead(readOfDocument(before), readOfDocument(now));
|
|
831
|
+
}
|
|
832
|
+
function compareRead(a, b) {
|
|
833
|
+
const before = [...a.facts].filter(([k]) => !b.words.has(k)).map(([, w]) => w);
|
|
834
|
+
const now = [...b.facts].filter(([k]) => !a.words.has(k)).map(([, w]) => w);
|
|
835
|
+
if (before.length)
|
|
836
|
+
return { result: "differ", before, now };
|
|
837
|
+
const surer = b.unclear < a.unclear || (b.found && !a.found);
|
|
838
|
+
return { result: now.length || surer ? "refined" : "same", before, now };
|
|
755
839
|
}
|
|
756
840
|
// ── what waits ──────────────────────────────────────────────────────────────────────────────────────────────────
|
|
757
841
|
/** The checks that wait for the person's decision here (none in an archive: nobody reads there). */
|
package/dist/commands/fixes.js
CHANGED
|
@@ -202,6 +202,12 @@ function blocksOf(tree, stem, media) {
|
|
|
202
202
|
const r = loadReadings(tree).find((x) => x.stem === stem);
|
|
203
203
|
return (r?.blocks ?? []).filter((b) => b.media === media.id || (b.recordset === media.recordset && b.num !== undefined && b.num === media.image));
|
|
204
204
|
}
|
|
205
|
+
/** A comparison as a reading again keeps it: its result, and what differs where something does. */
|
|
206
|
+
function compared(c) {
|
|
207
|
+
if (!c)
|
|
208
|
+
return { result: "failed" };
|
|
209
|
+
return { result: c.result, ...(c.before.length || c.now.length ? { differs: { before: c.before, now: c.now } } : {}) };
|
|
210
|
+
}
|
|
205
211
|
/** Each place read again, blind with the earlier question, from the sharper copy; the readings held against each other. */
|
|
206
212
|
async function readAgain(ctx, tree, t, items, model) {
|
|
207
213
|
const { def } = resolveCommand(["read"]);
|
|
@@ -231,7 +237,8 @@ async function readAgain(ctx, tree, t, items, model) {
|
|
|
231
237
|
}
|
|
232
238
|
const before = loadDocResult(tree.root, item.reading);
|
|
233
239
|
const secs = (Date.now() - began) / 1000;
|
|
234
|
-
|
|
240
|
+
const c = before ? compareDocuments(before, now) : undefined;
|
|
241
|
+
return [{ item, ...compared(c), ...(now?.id ? { reading: now.id } : {}), secs, ...(error ? { error } : {}) }];
|
|
235
242
|
}
|
|
236
243
|
const opts = { question: item.question, ...(model ? { model } : {}), ...(item.crop ? { crop: item.crop } : {}), parallel: "1", ...(item.blind ? { blind: true } : {}), ...(item.context ? { context: item.context } : {}) };
|
|
237
244
|
let reading;
|
|
@@ -249,7 +256,7 @@ async function readAgain(ctx, tree, t, items, model) {
|
|
|
249
256
|
const m = byId.get(i.media);
|
|
250
257
|
const now = reading && m ? blocksOf(tree, reading, m) : [];
|
|
251
258
|
const before = m ? blocksOf(tree, i.reading, m) : [];
|
|
252
|
-
return { item: i,
|
|
259
|
+
return { item: i, ...compared(compareReadings(before, now)), ...(reading ? { reading } : {}), secs, ...(error ? { error } : {}) };
|
|
253
260
|
});
|
|
254
261
|
})).flat();
|
|
255
262
|
}
|
|
@@ -267,7 +274,7 @@ function tasksFor(tree, check, done, now) {
|
|
|
267
274
|
priority: d.result === "differ" ? 2 : 3,
|
|
268
275
|
what: phrase(lang, `${p}.what`, { media: i.document ? i.document.files.join(", ") : i.media, old: i.reading, now: d.reading }),
|
|
269
276
|
where: [...(i.document ? i.document.files : [i.media]), ...i.sources],
|
|
270
|
-
why: phrase(lang, `${p}.why.${d.result}`, { since: check.since, from: i.from ?? "", before: i.before ? label(i.before) : "", now: now ? label(now) : "" }),
|
|
277
|
+
why: [phrase(lang, `${p}.why.${d.result}`, { since: check.since, from: i.from ?? "", before: i.before ? label(i.before) : "", now: now ? label(now) : "" }), differsText(lang, d)].filter(Boolean).join(" "),
|
|
271
278
|
doneWhen: phrase(lang, `${p}.done`, { old: i.reading, now: d.reading, sources: i.sources.join(", ") || i.media, show: i.document ? "strom document show" : "strom readings" }),
|
|
272
279
|
subject: [...i.sources],
|
|
273
280
|
state: "open",
|
|
@@ -276,6 +283,15 @@ function tasksFor(tree, check, done, now) {
|
|
|
276
283
|
d.task = task.id;
|
|
277
284
|
}
|
|
278
285
|
}
|
|
286
|
+
/** One sentence of what differs (the facts each read that the other does not), or what the new reading read beside. */
|
|
287
|
+
function differsText(lang, d) {
|
|
288
|
+
const list = (ws) => (ws.length ? ws.slice(0, 6).join(", ") + (ws.length > 6 ? ", …" : "") : "—");
|
|
289
|
+
if (!d.differs)
|
|
290
|
+
return "";
|
|
291
|
+
if (d.result === "differ")
|
|
292
|
+
return phrase(lang, "fix.differs", { before: list(d.differs.before), now: list(d.differs.now) });
|
|
293
|
+
return d.differs.now.length ? phrase(lang, "fix.more", { now: list(d.differs.now) }) : "";
|
|
294
|
+
}
|
|
279
295
|
/** `secs`: the real time the readings took, readers at once and all. Its places: the same, refined, different. */
|
|
280
296
|
function summaryText(t, done, secs) {
|
|
281
297
|
const n = (r) => done.filter((d) => d.result === r).length;
|
package/dist/core/phrases.js
CHANGED
|
@@ -111,6 +111,8 @@ export const PHRASES = {
|
|
|
111
111
|
"fix.model.why.differ": "Read again by {now} (earlier: {before}) where something stayed open, and the readings differ.",
|
|
112
112
|
"fix.model.why.refined": "Read again by {now} (earlier: {before}) where something stayed open, and the new reading reads more or is surer.",
|
|
113
113
|
"fix.model.done": "both readings held against each other ({show} {old} · {show} {now}) and against {sources}: what holds kept on the source, the facts on it checked; nothing taken away without a reason",
|
|
114
|
+
"fix.differs": "What differs: earlier {before}, now {now}.",
|
|
115
|
+
"fix.more": "Read now beside the earlier: {now}.",
|
|
114
116
|
"fix.model.unknown": "a model not recorded",
|
|
115
117
|
"fix.model.own": "its own model",
|
|
116
118
|
"batch.unnamed": "material from the Strom app",
|
package/dist/core/tree.js
CHANGED
|
@@ -27,7 +27,7 @@ import { lastBackup } from "./backup.js";
|
|
|
27
27
|
export const TREE_FILE = "strom.json";
|
|
28
28
|
// between releases the candidate being made (a bridge run from the sources says what it is: the Strom app goes by it);
|
|
29
29
|
// at a release the release's own (package.json)
|
|
30
|
-
export const VERSION = "1.19.0-beta.
|
|
30
|
+
export const VERSION = "1.19.0-beta.6";
|
|
31
31
|
const GITIGNORE = `# Strom Research — local state and large files are not versioned
|
|
32
32
|
.strom/
|
|
33
33
|
/media/
|