zero-slop 2.7.7 → 2.7.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +50 -36
- package/SKILL.md +16 -4
- package/bin/zero-slop.mjs +85 -15
- package/package.json +2 -2
- package/references/eval.md +5 -2
- package/scripts/learn.py +2 -2
- package/scripts/register.py +92 -12
- package/scripts/rerank.py +12 -3
- package/scripts/safeio.py +4 -3
- package/scripts/slopscore.py +99 -22
package/README.md
CHANGED
|
@@ -5,20 +5,28 @@
|
|
|
5
5
|
<img alt="tests" src="https://img.shields.io/badge/tests-passing-227B5B">
|
|
6
6
|
<img alt="dependencies" src="https://img.shields.io/badge/runtime%20dependencies-0-227B5B">
|
|
7
7
|
<img alt="privacy" src="https://img.shields.io/badge/learning-private-227B5B">
|
|
8
|
-
<img alt="version" src="https://img.shields.io/badge/version-2.7.
|
|
8
|
+
<img alt="version" src="https://img.shields.io/badge/version-2.7.8-72528F">
|
|
9
9
|
</p>
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
Less slop, more pop.
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
Zero Slop finds stock phrasing, mechanical rhythm, vague claims, and canned
|
|
14
|
+
formatting, then gives your AI assistant a guarded editing workflow. Its
|
|
15
|
+
MIT-licensed local checks run offline; your existing AI assistant does the editing.
|
|
16
|
+
See the benchmark at [zero-slop.ai](https://zero-slop.ai).
|
|
14
17
|
|
|
15
18
|

|
|
16
19
|
|
|
17
20
|
## Problem
|
|
18
21
|
|
|
19
|
-
AI writing
|
|
22
|
+
AI-assisted writing often converges on the same constructions: "It's not X. It's Y."
|
|
23
|
+
"Here's the thing nobody tells you." One phrase proves little. Repetition across
|
|
24
|
+
wording, rhythm, structure, and formatting is the problem.
|
|
20
25
|
|
|
21
|
-
Zero Slop is an Agent Skill and ships no model. Claude, GPT, or another compatible
|
|
26
|
+
Zero Slop is an Agent Skill and ships no model. Claude, GPT, or another compatible
|
|
27
|
+
model edits; local tools check names, numbers, quotations, links, code, tables, and
|
|
28
|
+
paths. Your AI compares meaning because matching words cannot catch every changed
|
|
29
|
+
claim. It runs in any harness that reads `SKILL.md`.
|
|
22
30
|
|
|
23
31
|
## How to install Zero Slop
|
|
24
32
|
|
|
@@ -42,7 +50,8 @@ npx zero-slop score draft.md # score without installing anything
|
|
|
42
50
|
```
|
|
43
51
|
|
|
44
52
|
ChatGPT users can download [`dist/zero-slop-single-file.md`](dist/zero-slop-single-file.md).
|
|
45
|
-
Claude.ai users can upload [
|
|
53
|
+
Claude.ai users can upload the [latest release ZIP](https://github.com/manavmishra/ZeroSlop/releases/latest/download/zero-slop.zip).
|
|
54
|
+
`npx skills update zero-slop --global` updates a skills CLI installation later.
|
|
46
55
|
|
|
47
56
|
## How to use Zero Slop
|
|
48
57
|
|
|
@@ -50,8 +59,9 @@ Claude.ai users can upload [`dist/zero-slop.zip`](dist/zero-slop.zip). `npx skil
|
|
|
50
59
|
/zero-slop (your writing)
|
|
51
60
|
```
|
|
52
61
|
|
|
53
|
-
You get the edited draft,
|
|
54
|
-
|
|
62
|
+
You get the edited draft, before-and-after scores, and quoted flagged phrases.
|
|
63
|
+
`/zero-slop inspect (your writing)` reviews without rewriting. For a folder,
|
|
64
|
+
`slopscore.py --batch drafts/ --gate 25` fails above the threshold.
|
|
55
65
|
|
|
56
66
|
## The slop that Zero Slop catches
|
|
57
67
|
|
|
@@ -68,9 +78,9 @@ For a folder, `slopscore.py --batch drafts/ --gate 25` fails the build above the
|
|
|
68
78
|
9. Synonym cycling: the agent, the assistant, the tool, all one thing.
|
|
69
79
|
10. Marketing riders: "robust" and "leverage" score only beside a marketing trigger, so a runbook stays quiet.
|
|
70
80
|
|
|
71
|
-
A reading pass covers
|
|
72
|
-
paragraphs that shuffle without loss. [`references/eval.md`](references/eval.md)
|
|
73
|
-
80 checks.
|
|
81
|
+
A reading pass covers document-wide problems: repeated shapes, crowded statistics,
|
|
82
|
+
and paragraphs that shuffle without loss. [`references/eval.md`](references/eval.md)
|
|
83
|
+
has all 80 checks.
|
|
74
84
|
|
|
75
85
|
Human writing scored 9 to 21 in [`data/corpus/must-not-flag/`](data/corpus/must-not-flag/);
|
|
76
86
|
unedited AI drafts averaged 77 across [`bench/examples.json`](bench/examples.json).
|
|
@@ -101,15 +111,13 @@ tools use only Python's standard library.
|
|
|
101
111
|
|
|
102
112
|
## Private learning from your edits
|
|
103
113
|
|
|
104
|
-
Learning starts only when you
|
|
105
|
-
|
|
114
|
+
Learning starts only when you provide the original output and your edited version.
|
|
115
|
+
Zero Slop does not monitor files, browsers, or publishing tools.
|
|
106
116
|
|
|
107
|
-
Private
|
|
108
|
-
data stays under `$ZERO_SLOP_HOME`.
|
|
117
|
+
Private data stays under `$ZERO_SLOP_HOME`.
|
|
109
118
|
|
|
110
|
-
This
|
|
111
|
-
|
|
112
|
-
words when selected by name; it does not learn cadence, tone, or a complete writing
|
|
119
|
+
This human-in-the-loop learning never retrains the model. A profile selected by name
|
|
120
|
+
can exempt existing watchlist words; it does not learn cadence, tone, or a complete
|
|
113
121
|
style.
|
|
114
122
|
|
|
115
123
|
## What's inside
|
|
@@ -120,16 +128,18 @@ with [`scripts/register.py`](scripts/register.py) running the reading pass.
|
|
|
120
128
|
[`bench/README.md`](bench/README.md) documents every benchmark with its limits.
|
|
121
129
|
[zero-slop.ai](https://zero-slop.ai) has the same reference as browsable pages, plus the [benchmark in full](https://zero-slop.ai/benchmark/).
|
|
122
130
|
|
|
123
|
-
##
|
|
131
|
+
## Testing and limits
|
|
124
132
|
|
|
125
133
|
### Against other tools, same model, same drafts
|
|
126
134
|
|
|
127
|
-
|
|
135
|
+
The saved replay ran Zero Slop, [avoid-ai-writing](https://github.com/conorbronsdon/avoid-ai-writing),
|
|
128
136
|
[no-ai-slop](https://github.com/petergyang/no-ai-slop) and
|
|
129
|
-
[humanizer](https://github.com/blader/humanizer) on the same
|
|
130
|
-
GPT-5.4, high reasoning, batches of three, and
|
|
137
|
+
[humanizer](https://github.com/blader/humanizer) on the same 18 drafts, each with
|
|
138
|
+
GPT-5.4, high reasoning, batches of three, and pinned instructions. The Zero Slop
|
|
139
|
+
outputs came from v2.5.9; later releases rescore those frozen outputs but do not
|
|
140
|
+
pretend to have regenerated them.
|
|
131
141
|
|
|
132
|
-
| Method | Mean writing score ↓ | Passed
|
|
142
|
+
| Method | Mean writing score ↓ | Passed Zero Slop's local gates | Source check passed | Average length change |
|
|
133
143
|
|---|---:|---:|---:|---:|
|
|
134
144
|
| Original drafts | 76.3 | 0/18 | — | — |
|
|
135
145
|
| Zero Slop | 12.8 | 18/18 | 18/18 | -8.9% |
|
|
@@ -151,19 +161,23 @@ rates, and a method-hidden quality ranking.
|
|
|
151
161
|

|
|
152
162
|
|
|
153
163
|
This is a small LLM-reviewed regression study. It measures neither field accuracy nor a
|
|
154
|
-
universal ranking. Drafts, mappings, verdicts, hashes and limits
|
|
155
|
-
[`bench/
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
164
|
+
universal ranking. Drafts, mappings, verdicts, hashes, version records, and limits are
|
|
165
|
+
in [`bench/README.md`](bench/README.md). The separate method-hidden two-way replay used
|
|
166
|
+
Zero Slop v2.6.0 and is preserved in
|
|
167
|
+
[`bench/incumbent-blind-replay/`](bench/incumbent-blind-replay/).
|
|
168
|
+
|
|
169
|
+
For the 38-item editorial panel, the current scorer matched the prior 84.2% result.
|
|
170
|
+
All frozen scores stayed unchanged, all 18 human controls remained below the gate,
|
|
171
|
+
and all 18 obvious search cases remained above it. These fixed-sample checks are not
|
|
172
|
+
proof of general accuracy. We recorded 6.51% higher median throughput across 12 interleaved
|
|
173
|
+
local runs; treat that as machine noise, not a speed claim.
|
|
161
174
|
|
|
162
175
|
### Speed
|
|
163
176
|
|
|
164
|
-
On one Apple silicon Mac: 1,000 documents in
|
|
165
|
-
15,201-word document in 0.
|
|
166
|
-
8,000-word learning pass in 0.
|
|
177
|
+
On one busy Apple silicon Mac: 1,000 documents in 3.2233 seconds (310.2 per second),
|
|
178
|
+
a 15,201-word document in 0.4757 seconds, the slowest stress case in 3.3487 seconds,
|
|
179
|
+
and an 8,000-word learning pass in 0.2284 seconds. Editing time is excluded; expect
|
|
180
|
+
different results on other machines and loads.
|
|
167
181
|
|
|
168
182
|
### Current models
|
|
169
183
|
|
|
@@ -183,9 +197,9 @@ Beemo, raw responses averaged
|
|
|
183
197
|
|
|
184
198
|
## Where Zero Slop came from
|
|
185
199
|
|
|
186
|
-
Zero Slop
|
|
187
|
-
avoid-ai-writing
|
|
188
|
-
private learning, portfolio analysis and release tests.
|
|
200
|
+
Zero Slop builds on work by no-ai-slop, humanizer, de-slop, stop-slop, unslop-text,
|
|
201
|
+
and avoid-ai-writing. It adds a writing score, source protection, separate editorial
|
|
202
|
+
passes, private learning, portfolio analysis, and release tests.
|
|
189
203
|
|
|
190
204
|

|
|
191
205
|
|
package/SKILL.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: zero-slop
|
|
3
3
|
license: MIT
|
|
4
4
|
metadata:
|
|
5
|
-
version: "2.7.
|
|
5
|
+
version: "2.7.8"
|
|
6
6
|
author: manavmishra
|
|
7
7
|
description: Turn drafts into sharp, natural prose or inspect them without rewriting. Zero Slop runs inside the user's existing AI assistant; Claude, GPT, or another compatible model reads and edits in context while local tools point to exact phrases and protect the source. Use when the user asks to humanize or de-slop writing, inspect AI-sounding patterns, fix text that reads like ChatGPT, polish outward-facing prose, draft social or LinkedIn content, or apply a final quality check to prose the agent generated. The workflow preserves facts, voice, and format and learns privately from repeated, reason-labelled human edits.
|
|
8
8
|
---
|
|
@@ -504,6 +504,17 @@ Re-run the local tools. A version clears the fact gate only when ALL hold:
|
|
|
504
504
|
wording may change; their content and nesting may not. This deterministic
|
|
505
505
|
check still cannot see a subtly reframed claim, changed emphasis, or shifted
|
|
506
506
|
implication, so the judgment pass below remains mandatory
|
|
507
|
+
- if a reviewer confirms that a dropped figure was an unsourced flourish rather
|
|
508
|
+
than a fact, record the decision in a source-bound JSON file and rerun:
|
|
509
|
+
|
|
510
|
+
```
|
|
511
|
+
python3 <skill-root>/scripts/slopscore.py --fidelity \
|
|
512
|
+
--adjudication <ruling.json> <original> <rewrite>
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
The file contains schema `1`, the SHA-256 of the exact original text, and
|
|
516
|
+
`allow_dropped_figures`. It can excuse only figures found in that source; it
|
|
517
|
+
cannot weaken checks for names, quotations, links, feelings, or structure
|
|
507
518
|
- shape (social genres only): the scorer reports `broetry` when most
|
|
508
519
|
paragraphs are single sentences and fragments run three or more deep. This
|
|
509
520
|
is its own axis, never folded into the score, because broetry is a slop tell
|
|
@@ -741,9 +752,10 @@ facts, meaning, structure, and whether the writing is performing rather than say
|
|
|
741
752
|
```
|
|
742
753
|
|
|
743
754
|
The "two-part contrasts / announcements" row is the performed-register count from
|
|
744
|
-
step 2. Add a line
|
|
745
|
-
|
|
746
|
-
|
|
755
|
+
step 2. Add a line beside it: `Final review: 80 checks, 0 failed` or the count
|
|
756
|
+
that did fail. The 80 checks include AI reading, local measurements, and source
|
|
757
|
+
protection; do not imply that one script performed all of them. **Print it even when
|
|
758
|
+
both numbers are zero**, and print it on a draft that
|
|
747
759
|
scored clean. It is the only evidence that the pass ran; a report without it is a
|
|
748
760
|
report that skipped it.
|
|
749
761
|
|
package/bin/zero-slop.mjs
CHANGED
|
@@ -5,10 +5,10 @@
|
|
|
5
5
|
// mirrors rather than people. This is the executable half: it installs the same
|
|
6
6
|
// runtime the tarball already carries, and runs the scorer without a checkout.
|
|
7
7
|
|
|
8
|
-
import { cp, mkdir, readFile, rm, stat } from "node:fs/promises";
|
|
8
|
+
import { cp, lstat, mkdir, readFile, readdir, rename, rm, stat } from "node:fs/promises";
|
|
9
9
|
import { spawn } from "node:child_process";
|
|
10
10
|
import { homedir } from "node:os";
|
|
11
|
-
import { dirname, join, resolve } from "node:path";
|
|
11
|
+
import { dirname, join, parse, resolve } from "node:path";
|
|
12
12
|
import { fileURLToPath } from "node:url";
|
|
13
13
|
|
|
14
14
|
const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
|
@@ -41,7 +41,7 @@ Usage
|
|
|
41
41
|
Install targets
|
|
42
42
|
--harness ${Object.keys(HARNESS_DIRS).join(", ")} (default: claude)
|
|
43
43
|
--dir install into an explicit directory instead
|
|
44
|
-
--force
|
|
44
|
+
--force replace a verified Zero Slop installation
|
|
45
45
|
|
|
46
46
|
Examples
|
|
47
47
|
npx zero-slop install # ~/.claude/skills/zero-slop
|
|
@@ -63,10 +63,18 @@ function parseArgs(argv) {
|
|
|
63
63
|
break;
|
|
64
64
|
}
|
|
65
65
|
if (a === "--force") flags.force = true;
|
|
66
|
-
else if (a === "--harness" || a === "--dir")
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
66
|
+
else if (a === "--harness" || a === "--dir") {
|
|
67
|
+
const value = argv[i + 1];
|
|
68
|
+
if (!value || value.startsWith("--")) throw new Error(`${a} needs a value`);
|
|
69
|
+
flags[a.slice(2)] = value;
|
|
70
|
+
i += 1;
|
|
71
|
+
} else if (a.startsWith("--harness=")) {
|
|
72
|
+
flags.harness = a.slice("--harness=".length);
|
|
73
|
+
if (!flags.harness) throw new Error("--harness needs a value");
|
|
74
|
+
} else if (a.startsWith("--dir=")) {
|
|
75
|
+
flags.dir = a.slice("--dir=".length);
|
|
76
|
+
if (!flags.dir) throw new Error("--dir needs a value");
|
|
77
|
+
} else rest.push(a);
|
|
70
78
|
}
|
|
71
79
|
return { flags, rest, passthrough };
|
|
72
80
|
}
|
|
@@ -92,21 +100,77 @@ async function exists(p) {
|
|
|
92
100
|
}
|
|
93
101
|
}
|
|
94
102
|
|
|
103
|
+
function assertNarrowTarget(dest) {
|
|
104
|
+
const target = resolve(dest);
|
|
105
|
+
const forbidden = new Set([
|
|
106
|
+
parse(target).root,
|
|
107
|
+
resolve(homedir()),
|
|
108
|
+
resolve(process.cwd()),
|
|
109
|
+
ROOT,
|
|
110
|
+
]);
|
|
111
|
+
if (forbidden.has(target)) {
|
|
112
|
+
throw new Error(`refusing broad install target: ${target}`);
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
async function verifiedExistingInstall(dest) {
|
|
117
|
+
const info = await lstat(dest);
|
|
118
|
+
if (info.isSymbolicLink() || !info.isDirectory()) return false;
|
|
119
|
+
const entries = await readdir(dest);
|
|
120
|
+
if (!entries.length) return true;
|
|
121
|
+
try {
|
|
122
|
+
const skill = await readFile(join(dest, "SKILL.md"), "utf8");
|
|
123
|
+
const header = skill.split("---", 3)[1] ?? "";
|
|
124
|
+
const runtime = await lstat(join(dest, "scripts", "slopscore.py"));
|
|
125
|
+
const patterns = await lstat(join(dest, "data", "patterns.json"));
|
|
126
|
+
return /^name:\s*zero-slop\s*$/m.test(header)
|
|
127
|
+
&& runtime.isFile() && patterns.isFile();
|
|
128
|
+
} catch {
|
|
129
|
+
return false;
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
|
|
95
133
|
async function install(flags) {
|
|
96
134
|
const dest = targetDir(flags);
|
|
97
|
-
|
|
135
|
+
assertNarrowTarget(dest);
|
|
136
|
+
const present = await exists(dest);
|
|
137
|
+
if (present && !flags.force) {
|
|
98
138
|
console.error(
|
|
99
139
|
`zero-slop is already installed at ${dest}\n` +
|
|
100
140
|
`Re-run with --force to overwrite it, or --dir to install elsewhere.`,
|
|
101
141
|
);
|
|
102
142
|
return 1;
|
|
103
143
|
}
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
144
|
+
if (present && !(await verifiedExistingInstall(dest))) {
|
|
145
|
+
throw new Error(
|
|
146
|
+
`refusing to overwrite ${dest}: it is not a Zero Slop installation`,
|
|
147
|
+
);
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
// Build the complete payload beside the destination before replacing an
|
|
151
|
+
// existing install. A copy failure therefore leaves the working version
|
|
152
|
+
// untouched, and the backup is restored if the final rename fails.
|
|
153
|
+
const nonce = `${process.pid}-${Date.now()}`;
|
|
154
|
+
const staged = `${dest}.installing-${nonce}`;
|
|
155
|
+
const backup = `${dest}.backup-${nonce}`;
|
|
156
|
+
await mkdir(dirname(dest), { recursive: true });
|
|
157
|
+
await mkdir(staged);
|
|
158
|
+
try {
|
|
159
|
+
for (const entry of PAYLOAD) {
|
|
160
|
+
await cp(join(ROOT, entry), join(staged, entry), { recursive: true });
|
|
161
|
+
}
|
|
162
|
+
if (present) await rename(dest, backup);
|
|
163
|
+
try {
|
|
164
|
+
await rename(staged, dest);
|
|
165
|
+
} catch (exc) {
|
|
166
|
+
if (present && (await exists(backup)) && !(await exists(dest))) {
|
|
167
|
+
await rename(backup, dest);
|
|
168
|
+
}
|
|
169
|
+
throw exc;
|
|
170
|
+
}
|
|
171
|
+
if (present) await rm(backup, { recursive: true, force: true });
|
|
172
|
+
} finally {
|
|
173
|
+
await rm(staged, { recursive: true, force: true });
|
|
110
174
|
}
|
|
111
175
|
console.log(`Installed zero-slop ${await version()} into ${dest}`);
|
|
112
176
|
console.log("Restart your agent, then run: /zero-slop (your writing)");
|
|
@@ -147,7 +211,13 @@ async function main() {
|
|
|
147
211
|
console.log(usage(v));
|
|
148
212
|
return command || argv.length ? 0 : 0;
|
|
149
213
|
}
|
|
150
|
-
if (command === "install")
|
|
214
|
+
if (command === "install") {
|
|
215
|
+
if (rest.length !== 1 || passthrough.length) {
|
|
216
|
+
console.error("install accepts only --harness, --dir, and --force");
|
|
217
|
+
return 2;
|
|
218
|
+
}
|
|
219
|
+
return install(flags);
|
|
220
|
+
}
|
|
151
221
|
if (command === "where") {
|
|
152
222
|
console.log(targetDir(flags));
|
|
153
223
|
return 0;
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "zero-slop",
|
|
3
|
-
"version": "2.7.
|
|
4
|
-
"description": "An Agent Skill that scores AI-sounding prose
|
|
3
|
+
"version": "2.7.8",
|
|
4
|
+
"description": "An Agent Skill that scores AI-sounding prose locally, guides the host AI through an editorial rewrite, and checks changed source details. The local tools run offline with zero dependencies.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"zero-slop": "bin/zero-slop.mjs"
|
|
7
7
|
},
|
package/references/eval.md
CHANGED
|
@@ -232,8 +232,11 @@ the report even when they are zero.
|
|
|
232
232
|
(`figure_evidence`) and waits. Rule on each one: a figure with a source, a unit,
|
|
233
233
|
a baseline or a citation must be restored; an unsourced multiplier or statistic
|
|
234
234
|
doing an intensifier's job -- "the 10x move", "tenfold", "~70% of pilots fail" --
|
|
235
|
-
was right to cut
|
|
236
|
-
|
|
235
|
+
was right to cut. Record that ruling in a source-bound JSON file and rerun
|
|
236
|
+
`slopscore.py --fidelity --adjudication ruling.json original rewrite`; the
|
|
237
|
+
file carries schema `1`, the exact original text's SHA-256, and an
|
|
238
|
+
`allow_dropped_figures` list. Never invent a citation to keep a number alive.
|
|
239
|
+
Unruled means unresolved:
|
|
237
240
|
the default is that the figure stays, so silence here is a failed check and not
|
|
238
241
|
a pass. Both errors are real. Restoring a flourish is how a rewrite ships fake
|
|
239
242
|
precision, and dropping a measurement is how it ships a lie.
|
package/scripts/learn.py
CHANGED
|
@@ -71,8 +71,8 @@ SHARED_LOG = DATA / "learned-log.md"
|
|
|
71
71
|
# default and lives outside the repository — a checkout should never carry one
|
|
72
72
|
# person's writing. Override with ZERO_SLOP_HOME to relocate or to share a
|
|
73
73
|
# deliberate team-scoped store.
|
|
74
|
-
HOME = Path(os.environ.get("ZERO_SLOP_HOME"
|
|
75
|
-
|
|
74
|
+
HOME = Path(os.environ.get("ZERO_SLOP_HOME")
|
|
75
|
+
or Path.home() / ".zero-slop").expanduser()
|
|
76
76
|
OBS = HOME / "reflections.json"
|
|
77
77
|
LOCAL = HOME / "learned.json"
|
|
78
78
|
LOCAL_LOG = HOME / "learned-log.md"
|
package/scripts/register.py
CHANGED
|
@@ -51,6 +51,15 @@ BUDGETS = {
|
|
|
51
51
|
"referent_cluster": (1.5, 1),
|
|
52
52
|
"adjective_inflation": (1.5, 1),
|
|
53
53
|
}
|
|
54
|
+
# A short draft cannot support a per-1,000-word rate, so it uses an absolute
|
|
55
|
+
# recurrence floor instead. Two families appear once in the certified-human
|
|
56
|
+
# corpus; requiring two keeps those controls quiet while still closing the old
|
|
57
|
+
# "everything under 300 words passes" bypass.
|
|
58
|
+
SHORT_FLOORS = {
|
|
59
|
+
**{key: floor for key, (_budget, floor) in BUDGETS.items()},
|
|
60
|
+
"dangling_pointer": 2,
|
|
61
|
+
"referent_cluster": 2,
|
|
62
|
+
}
|
|
54
63
|
|
|
55
64
|
# "X, not Y." and "A rather than B." The corrective appositive. Each instance is
|
|
56
65
|
# usually careful writing, which is why no pattern list contains it.
|
|
@@ -287,7 +296,7 @@ def measure(text: str) -> dict:
|
|
|
287
296
|
|
|
288
297
|
paras = paragraphs(prose)
|
|
289
298
|
lengths = [len(p.split()) for p in paras]
|
|
290
|
-
|
|
299
|
+
paragraph_uniformity = (
|
|
291
300
|
round(statistics.pstdev(lengths) / statistics.mean(lengths), 2)
|
|
292
301
|
if len(lengths) > 2 and statistics.mean(lengths)
|
|
293
302
|
else None
|
|
@@ -300,7 +309,7 @@ def measure(text: str) -> dict:
|
|
|
300
309
|
fragments = verbless_fragments(prose)
|
|
301
310
|
thin = thin_sections(text)
|
|
302
311
|
clusters = referent_clusters(text)
|
|
303
|
-
|
|
312
|
+
table_uniformity, column = table_row_uniformity(text)
|
|
304
313
|
|
|
305
314
|
return {
|
|
306
315
|
"words": words,
|
|
@@ -311,14 +320,14 @@ def measure(text: str) -> dict:
|
|
|
311
320
|
"verbless_fragment": {"count": len(fragments), "per_1k": per_k(len(fragments)), "hits": fragments[:5]},
|
|
312
321
|
"thin_section": {"count": len(thin), "per_1k": per_k(len(thin)), "hits": thin[:6]},
|
|
313
322
|
"referent_cluster": {"count": len(clusters), "per_1k": per_k(len(clusters)), "hits": clusters[:3]},
|
|
314
|
-
"table_uniformity": {"share":
|
|
323
|
+
"table_uniformity": {"share": table_uniformity, "column": column},
|
|
315
324
|
"subtractive_contrast": {"count": len(subtractive), "per_1k": per_k(len(subtractive)), "hits": subtractive[:12]},
|
|
316
325
|
"comma_series": {"count": len(series), "per_1k": per_k(len(series))},
|
|
317
326
|
"significance_scaffolding": {"count": len(significance), "per_1k": per_k(len(significance)), "hits": significance[:6]},
|
|
318
327
|
"classifier_scaffolding": {"count": len(classifier), "per_1k": per_k(len(classifier)), "hits": classifier[:6]},
|
|
319
328
|
"inanimate_agent": {"count": len(inanimate), "per_1k": per_k(len(inanimate)), "hits": inanimate[:8]},
|
|
320
329
|
"repeated_openings": {"count": len(repeated), "per_1k": per_k(len(repeated)), "hits": repeated[:6]},
|
|
321
|
-
"paragraph_uniformity":
|
|
330
|
+
"paragraph_uniformity": paragraph_uniformity,
|
|
322
331
|
}
|
|
323
332
|
|
|
324
333
|
|
|
@@ -329,7 +338,10 @@ def verdicts(m: dict) -> list[tuple[str, float, float, bool]]:
|
|
|
329
338
|
for key, (budget, floor) in BUDGETS.items():
|
|
330
339
|
value = m[key]["per_1k"]
|
|
331
340
|
count = m[key]["count"]
|
|
332
|
-
|
|
341
|
+
# Rates are unstable on short drafts, but an absolute budget is not.
|
|
342
|
+
# Below MIN_WORDS, use only the recurrence floor; otherwise require
|
|
343
|
+
# both enough instances and a rate over budget before failing.
|
|
344
|
+
ok = count < SHORT_FLOORS[key] if short else value <= budget or count < floor
|
|
333
345
|
rows.append((key, value, budget, ok))
|
|
334
346
|
return rows
|
|
335
347
|
|
|
@@ -360,8 +372,11 @@ def render(m: dict, name: str) -> str:
|
|
|
360
372
|
out.append(f" Under {MIN_WORDS} words. Rates are not reported: one instance in a short")
|
|
361
373
|
out.append(" document swamps the rate. Counts only.")
|
|
362
374
|
out.append("")
|
|
375
|
+
states = {key: ok for key, _value, _budget, ok in verdicts(m)}
|
|
363
376
|
for key in BUDGETS:
|
|
364
|
-
|
|
377
|
+
mark = "ok " if states[key] else "OVER"
|
|
378
|
+
out.append(f" {mark} {LABEL[key]:<32} {m[key]['count']:>6} found"
|
|
379
|
+
f" limit {SHORT_FLOORS[key] - 1}")
|
|
365
380
|
return "\n".join(out)
|
|
366
381
|
for key, value, budget, ok in verdicts(m):
|
|
367
382
|
mark = "ok " if ok else "OVER"
|
|
@@ -500,8 +515,8 @@ def read_packet(text: str, name: str) -> dict:
|
|
|
500
515
|
"file": name,
|
|
501
516
|
"instruction": (
|
|
502
517
|
"Work section by section, one pass per section: answer all of section A "
|
|
503
|
-
"before opening B, and so on.
|
|
504
|
-
"
|
|
518
|
+
"before opening B, and so on. Do not hold the whole checklist in "
|
|
519
|
+
"attention at once; work in small sections and answer each item. Answer with "
|
|
505
520
|
"pass or fail; where a question asks for a count, give the number. Quote "
|
|
506
521
|
"exact spans as evidence; never paraphrase. Then fill _coverage: map "
|
|
507
522
|
"every paragraph id to \"clean\" or to the list of check ids that fire "
|
|
@@ -547,9 +562,22 @@ def check_evidence(raw: str, answers: dict, checks: list[dict]) -> list[str]:
|
|
|
547
562
|
if not isinstance(got, dict):
|
|
548
563
|
continue
|
|
549
564
|
answer = got.get("answer")
|
|
550
|
-
|
|
565
|
+
raw_quotes = got.get("evidence")
|
|
566
|
+
if raw_quotes is not None and not isinstance(raw_quotes, list):
|
|
567
|
+
problems.append(f"{check['id']} evidence must be a list of exact quotes")
|
|
568
|
+
raw_quotes = []
|
|
569
|
+
elif isinstance(raw_quotes, list) and any(not isinstance(q, str) for q in raw_quotes):
|
|
570
|
+
problems.append(f"{check['id']} evidence entries must all be strings")
|
|
571
|
+
quotes = [q for q in (raw_quotes or []) if isinstance(q, str)]
|
|
551
572
|
count = got.get("count")
|
|
552
573
|
|
|
574
|
+
if ("___" in check["title"] and "count" in check["title"].lower()
|
|
575
|
+
and (isinstance(count, bool) or not isinstance(count, int) or count < 0)):
|
|
576
|
+
problems.append(f"{check['id']} is missing a required count")
|
|
577
|
+
elif count is not None and (isinstance(count, bool)
|
|
578
|
+
or not isinstance(count, int) or count < 0):
|
|
579
|
+
problems.append(f"{check['id']} count must be a non-negative integer or null")
|
|
580
|
+
|
|
553
581
|
if answer == "fail" and not quotes:
|
|
554
582
|
problems.append(f"{check['id']} failed with no quoted evidence")
|
|
555
583
|
if answer == "fail" and isinstance(count, int) and count == 0:
|
|
@@ -572,6 +600,9 @@ def check_evidence(raw: str, answers: dict, checks: list[dict]) -> list[str]:
|
|
|
572
600
|
|
|
573
601
|
def verdict(text: str, answers: dict) -> tuple[int, str]:
|
|
574
602
|
"""Combine the measured rates with the model's read. Both must clear."""
|
|
603
|
+
if not isinstance(answers, dict):
|
|
604
|
+
return 1, ("Register verdict\n\n FAIL answer packet must be a JSON object "
|
|
605
|
+
"keyed by checklist id. No checks were accepted.")
|
|
575
606
|
m = measure(text)
|
|
576
607
|
checks = [c for c in load_checks() if not c["skip"] and not c["auto"]]
|
|
577
608
|
out = ["Register verdict", ""]
|
|
@@ -602,7 +633,10 @@ def verdict(text: str, answers: dict) -> tuple[int, str]:
|
|
|
602
633
|
out.append(f" {state:<5} {qid:<5}{check['title'][:48]}{shown}")
|
|
603
634
|
if got["answer"] == "fail":
|
|
604
635
|
failed.append(qid)
|
|
605
|
-
|
|
636
|
+
quotes = got.get("evidence")
|
|
637
|
+
if not isinstance(quotes, list):
|
|
638
|
+
quotes = []
|
|
639
|
+
for quote in quotes[:3]:
|
|
606
640
|
out.append(f" · {str(quote)[:84]}")
|
|
607
641
|
out.append("")
|
|
608
642
|
|
|
@@ -614,12 +648,53 @@ def verdict(text: str, answers: dict) -> tuple[int, str]:
|
|
|
614
648
|
out.append(" paragraph nobody read; the checklist was answered from memory.")
|
|
615
649
|
failed.append("coverage (missing)")
|
|
616
650
|
else:
|
|
651
|
+
known_paragraphs = set(para_ids)
|
|
652
|
+
# JSON object keys arrive as strings, but verdict() is also a public
|
|
653
|
+
# library function. Treat an in-process packet with non-string keys as
|
|
654
|
+
# invalid input instead of letting sorting or joining raise TypeError.
|
|
655
|
+
unknown = sorted(
|
|
656
|
+
(key for key in coverage if not isinstance(key, str)
|
|
657
|
+
or key not in known_paragraphs),
|
|
658
|
+
key=str,
|
|
659
|
+
)
|
|
617
660
|
unread = [i for i in para_ids if i not in coverage]
|
|
661
|
+
coverage_problems = []
|
|
662
|
+
valid_check_ids = {c["id"] for c in checks}
|
|
663
|
+
if unknown:
|
|
664
|
+
coverage_problems.append(
|
|
665
|
+
"unknown paragraph id(s): " + ", ".join(map(str, unknown[:8])))
|
|
666
|
+
for para_id in para_ids:
|
|
667
|
+
if para_id not in coverage:
|
|
668
|
+
continue
|
|
669
|
+
value = coverage[para_id]
|
|
670
|
+
if value == "clean":
|
|
671
|
+
continue
|
|
672
|
+
if not isinstance(value, list) or not value:
|
|
673
|
+
coverage_problems.append(
|
|
674
|
+
f"{para_id} must be clean or a non-empty list of failed check ids")
|
|
675
|
+
continue
|
|
676
|
+
if any(not isinstance(check_id, str) for check_id in value):
|
|
677
|
+
coverage_problems.append(f"{para_id} check ids must be strings")
|
|
678
|
+
continue
|
|
679
|
+
if len(value) != len(set(value)):
|
|
680
|
+
coverage_problems.append(f"{para_id} repeats a check id")
|
|
681
|
+
for check_id in value:
|
|
682
|
+
if check_id not in valid_check_ids:
|
|
683
|
+
coverage_problems.append(f"{para_id} names unknown check {check_id}")
|
|
684
|
+
else:
|
|
685
|
+
referenced = answers.get(check_id)
|
|
686
|
+
if not isinstance(referenced, dict) or referenced.get("answer") != "fail":
|
|
687
|
+
coverage_problems.append(
|
|
688
|
+
f"{para_id} names {check_id}, but that check did not fail")
|
|
618
689
|
if unread:
|
|
619
690
|
out.append(f" FAIL {len(unread)} paragraph(s) never dispositioned: "
|
|
620
691
|
+ ", ".join(unread[:8]))
|
|
621
692
|
failed.append("coverage (incomplete)")
|
|
622
|
-
|
|
693
|
+
if coverage_problems:
|
|
694
|
+
for problem in coverage_problems:
|
|
695
|
+
out.append(f" FAIL {problem}")
|
|
696
|
+
failed.append("coverage (invalid)")
|
|
697
|
+
if not unread and not coverage_problems:
|
|
623
698
|
flagged = sum(1 for v in coverage.values() if v != "clean")
|
|
624
699
|
out.append(f" ok all {len(para_ids)} paragraphs dispositioned, "
|
|
625
700
|
f"{flagged} carrying findings")
|
|
@@ -634,7 +709,12 @@ def verdict(text: str, answers: dict) -> tuple[int, str]:
|
|
|
634
709
|
out.append(" Re-read the draft and quote the exact span, or change the answer.")
|
|
635
710
|
else:
|
|
636
711
|
answered = sum(1 for c in checks if isinstance(answers.get(c["id"]), dict))
|
|
637
|
-
quoted =
|
|
712
|
+
quoted = 0
|
|
713
|
+
for check in checks:
|
|
714
|
+
got = answers.get(check["id"])
|
|
715
|
+
quotes = got.get("evidence") if isinstance(got, dict) else None
|
|
716
|
+
if isinstance(quotes, list):
|
|
717
|
+
quoted += sum(isinstance(quote, str) for quote in quotes)
|
|
638
718
|
if not answered:
|
|
639
719
|
out.append(" none no answer matched any check id. The answers file is for a")
|
|
640
720
|
out.append(" different checklist, or the ids are wrong.")
|
package/scripts/rerank.py
CHANGED
|
@@ -36,7 +36,7 @@ def _tier(s):
|
|
|
36
36
|
return 0 if s["preserved"] else 1
|
|
37
37
|
|
|
38
38
|
|
|
39
|
-
def rank(original, candidates, genre=None):
|
|
39
|
+
def rank(original, candidates, genre=None, adjudicated=None):
|
|
40
40
|
"""candidates: {name: text}. Returns them scored and sorted, best first.
|
|
41
41
|
|
|
42
42
|
Sort key: fidelity tier first (a fabrication can never win), then soft quality,
|
|
@@ -55,7 +55,7 @@ def rank(original, candidates, genre=None):
|
|
|
55
55
|
data = slopscore.load_patterns()
|
|
56
56
|
scored = []
|
|
57
57
|
for name, text in candidates.items():
|
|
58
|
-
s = slopscore.rewrite_score(original, text, genre, data)
|
|
58
|
+
s = slopscore.rewrite_score(original, text, genre, data, adjudicated)
|
|
59
59
|
s["name"], s["text"] = name, text
|
|
60
60
|
scored.append(s)
|
|
61
61
|
scored.sort(key=lambda s: (_tier(s), -s["soft"], s["after_ai"],
|
|
@@ -93,6 +93,8 @@ def main(argv=None):
|
|
|
93
93
|
ap.add_argument("--original", required=True, metavar="FILE")
|
|
94
94
|
ap.add_argument("--candidates", metavar="JSON_FILE")
|
|
95
95
|
ap.add_argument("--genre")
|
|
96
|
+
ap.add_argument("--adjudication", metavar="JSON_FILE",
|
|
97
|
+
help="source-bound rulings for figures intentionally removed")
|
|
96
98
|
ap.add_argument("--out", metavar="FILE")
|
|
97
99
|
ap.add_argument("--emit", action="store_true")
|
|
98
100
|
ap.add_argument("files", nargs="*")
|
|
@@ -122,8 +124,15 @@ def main(argv=None):
|
|
|
122
124
|
if len(candidates) < 2:
|
|
123
125
|
ap.error("give at least two candidate rewrites to choose between")
|
|
124
126
|
|
|
127
|
+
adjudicated = None
|
|
128
|
+
if args.adjudication:
|
|
129
|
+
import slopscore
|
|
130
|
+
try:
|
|
131
|
+
adjudicated = slopscore.load_adjudication(args.adjudication, original)
|
|
132
|
+
except ValueError as exc:
|
|
133
|
+
ap.error(str(exc))
|
|
125
134
|
try:
|
|
126
|
-
scored = rank(original, candidates, args.genre)
|
|
135
|
+
scored = rank(original, candidates, args.genre, adjudicated)
|
|
127
136
|
except ValueError as exc:
|
|
128
137
|
ap.error(str(exc))
|
|
129
138
|
print(render(scored))
|
package/scripts/safeio.py
CHANGED
|
@@ -1,9 +1,10 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
2
|
"""Small, stdlib-only primitives for durable and concurrency-safe state updates.
|
|
3
3
|
|
|
4
|
-
Zero Slop
|
|
5
|
-
|
|
6
|
-
|
|
4
|
+
Zero Slop never alters a scored draft. Learning and calibration update private state,
|
|
5
|
+
and an interactive score can update one local, one-time note marker. Atomic
|
|
6
|
+
replacement prevents torn files; lock directories prevent concurrent processes from
|
|
7
|
+
silently overwriting each other's state.
|
|
7
8
|
"""
|
|
8
9
|
from contextlib import contextmanager
|
|
9
10
|
import hashlib
|
package/scripts/slopscore.py
CHANGED
|
@@ -24,6 +24,8 @@ The phrase lists live beside this script in ../data/patterns.json and
|
|
|
24
24
|
"""
|
|
25
25
|
import bisect
|
|
26
26
|
import functools
|
|
27
|
+
import hashlib
|
|
28
|
+
import importlib.util
|
|
27
29
|
import json
|
|
28
30
|
import math
|
|
29
31
|
import re
|
|
@@ -37,7 +39,19 @@ SHAPE_SOLO_THRESHOLD = 0.62 # calibrated, see calibrate.py --shape
|
|
|
37
39
|
# Where personal voice profiles live — outside the repo, since they are the
|
|
38
40
|
# user's own writing. One file per author, git-ignored by construction.
|
|
39
41
|
import os
|
|
40
|
-
|
|
42
|
+
|
|
43
|
+
# The scorer is both a CLI and an importable single-file module. Load its
|
|
44
|
+
# adjacent helper by path so importlib callers do not have to modify sys.path,
|
|
45
|
+
# and so an unrelated third-party module named ``safeio`` cannot be selected.
|
|
46
|
+
_SAFEIO_SPEC = importlib.util.spec_from_file_location(
|
|
47
|
+
"_zero_slop_safeio", Path(__file__).resolve().with_name("safeio.py"))
|
|
48
|
+
if _SAFEIO_SPEC is None or _SAFEIO_SPEC.loader is None: # pragma: no cover
|
|
49
|
+
raise ImportError("cannot load Zero Slop's adjacent safeio.py")
|
|
50
|
+
_SAFEIO = importlib.util.module_from_spec(_SAFEIO_SPEC)
|
|
51
|
+
_SAFEIO_SPEC.loader.exec_module(_SAFEIO)
|
|
52
|
+
atomic_write_text = _SAFEIO.atomic_write_text
|
|
53
|
+
file_locks = _SAFEIO.file_locks
|
|
54
|
+
HOME = Path(os.environ.get("ZERO_SLOP_HOME") or Path.home() / ".zero-slop").expanduser()
|
|
41
55
|
VOICE_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}\Z")
|
|
42
56
|
|
|
43
57
|
|
|
@@ -77,13 +91,10 @@ def _load_notes():
|
|
|
77
91
|
|
|
78
92
|
def _save_notes(state):
|
|
79
93
|
try:
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
with open(tmp, "w", encoding="utf-8") as fh:
|
|
83
|
-
json.dump(state, fh)
|
|
84
|
-
os.replace(tmp, NOTES_FILE)
|
|
94
|
+
atomic_write_text(NOTES_FILE, json.dumps(state) + "\n", mode=0o600)
|
|
95
|
+
return True
|
|
85
96
|
except OSError:
|
|
86
|
-
|
|
97
|
+
return False # a read-only home must never break a score
|
|
87
98
|
|
|
88
99
|
|
|
89
100
|
def star_note_is_due(argv=None, isatty=None, env=None):
|
|
@@ -112,14 +123,22 @@ def record_human_run(argv=None, isatty=None, env=None):
|
|
|
112
123
|
return None
|
|
113
124
|
if not (sys.stdout.isatty() if isatty is None else isatty):
|
|
114
125
|
return None
|
|
115
|
-
|
|
116
|
-
|
|
126
|
+
try:
|
|
127
|
+
# A short best-effort lock prevents simultaneous terminal runs from
|
|
128
|
+
# both printing the one-time note. Contention or a read-only state
|
|
129
|
+
# directory must never delay or fail the score itself.
|
|
130
|
+
with file_locks([NOTES_FILE], timeout=0.25):
|
|
131
|
+
state = _load_notes()
|
|
132
|
+
if state.get("star_note_shown"):
|
|
133
|
+
return None
|
|
134
|
+
state["human_runs"] = int(state.get("human_runs", 0)) + 1
|
|
135
|
+
due = state["human_runs"] >= STAR_NOTE_AFTER_RUNS
|
|
136
|
+
if due:
|
|
137
|
+
state["star_note_shown"] = True
|
|
138
|
+
if not _save_notes(state):
|
|
139
|
+
return None
|
|
140
|
+
except (OSError, SystemExit):
|
|
117
141
|
return None
|
|
118
|
-
state["human_runs"] = int(state.get("human_runs", 0)) + 1
|
|
119
|
-
due = state["human_runs"] >= STAR_NOTE_AFTER_RUNS
|
|
120
|
-
if due:
|
|
121
|
-
state["star_note_shown"] = True
|
|
122
|
-
_save_notes(state)
|
|
123
142
|
if not due:
|
|
124
143
|
return None
|
|
125
144
|
return ("\n If Zero Slop has been useful, a star helps people find it: "
|
|
@@ -1201,14 +1220,15 @@ def facts(text, _other=""):
|
|
|
1201
1220
|
out = {}
|
|
1202
1221
|
for kind, rx in FACT_RX:
|
|
1203
1222
|
found = set()
|
|
1204
|
-
|
|
1223
|
+
flags = re.I if kind == "figure" else 0
|
|
1224
|
+
for m in re.finditer(rx, urls if kind == "url" else prose, flags):
|
|
1205
1225
|
v = (m.group(1) if m.lastindex else m.group(0)).strip()
|
|
1206
1226
|
if kind == "name":
|
|
1207
1227
|
v = _peel_entity(v, prose, other)
|
|
1208
1228
|
if not v:
|
|
1209
1229
|
continue
|
|
1210
1230
|
if kind == "figure":
|
|
1211
|
-
v = v.replace(",", "").lstrip("$").rstrip()
|
|
1231
|
+
v = v.replace(",", "").lstrip("$").rstrip().lower()
|
|
1212
1232
|
v = re.sub(r"\s*percent$", "%", v)
|
|
1213
1233
|
v = re.sub(r"\s*(million|bn|billion|m|k)$",
|
|
1214
1234
|
lambda x: {"million":"m","billion":"bn"}.get(x.group(1), x.group(1)), v)
|
|
@@ -1276,7 +1296,9 @@ BLOCKQUOTE_LINE_RX = re.compile(r"^[ \t]*>[^\n]*$", re.M)
|
|
|
1276
1296
|
HEADING_RX = re.compile(r"^(#{1,6})[ \t]+(.+?)[ \t]*$", re.M)
|
|
1277
1297
|
PATH_RX = re.compile(
|
|
1278
1298
|
r"(?<![\w:])((?:\.\.?/|/)[A-Za-z0-9._~\-]+"
|
|
1279
|
-
r"(?:/[A-Za-z0-9._~\-]+)*|[A-Za-
|
|
1299
|
+
r"(?:/[A-Za-z0-9._~\-]+)*|(?:[A-Za-z0-9._~\-]+/)+"
|
|
1300
|
+
r"[A-Za-z0-9._~\-]+\.[A-Za-z0-9._~\-]+|"
|
|
1301
|
+
r"[A-Za-z]:\\[A-Za-z0-9._\\~\-]+)"
|
|
1280
1302
|
)
|
|
1281
1303
|
|
|
1282
1304
|
|
|
@@ -1438,6 +1460,49 @@ def figure_contexts(text, figures):
|
|
|
1438
1460
|
return out
|
|
1439
1461
|
|
|
1440
1462
|
|
|
1463
|
+
def load_adjudication(path, original):
|
|
1464
|
+
"""Load explicit dropped-figure rulings bound to one exact source text.
|
|
1465
|
+
|
|
1466
|
+
The file is intentionally small and closed-schema. It cannot weaken name,
|
|
1467
|
+
quote, URL, feeling, or structure checks, and it cannot excuse a number that
|
|
1468
|
+
was not present in the source it names.
|
|
1469
|
+
"""
|
|
1470
|
+
source = Path(path)
|
|
1471
|
+
try:
|
|
1472
|
+
if source.stat().st_size > 65_536:
|
|
1473
|
+
raise ValueError("adjudication file exceeds 64 KiB")
|
|
1474
|
+
payload = json.loads(source.read_text(encoding="utf-8"))
|
|
1475
|
+
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
|
|
1476
|
+
raise ValueError(f"cannot read adjudication file: {exc}") from exc
|
|
1477
|
+
if not isinstance(payload, dict):
|
|
1478
|
+
raise ValueError("adjudication file must be a JSON object")
|
|
1479
|
+
expected = {"schema", "original_sha256", "allow_dropped_figures"}
|
|
1480
|
+
if set(payload) != expected or payload.get("schema") != 1:
|
|
1481
|
+
raise ValueError("adjudication file must use schema 1 and only documented keys")
|
|
1482
|
+
digest = payload.get("original_sha256")
|
|
1483
|
+
actual = hashlib.sha256(original.encode("utf-8")).hexdigest()
|
|
1484
|
+
if not isinstance(digest, str) or digest != actual:
|
|
1485
|
+
raise ValueError("adjudication source hash does not match the original text")
|
|
1486
|
+
raw = payload.get("allow_dropped_figures")
|
|
1487
|
+
if not isinstance(raw, list) or len(raw) > 100:
|
|
1488
|
+
raise ValueError("allow_dropped_figures must be a list of at most 100 figures")
|
|
1489
|
+
original_figures = facts(original)["figure"]
|
|
1490
|
+
allowed = []
|
|
1491
|
+
for item in raw:
|
|
1492
|
+
if not isinstance(item, str) or not item.strip() or len(item) > 80:
|
|
1493
|
+
raise ValueError("each allowed figure must be a short non-empty string")
|
|
1494
|
+
parsed = facts(item)["figure"]
|
|
1495
|
+
if len(parsed) != 1:
|
|
1496
|
+
raise ValueError(f"allowed figure is not one unambiguous figure: {item!r}")
|
|
1497
|
+
canonical = next(iter(parsed))
|
|
1498
|
+
if canonical not in original_figures:
|
|
1499
|
+
raise ValueError(f"allowed figure is absent from the original: {item!r}")
|
|
1500
|
+
allowed.append(canonical)
|
|
1501
|
+
if len(allowed) != len(set(allowed)):
|
|
1502
|
+
raise ValueError("allow_dropped_figures contains a duplicate")
|
|
1503
|
+
return set(allowed)
|
|
1504
|
+
|
|
1505
|
+
|
|
1441
1506
|
def fidelity(before, after, adjudicated=None):
|
|
1442
1507
|
"""Did the rewrite keep every fact, and did it add any?
|
|
1443
1508
|
|
|
@@ -1620,8 +1685,8 @@ def rewrite_score(before_text, after_text, genre=None, data=None,
|
|
|
1620
1685
|
"preserved": fid["preserved"], "invented": fid["invented"]}
|
|
1621
1686
|
|
|
1622
1687
|
|
|
1623
|
-
def render_fidelity(before, after):
|
|
1624
|
-
r = fidelity(before, after)
|
|
1688
|
+
def render_fidelity(before, after, adjudicated=None):
|
|
1689
|
+
r = fidelity(before, after, adjudicated)
|
|
1625
1690
|
out = ["", " FACT AND MEANING CHECK · original vs edited text", ""]
|
|
1626
1691
|
for kind, kept, dropped, added in r["rows"]:
|
|
1627
1692
|
out.append(f" {kind:<8} {len(kept)} kept"
|
|
@@ -1640,6 +1705,9 @@ def render_fidelity(before, after):
|
|
|
1640
1705
|
out.append(" protected document content changed:")
|
|
1641
1706
|
for finding in r["structure"][:8]:
|
|
1642
1707
|
out.append(f" {finding['code']:<23} {finding['message']}")
|
|
1708
|
+
if r.get("unsourced"):
|
|
1709
|
+
for figure in sorted(r["unsourced"]):
|
|
1710
|
+
out.append(f" ruled cut {figure!r} (reviewer marked it unsourced)")
|
|
1643
1711
|
out += ["",
|
|
1644
1712
|
" Result: " + ("facts preserved; nothing added"
|
|
1645
1713
|
if r["preserved"] and not r["invented"] else
|
|
@@ -1726,7 +1794,7 @@ def main():
|
|
|
1726
1794
|
if "--help" in argv or "-h" in argv:
|
|
1727
1795
|
print(__doc__)
|
|
1728
1796
|
return 0
|
|
1729
|
-
value_flags = {"--gate", "--genre", "--voice"}
|
|
1797
|
+
value_flags = {"--gate", "--genre", "--voice", "--adjudication"}
|
|
1730
1798
|
bool_flags = {"--json", "--explain", "--formal", "--fidelity", "--dna",
|
|
1731
1799
|
"--portfolio", "--batch", "--heatmap"}
|
|
1732
1800
|
unknown = [arg for arg in argv if arg.startswith("--")
|
|
@@ -1739,6 +1807,8 @@ def main():
|
|
|
1739
1807
|
if flag in argv]
|
|
1740
1808
|
if len(modes) > 1:
|
|
1741
1809
|
raise SystemExit("choose only one mode: " + ", ".join(modes))
|
|
1810
|
+
if "--adjudication" in argv and "--fidelity" not in argv:
|
|
1811
|
+
raise SystemExit("--adjudication is valid only with --fidelity")
|
|
1742
1812
|
|
|
1743
1813
|
gv, _ = gate_value()
|
|
1744
1814
|
# Values that belong to a flag (--gate 25, --genre social, --voice manav)
|
|
@@ -1773,9 +1843,16 @@ def main():
|
|
|
1773
1843
|
if len(args) != 2:
|
|
1774
1844
|
sys.exit("--fidelity needs exactly two files: before and after")
|
|
1775
1845
|
before, after = _read_text_file(args[0]), _read_text_file(args[1])
|
|
1776
|
-
|
|
1846
|
+
adjudicated = None
|
|
1847
|
+
ruling_path = _required_option_value(argv, "--adjudication")
|
|
1848
|
+
if ruling_path:
|
|
1849
|
+
try:
|
|
1850
|
+
adjudicated = load_adjudication(ruling_path, before)
|
|
1851
|
+
except ValueError as exc:
|
|
1852
|
+
raise SystemExit(str(exc)) from exc
|
|
1853
|
+
for line in render_fidelity(before, after, adjudicated):
|
|
1777
1854
|
print(line)
|
|
1778
|
-
r = fidelity(before, after)
|
|
1855
|
+
r = fidelity(before, after, adjudicated)
|
|
1779
1856
|
sys.exit(0 if (r["preserved"] and not r["invented"]) else 1)
|
|
1780
1857
|
|
|
1781
1858
|
if "--dna" in sys.argv:
|