zero-slop 2.5.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +193 -0
- package/SKILL.md +803 -0
- package/data/corpus/community-register/README.md +14 -0
- package/data/corpus/community-register/judgment/communicative-drift.txt +1 -0
- package/data/corpus/community-register/judgment/reflexive-agreement.txt +1 -0
- package/data/corpus/community-register/judgment/rhetorical-scale-mismatch.txt +1 -0
- package/data/corpus/must-not-flag/README.md +49 -0
- package/data/corpus/must-not-flag/esl-engineer-email.txt +7 -0
- package/data/corpus/must-not-flag/exec-memo.txt +9 -0
- package/data/corpus/must-not-flag/federalist.txt +5 -0
- package/data/corpus/must-not-flag/gettysburg.txt +1 -0
- package/data/corpus/must-not-flag/grant-abstract.txt +5 -0
- package/data/corpus/must-not-flag/ml-methods.txt +1 -0
- package/data/corpus/must-not-flag/personal-essay.txt +1 -0
- package/data/corpus/must-not-flag/press-release.txt +7 -0
- package/data/corpus/must-not-flag/recipe.txt +11 -0
- package/data/corpus/must-not-flag/sre-runbook.txt +1 -0
- package/data/corpus/must-not-flag/technical-postmortem.txt +1 -0
- package/data/corpus/must-not-flag/terse-engineer-note.txt +1 -0
- package/data/corpus/must-not-flag-shape/README.md +22 -0
- package/data/corpus/must-not-flag-shape/aphorism-list.txt +9 -0
- package/data/corpus/must-not-flag-shape/changelog.txt +13 -0
- package/data/corpus/must-not-flag-shape/dialogue.txt +19 -0
- package/data/corpus/must-not-flag-shape/lyrics.txt +15 -0
- package/data/corpus/must-not-flag-shape/poem.txt +8 -0
- package/data/corpus/must-not-flag-shape/sms-transcript.txt +15 -0
- package/data/corpus/performed-register/README.md +38 -0
- package/data/corpus/performed-register/judgment/detection-fragile.txt +1 -0
- package/data/corpus/performed-register/judgment/draft-cheap-signal.txt +1 -0
- package/data/corpus/performed-register/judgment/ear-chiasmus.txt +1 -0
- package/data/corpus/performed-register/judgment/fight-against-tagline.txt +1 -0
- package/data/corpus/performed-register/judgment/fine-beat.txt +1 -0
- package/data/corpus/performed-register/judgment/glow-finishing.txt +1 -0
- package/data/corpus/performed-register/judgment/jargon-compression.txt +1 -0
- package/data/corpus/performed-register/judgment/not-perfect-honest.txt +1 -0
- package/data/corpus/performed-register/judgment/punctuation-convict.txt +1 -0
- package/data/corpus/performed-register/judgment/recipe-ingredients.txt +1 -0
- package/data/corpus/performed-register/judgment/replay-lands.txt +1 -0
- package/data/corpus/performed-register/judgment/rhymes-with.txt +1 -0
- package/data/corpus/performed-register/judgment/rhythm-prints.txt +1 -0
- package/data/corpus/performed-register/judgment/same-leak.txt +1 -0
- package/data/corpus/performed-register/judgment/tools-work.txt +1 -0
- package/data/corpus/performed-register/judgment/tuning-gradients.txt +1 -0
- package/data/corpus/performed-register/judgment/verdict-arithmetic.txt +1 -0
- package/data/corpus/performed-register/judgment/whole-product.txt +1 -0
- package/data/corpus/performed-register/judgment/words-cheap.txt +1 -0
- package/data/corpus/performed-register/mechanical/argue-with-artifact.txt +1 -0
- package/data/corpus/performed-register/mechanical/bill-lands.txt +1 -0
- package/data/corpus/performed-register/mechanical/billed-to-reader.txt +1 -0
- package/data/corpus/performed-register/mechanical/economics-brutal.txt +1 -0
- package/data/corpus/performed-register/mechanical/has-receipts.txt +1 -0
- package/data/corpus/performed-register/mechanical/hired-adversary.txt +1 -0
- package/data/corpus/performed-register/mechanical/hyperbole-universal.txt +1 -0
- package/data/corpus/performed-register/mechanical/metaphor-sender-name.txt +1 -0
- package/data/corpus/performed-register/mechanical/minding-own-business.txt +1 -0
- package/data/corpus/performed-register/mechanical/on-the-tin.txt +1 -0
- package/data/corpus/performed-register/mechanical/opens-the-hood.txt +1 -0
- package/data/corpus/performed-register/mechanical/turns-out-payoff.txt +1 -0
- package/data/corpus/performed-register/mechanical/vibe-check.txt +1 -0
- package/data/corpus/performed-register/mechanical/vibe-isnt-a-vibe.txt +1 -0
- package/data/corpus/performed-register/mechanical/vibe-number-attached.txt +1 -0
- package/data/corpus/performed-register/mechanical/where-slop-lives.txt +1 -0
- package/data/learned-log.md +339 -0
- package/data/learned.json +128 -0
- package/data/patterns.json +2298 -0
- package/package.json +54 -0
- package/references/copy-desk.md +87 -0
- package/references/evidence.md +382 -0
- package/references/overcorrection.md +77 -0
- package/references/platforms.md +102 -0
- package/references/readalong.md +118 -0
- package/references/rewrite-moves.md +177 -0
- package/references/tells.md +190 -0
- package/scripts/calibrate.py +274 -0
- package/scripts/learn.py +1588 -0
- package/scripts/predictability.py +227 -0
- package/scripts/rerank.py +150 -0
- package/scripts/safeio.py +146 -0
- package/scripts/slopscore.py +1352 -0
- package/scripts/version_check.py +104 -0
package/package.json
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "zero-slop",
|
|
3
|
+
"version": "2.5.8",
|
|
4
|
+
"description": "An Agent Skill that scores AI-sounding prose 0-100 locally and rewrites it without losing a fact. Runs inside Claude Code, Codex, Cursor, Warp, Zed and other SKILL.md agents. Offline, zero dependencies, MIT.",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"agent-skill",
|
|
7
|
+
"agent-skills",
|
|
8
|
+
"skills",
|
|
9
|
+
"claude-code",
|
|
10
|
+
"claude-skill",
|
|
11
|
+
"codex",
|
|
12
|
+
"cursor",
|
|
13
|
+
"ai-slop",
|
|
14
|
+
"de-slop",
|
|
15
|
+
"humanizer",
|
|
16
|
+
"humanize-ai-text",
|
|
17
|
+
"ai-detection",
|
|
18
|
+
"ai-writing",
|
|
19
|
+
"writing-tools",
|
|
20
|
+
"editing",
|
|
21
|
+
"prose"
|
|
22
|
+
],
|
|
23
|
+
"homepage": "https://zero-slop.ai",
|
|
24
|
+
"bugs": "https://github.com/manavmishra/ZeroSlop/issues",
|
|
25
|
+
"repository": {
|
|
26
|
+
"type": "git",
|
|
27
|
+
"url": "git+https://github.com/manavmishra/ZeroSlop.git"
|
|
28
|
+
},
|
|
29
|
+
"license": "MIT",
|
|
30
|
+
"author": {
|
|
31
|
+
"name": "Manav Mishra",
|
|
32
|
+
"url": "https://zero-slop.ai/about/"
|
|
33
|
+
},
|
|
34
|
+
"files": [
|
|
35
|
+
"SKILL.md",
|
|
36
|
+
"references/",
|
|
37
|
+
"scripts/",
|
|
38
|
+
"data/",
|
|
39
|
+
"README.md",
|
|
40
|
+
"LICENSE",
|
|
41
|
+
"!scripts/build_bundle.py",
|
|
42
|
+
"!scripts/build_onepager_pdf.py",
|
|
43
|
+
"!scripts/build_plugin.py",
|
|
44
|
+
"!scripts/build_skill_zip.py",
|
|
45
|
+
"!scripts/check_svg.py",
|
|
46
|
+
"!scripts/contextual.py",
|
|
47
|
+
"!scripts/contextual-signals.md",
|
|
48
|
+
"!references/contextual-signals.md",
|
|
49
|
+
"!scripts/growth-snapshot.mjs",
|
|
50
|
+
"!data/voices/",
|
|
51
|
+
"!**/__pycache__/"
|
|
52
|
+
],
|
|
53
|
+
"private": false
|
|
54
|
+
}
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# The final copy desk
|
|
2
|
+
|
|
3
|
+
Run this pass on every deliverable after the rewrite and initial verification
|
|
4
|
+
work. It corrects mechanics and line-level style before the final read-aloud
|
|
5
|
+
pass. Its job is to produce corrected copy, not an assessment of copy that still
|
|
6
|
+
needs work.
|
|
7
|
+
|
|
8
|
+
Use a dedicated copy-editor agent with fresh eyes when the harness supports
|
|
9
|
+
subagents. Otherwise, set aside the writing mindset and perform the same review
|
|
10
|
+
as a separate, role-isolated pass. Either way, apply every accepted correction
|
|
11
|
+
to the actual deliverable before sending it to the final read-aloud pass.
|
|
12
|
+
|
|
13
|
+
## What to correct
|
|
14
|
+
|
|
15
|
+
Read the complete piece from title to final line, including headings, captions,
|
|
16
|
+
list items, table labels, calls to action, and user-authored alt text. Correct:
|
|
17
|
+
|
|
18
|
+
- spelling, typos, duplicated or missing words, and inconsistent regional usage;
|
|
19
|
+
- grammar, agreement, tense, articles, prepositions, pronouns, and modifiers;
|
|
20
|
+
- punctuation, capitalization, quotation marks, hyphenation, and compound terms;
|
|
21
|
+
- broken parallelism, unclear antecedents, misplaced clauses, and ambiguity;
|
|
22
|
+
- awkward syntax, needless repetition, imprecise diction, and choppy transitions;
|
|
23
|
+
- phrasing that is stiff, clumsy, fussy, unintentionally casual, or unprofessional;
|
|
24
|
+
- internal scoring, editing, or workflow labels that have leaked into reader-facing
|
|
25
|
+
prose; replace each with plain language unless the audience needs the technical
|
|
26
|
+
term;
|
|
27
|
+
- inconsistencies in terminology, headings, labels, voice, and editorial style.
|
|
28
|
+
|
|
29
|
+
Aim for tasteful, elegant, professional prose in the register the piece calls
|
|
30
|
+
for. Elegance means precise, natural, and restrained. It does not mean ornate,
|
|
31
|
+
formal, promotional, or generic. Preserve warmth, humor, contractions, fragments,
|
|
32
|
+
dialect, and technical language when they are intentional and suit the genre.
|
|
33
|
+
Keep the author's regional spelling unless the user or publication specifies a
|
|
34
|
+
different house style.
|
|
35
|
+
|
|
36
|
+
## Boundaries
|
|
37
|
+
|
|
38
|
+
- Preserve every fact, claim, qualifier, number, name, date, link, and source.
|
|
39
|
+
- Do not invent detail, strengthen certainty, soften a warranted conclusion, or
|
|
40
|
+
change the author's position for smoother prose.
|
|
41
|
+
- Do not edit quotations, code, commands, identifiers, file paths, data, legal
|
|
42
|
+
boilerplate, or proper names unless the user explicitly asks.
|
|
43
|
+
- Preserve the input format and all non-prose structure. Correct content inside
|
|
44
|
+
a structured form without changing the form itself.
|
|
45
|
+
- Do not replace a distinctive human phrase merely because a more conventional
|
|
46
|
+
phrase exists. Correct errors and genuine awkwardness, not personality.
|
|
47
|
+
- When a sentence is ambiguous and the intended meaning cannot be recovered
|
|
48
|
+
safely, leave it unchanged and flag the ambiguity instead of guessing.
|
|
49
|
+
|
|
50
|
+
## Copy-editor agent brief
|
|
51
|
+
|
|
52
|
+
Give the agent the genre, intended audience, original draft, selected rewrite,
|
|
53
|
+
known voice signals, and immutable facts when available. Then use this brief:
|
|
54
|
+
|
|
55
|
+
> You are the copy editor for a publication-ready piece. Edit the complete
|
|
56
|
+
> text directly and return the corrected version in the same format. Fix every
|
|
57
|
+
> genuine error in spelling, grammar, punctuation, capitalization, agreement,
|
|
58
|
+
> tense, modifiers, syntax, diction, consistency, and usage. Improve awkward,
|
|
59
|
+
> unclear, repetitive, clumsy, or unprofessional phrasing where the meaning is
|
|
60
|
+
> certain. Remove internal evaluation or workflow language from outward-facing
|
|
61
|
+
> prose when plain language says the same thing. Make the prose tasteful, elegant,
|
|
62
|
+
> precise, natural, and appropriate for its genre and audience. Preserve the
|
|
63
|
+
> author's voice, regional spelling, facts, claims, qualifiers, names, numbers,
|
|
64
|
+
> links, structure, and intentional fragments. Do not add detail, hype, certainty,
|
|
65
|
+
> or generic polish. Do not alter
|
|
66
|
+
> quotations, code, commands, identifiers, or data. If a correction would
|
|
67
|
+
> require guessing what the author means, leave that span unchanged and flag it.
|
|
68
|
+
> Return: (1) the fully corrected text, not a list of suggestions; and (2) a
|
|
69
|
+
> terse note only for unresolved ambiguities or unusual forms deliberately kept.
|
|
70
|
+
|
|
71
|
+
Apply the corrected version to the deliverable, then send that exact artifact
|
|
72
|
+
through the final read-aloud pass in `references/readalong.md`. Verify the
|
|
73
|
+
artifact returned by the read-aloud editor:
|
|
74
|
+
|
|
75
|
+
1. Rerun the heuristic surface scorer and scripted fidelity check.
|
|
76
|
+
2. Compare it directly with both the original and selected rewrite for preserved
|
|
77
|
+
claims, qualifiers, intended voice, regional spelling, format, and non-prose
|
|
78
|
+
structure. The script cannot detect every semantic or stylistic change.
|
|
79
|
+
3. If any check or comparison requires a repair, apply it, send the repaired
|
|
80
|
+
text through the copy desk and final read-aloud pass again, and repeat all
|
|
81
|
+
final checks.
|
|
82
|
+
|
|
83
|
+
Stop only when the same artifact clears the copy desk, final read-aloud pass,
|
|
84
|
+
semantic and format review, scorer, and fidelity check. Limit this repair loop to
|
|
85
|
+
three rounds. If an issue still cannot be resolved without guessing, return the best
|
|
86
|
+
source-preserving version that completed both editorial passes, flag the unresolved
|
|
87
|
+
span and failed check plainly, and do not describe the fallback as fully verified.
|
|
@@ -0,0 +1,382 @@
|
|
|
1
|
+
# Evidence Base — why the ladder is ordered the way it is
|
|
2
|
+
|
|
3
|
+
Every rule in this skill traces to measured findings. This file is the chain
|
|
4
|
+
of custody. Full dossier with per-paper notes lives in the research archive;
|
|
5
|
+
citations here are the load-bearing ones.
|
|
6
|
+
|
|
7
|
+
## The central finding
|
|
8
|
+
|
|
9
|
+
Commercial detectors (GPTZero, Pangram) rate *base-model* text as ~97–99%
|
|
10
|
+
human while flagging *instruction-tuned* output from the same model
|
|
11
|
+
(arXiv:2605.19516). Detectors are classifiers of the **post-training (RLHF)
|
|
12
|
+
register** — the polished, uniform, preference-optimized voice — not of
|
|
13
|
+
machine generation itself. That register lives entirely in surface
|
|
14
|
+
realization, which is why rewriting can remove it without touching meaning.
|
|
15
|
+
It is also why "humanizing" is legitimate editing: the target is a register,
|
|
16
|
+
not a deception.
|
|
17
|
+
|
|
18
|
+
## Detection features, ranked by evidence strength
|
|
19
|
+
|
|
20
|
+
1. **Token-level predictability.** AI text sits at local maxima of model
|
|
21
|
+
log-probability (DetectGPT, arXiv:2301.11305; Binoculars, 2401.12070).
|
|
22
|
+
Human text doesn't. **Now measured** (v2.2, `scripts/predictability.py`) —
|
|
23
|
+
see the model-channel note below. Counter: Ladder L1: specific, slightly
|
|
24
|
+
surprising phrasing and concrete facts are the direct counter.
|
|
25
|
+
2. **The LLM lexicon.** A few hundred style words carry huge evidential
|
|
26
|
+
weight: "meticulous" +34.7x, "commendable" +9.8x, "intricate" +11.2x in
|
|
27
|
+
post-ChatGPT scientific text (Liang, 2403.07183); ~900 excess words
|
|
28
|
+
catalogued across 15M PubMed abstracts (Kobak, 2406.07016 —
|
|
29
|
+
github.com/berenslab/llm-excess-vocab); 21 focal words traced to RLHF
|
|
30
|
+
(Juzek & Ward, 2412.11385). Era-dependent: delve peaked 2023–24;
|
|
31
|
+
enhance/highlight/showcase dominate 2025+. Counter: Ladder L5 + the scorer's
|
|
32
|
+
weighted lexicon + the learning loop's era updates.
|
|
33
|
+
3. **Surprisal uniformity.** Humans spike information density unevenly; LLMs
|
|
34
|
+
smooth it (GPT-who/UID, 2310.06202 — beats commercial detectors by >20%). Counter: L3's "don't pad every claim to equal weight".
|
|
35
|
+
4. **Low burstiness.** Uniform sentence length/structure: GPTZero's founding
|
|
36
|
+
feature, corroborated independently (Muñoz-Ortiz, 2308.09067; Reinhart,
|
|
37
|
+
PNAS 2410.16107). Human sentence-length CV is simply higher. Counter: the
|
|
38
|
+
scorer's burstiness metric and the ≥0.45 gate.
|
|
39
|
+
5. **Register rigidity.** LLMs hold one polished expository voice regardless
|
|
40
|
+
of situation; humans shift register (Reinhart). Counter: L4.
|
|
41
|
+
6. **Stance asymmetry.** Humans: first-person stance, modals, selective
|
|
42
|
+
epistemic hedging, discourse-marker cohesion. LLMs: nominalizations,
|
|
43
|
+
formal connectives, paragraph-architecture cohesion (Herbold, 2304.14276). Counter: L4's hedging rules and "connective texture over scaffolding".
|
|
44
|
+
7. **Syntax signature.** Nominalization density, present-participial
|
|
45
|
+
clauses, longer constituents (Reinhart; Muñoz-Ortiz). Counter: de-nominalize;
|
|
46
|
+
kill participial openers.
|
|
47
|
+
8. **Affect skew.** LLM text is joy-skewed and uniformly positive
|
|
48
|
+
(Muñoz-Ortiz). Counter: widen affect.
|
|
49
|
+
|
|
50
|
+
## Why the scorer measures features, not detector verdicts
|
|
51
|
+
|
|
52
|
+
Detectors are brittle: RAID (2405.07940) shows trivial perturbations fool
|
|
53
|
+
them, and DIPPER (2303.13408) shows one paraphrase pass drops DetectGPT from
|
|
54
|
+
70% to 4.6% detection. Verdicts are therefore neither necessary nor
|
|
55
|
+
sufficient. The features they key on, however, are exactly what human readers
|
|
56
|
+
report as "sounds like AI" — so the scorer tracks the features directly:
|
|
57
|
+
weighted tell density, lexicon hits, burstiness, formatting densities,
|
|
58
|
+
register signals. Passing the gate means "the measurable tells are gone",
|
|
59
|
+
which is the honest, robust target.
|
|
60
|
+
|
|
61
|
+
## Span-first diagnosis, not a binary vibe check
|
|
62
|
+
|
|
63
|
+
Shaib et al., *Measuring AI “Slop” in Text* (arXiv:2509.19163), built a
|
|
64
|
+
taxonomy from 19 experts and span annotations by professional copy editors.
|
|
65
|
+
The useful split is broader than style alone: information utility (density and
|
|
66
|
+
relevance), information quality (factuality and appropriate perspective), and
|
|
67
|
+
style quality (repetition, templatedness, coherence, fluency, verbosity, word
|
|
68
|
+
complexity, and tone). Which dimensions mattered changed by domain. Factual and
|
|
69
|
+
structural problems mattered most in short answers; utility and tone mattered
|
|
70
|
+
more in news.
|
|
71
|
+
|
|
72
|
+
The negative results are just as important. Pairwise agreement on the binary
|
|
73
|
+
slop label was poor to fair, automatic linear models reached only 0.52 and 0.55
|
|
74
|
+
AUPRC on the two datasets, and zero-shot LLM judges under-predicted slop with
|
|
75
|
+
recall of 0.08–0.12. Prompted span extraction also aligned poorly with the human
|
|
76
|
+
annotations. That evidence rules out an ungrounded "does this feel like slop?"
|
|
77
|
+
model verdict as a reliable gate. Zero Slop therefore uses a span-first,
|
|
78
|
+
category-specific diagnosis, keeps deterministic surface measurements separate,
|
|
79
|
+
and reserves relevance, coherence, tone, and factual judgments for explicit
|
|
80
|
+
review rather than laundering them into the 0–100 meter.
|
|
81
|
+
|
|
82
|
+
Source: <https://arxiv.org/abs/2509.19163> (CC BY 4.0).
|
|
83
|
+
|
|
84
|
+
## Cross-draft templating
|
|
85
|
+
|
|
86
|
+
The Slop Index evaluates 19,928 model generations against pre-ChatGPT human
|
|
87
|
+
baselines. Its most portable idea for an editor is not its composite ranking;
|
|
88
|
+
it is measuring repeated five-word openings across several responses to the
|
|
89
|
+
same prompt. A single document cannot expose that failure. The same project
|
|
90
|
+
also found that some plausible measures reverse direction by genre, so its
|
|
91
|
+
rhythm axis is used only for email and its weights are renormalized when a
|
|
92
|
+
baseline cannot support an axis.
|
|
93
|
+
|
|
94
|
+
Zero Slop adopts the conservative part of that method in `--portfolio`: report
|
|
95
|
+
exact repeated openings and shared multiword templates across three or more
|
|
96
|
+
related drafts. The result does not change the surface score. There is not yet
|
|
97
|
+
enough labeled, cross-genre evidence to assign it a safe universal weight, and
|
|
98
|
+
necessary domain phrases can legitimately recur.
|
|
99
|
+
|
|
100
|
+
Source: <https://github.com/hgaddipati1118/slop-index> (MIT).
|
|
101
|
+
|
|
102
|
+
## What rewriting cannot do (the honesty boundary)
|
|
103
|
+
|
|
104
|
+
- Retrieval/watermarking by providers survives any rewrite (DIPPER's
|
|
105
|
+
conclusion). Fine — this skill's goal is reader-experienced quality, not
|
|
106
|
+
evasion.
|
|
107
|
+
- Character-level tricks and fake typos fool detectors (RAID) but degrade
|
|
108
|
+
writing. Banned.
|
|
109
|
+
- Hollow content scores clean on every surface metric. Only the removal test
|
|
110
|
+
catches it, which is why the judgment pass can never be skipped and why
|
|
111
|
+
hollow spans are flagged, not padded.
|
|
112
|
+
- The fidelity check compares tokens and entities, not meaning. v2 hardened it
|
|
113
|
+
against the two false alarms it raised most — a number spelled out ("18"
|
|
114
|
+
versus "eighteen") and a common word capitalised at a sentence start read as
|
|
115
|
+
an invented name — but a paraphrase that keeps every entity while bending a
|
|
116
|
+
claim can still pass it. Semantic fidelity, entailment between the draft and
|
|
117
|
+
the rewrite, is the roadmap; until then the judgment pass carries what the
|
|
118
|
+
token check cannot see.
|
|
119
|
+
|
|
120
|
+
## Reader-reported salience: the Reddit study
|
|
121
|
+
|
|
122
|
+
JCarterJohnson's 2026 analysis pulled 89,239 Reddit posts from 47 subreddits
|
|
123
|
+
covering 2021–2026, filtered 7,984 posts about recognizing AI-flavoured
|
|
124
|
+
writing, and manually reviewed a 600-post high-engagement sample. In that
|
|
125
|
+
sample, readers cited flat rhythm, reflexive praise or agreement, formulaic
|
|
126
|
+
shape, and fluent-but-empty prose more often than most individual words. The
|
|
127
|
+
keyword pass produced the opposite error: ordinary words such as "however",
|
|
128
|
+
"thus", "hence", "nuanced", "comprehensive", and "utilize" matched often
|
|
129
|
+
even though audited readers almost never named them as tells.
|
|
130
|
+
|
|
131
|
+
This is useful evidence about reader salience, not a prevalence estimate or
|
|
132
|
+
authorship detector. The source describes its audience as vocal, online
|
|
133
|
+
people; the sample is biased toward recent, high-engagement posts, and its
|
|
134
|
+
keyword analysis can confuse using a term with discussing it. Zero Slop uses
|
|
135
|
+
the relative ordering to put meaning, stance, rhythm, and document shape
|
|
136
|
+
before isolated vocabulary. It adds three context-only review traits:
|
|
137
|
+
reflexive agreement, communicative drift, and rhetorical scale mismatch. It
|
|
138
|
+
does not import the study's percentages as weights, ban em dashes, or promote
|
|
139
|
+
ordinary connectors into the mechanical scorer.
|
|
140
|
+
|
|
141
|
+
The accompanying `unslop-ai-text` skill and scanner were audited at commit
|
|
142
|
+
`f7c4aefc2c797a66e55b49354a93917ab60d33ac`. Its severity levels, JSON/CI
|
|
143
|
+
interface, sample-based register guidance, and read-aloud review are useful
|
|
144
|
+
corroboration. Zero Slop already covered all 25 categories in the published
|
|
145
|
+
tally through its six-family taxonomy, contextual review, scorer, voice
|
|
146
|
+
profile, and final editorial passes. We adopted the useful machine-readable
|
|
147
|
+
batch contract and reader-priority lesson, but no code, weights, or blanket
|
|
148
|
+
single-mark rule. On Zero Slop's 13-file known-human safety set, that upstream
|
|
149
|
+
scanner reported three high, two medium, and one low finding and returned a
|
|
150
|
+
"strong" verdict; Zero Slop kept all 13 files below its gate. That comparison
|
|
151
|
+
is a false-positive safety check, not an independent test of editorial quality.
|
|
152
|
+
|
|
153
|
+
Sources:
|
|
154
|
+
|
|
155
|
+
- Study and methodology: <https://www.reddit.com/r/ClaudeAI/comments/1ucpw87/i_pulled_90000_reddit_posts_about_what_makes/>
|
|
156
|
+
- Skill comparison: <https://www.reddit.com/r/ClaudeAI/comments/1uel1dc/unsloptext_skill_vs_humanizer_skill_part_2/>
|
|
157
|
+
- Launch discussion: <https://www.reddit.com/r/ClaudeAI/comments/1udl9hg/unsloptext_a_claude_skill_that_flags_and_removes/>
|
|
158
|
+
- Data, scanner, and skill: <https://github.com/JCarterJohnson/vibecoded-design-tells/tree/main/unslop-ai-text> (MIT).
|
|
159
|
+
|
|
160
|
+
## Practitioner corroboration
|
|
161
|
+
|
|
162
|
+
Paul Graham's "Write Like You Talk" and "How to Write Usefully" predate LLMs
|
|
163
|
+
and independently prescribe the same counters: spoken register (high
|
|
164
|
+
perplexity relative to formal boilerplate), maximal-strength claims without
|
|
165
|
+
overclaiming, qualification as precision. Wikipedia's WP:AICATCH — the
|
|
166
|
+
largest human-curated corpus of caught-in-the-wild AI text — converges on the
|
|
167
|
+
same tell families and adds the cluster rule this skill inherits: one tell is
|
|
168
|
+
coincidence; repeated agreement across several tells is meaningful.
|
|
169
|
+
|
|
170
|
+
## Negative results: trained classifiers
|
|
171
|
+
|
|
172
|
+
Several trained-classifier approaches were evaluated for an additional
|
|
173
|
+
detection channel. None shipped, and the reasoning generalises beyond the
|
|
174
|
+
specific methods.
|
|
175
|
+
|
|
176
|
+
**The ones that added nothing.** Two well-established classifier families were
|
|
177
|
+
ruled out on published evidence: one shows no measurable gain over a simpler
|
|
178
|
+
model on text classification while adding a dependency to a package whose value
|
|
179
|
+
includes having none, and the other models a sequence signal that the burstiness
|
|
180
|
+
and followability statistics already capture more cheaply and more legibly.
|
|
181
|
+
|
|
182
|
+
**The one that worked, and was cut anyway.** A trained channel built on lexical
|
|
183
|
+
frequency and stylometric features performed well in-domain: 0.985 AUC and 94%
|
|
184
|
+
accuracy on held-out data, properly calibrated with an abstain band. It was
|
|
185
|
+
built, integrated, and then removed. The transfer test is why. Trained on
|
|
186
|
+
2022-era text, it rated 2026-era AI drafts as human at a mean probability of
|
|
187
|
+
0.038, and in a live check returned 0.33 on a passage the pattern meter scored
|
|
188
|
+
100 out of 100. Detector decay across model generations is well documented
|
|
189
|
+
(RAID, arXiv:2405.07940); this was that decay measured directly. A channel that
|
|
190
|
+
is confidently wrong on current text is worse than no channel, even reported
|
|
191
|
+
separately and labelled a second opinion.
|
|
192
|
+
|
|
193
|
+
The lesson generalises. Interpretable surface features degrade gracefully as
|
|
194
|
+
models change, because updating them means editing a data file and the drift is
|
|
195
|
+
visible in the diff. A trained classifier degrades silently, and silence is the
|
|
196
|
+
failure mode you cannot audit.
|
|
197
|
+
|
|
198
|
+
## Fairness: who a false positive lands on
|
|
199
|
+
|
|
200
|
+
Liang et al. ran seven commercial GPT detectors over two human-written corpora,
|
|
201
|
+
US eighth-grade essays and TOEFL essays by non-native English speakers. Native
|
|
202
|
+
samples scored near-perfectly. **More than half of the non-native samples were
|
|
203
|
+
misclassified as AI-generated** (arXiv:2304.02819; *Patterns* 4:100779, 2023).
|
|
204
|
+
|
|
205
|
+
The mechanism matters more than the headline. The same study found that
|
|
206
|
+
enriching word choice in the non-native samples reduced misclassification,
|
|
207
|
+
while simplifying the native samples increased it. The detectors were keying on
|
|
208
|
+
linguistic complexity, so anything measuring "does this read as polished" will
|
|
209
|
+
penalize the writers with the least room to perform polish.
|
|
210
|
+
|
|
211
|
+
That is a direct hazard for this skill, which measures surface features by
|
|
212
|
+
design. Two consequences are load-bearing rather than decorative. Burstiness
|
|
213
|
+
and followability are scored as *bands*, not as "more is better", so plain
|
|
214
|
+
sentences are never evidence on their own. And
|
|
215
|
+
`data/corpus/must-not-flag/esl-engineer-email.txt` is in the corpus every new
|
|
216
|
+
pattern must clear, so the reflect loop cannot learn a rule that flags
|
|
217
|
+
competent non-native writing however many people cut the phrase.
|
|
218
|
+
|
|
219
|
+
## The reflect loop: why recurrence is the gate
|
|
220
|
+
|
|
221
|
+
The learning path (`scripts/learn.py`) takes its labels from the difference
|
|
222
|
+
between what the skill returned and what the author published. Two design
|
|
223
|
+
choices come from the failure modes above rather than from preference.
|
|
224
|
+
|
|
225
|
+
*Learning must be two-directional.* A loop that only adds patterns can only
|
|
226
|
+
grow, and a detector that only grows converges on flagging everything —
|
|
227
|
+
which, per Liang, lands hardest on the writers already over-flagged. So a
|
|
228
|
+
pattern the author overrules repeatedly loses weight.
|
|
229
|
+
|
|
230
|
+
*A single edit is not evidence.* Authors cut for length, fix facts, and change
|
|
231
|
+
their minds, and none of that is a style signal. Requiring the same span across
|
|
232
|
+
three independent documents is a crude significance test: it separates a
|
|
233
|
+
construction that recurs across unrelated writing from one person's sentence.
|
|
234
|
+
The same property is what makes an upstream contribution shareable without
|
|
235
|
+
carrying private text.
|
|
236
|
+
|
|
237
|
+
The honest limit: this measures what *writers who use the skill* strike, not
|
|
238
|
+
what readers detect. It tracks the register those authors are editing away
|
|
239
|
+
from, which is the target, but it is a convenience sample and no substitute for
|
|
240
|
+
the frequency work in `calibrate.py` against a real corpus.
|
|
241
|
+
|
|
242
|
+
## Practitioner corroboration: Kagi SlopStop
|
|
243
|
+
|
|
244
|
+
Kagi ships a production system for the same problem at web scale, and it
|
|
245
|
+
converged on three of the design decisions here independently, which is the
|
|
246
|
+
closest thing to external validation this architecture has.
|
|
247
|
+
|
|
248
|
+
**Corroboration before classification.** A domain is downranked when it is *mostly*
|
|
249
|
+
AI-generated, typically above 80% of its pages, rather than on a single hit.
|
|
250
|
+
That is the same principle as requiring several signals to agree, arrived at from
|
|
251
|
+
ranking rather than from linting.
|
|
252
|
+
|
|
253
|
+
**Multiple reports accelerate review.** Kagi's community reporting treats
|
|
254
|
+
repeated independent flags on one domain as stronger evidence than one flag.
|
|
255
|
+
The reflect loop's three-document threshold is the same idea at document scale,
|
|
256
|
+
and for the same reason: one reporter's judgment is taste, several unrelated
|
|
257
|
+
reporters agreeing is signal.
|
|
258
|
+
|
|
259
|
+
**An appeals path is a first-class feature, not an afterthought.** Users can
|
|
260
|
+
report content as *not* slop, which triggers re-review and restores ranking.
|
|
261
|
+
A system that can only add suspicion converges on suspecting everything. This
|
|
262
|
+
is what `learn.py --demote` exists for, and Kagi treating it as core rather
|
|
263
|
+
than optional is the argument for building it before you think you need it.
|
|
264
|
+
|
|
265
|
+
**Downrank, never remove.** Flagged results stay visible and simply rank lower.
|
|
266
|
+
The analogue here is hard rule 2 — flag hollow spans, do not fill them — and
|
|
267
|
+
the reason both hold is that a false positive under a removal policy is
|
|
268
|
+
unrecoverable, while a false positive under a flagging policy is an annoyance.
|
|
269
|
+
|
|
270
|
+
Source: <https://help.kagi.com/kagi/features/slopstop.html>
|
|
271
|
+
|
|
272
|
+
## Platform enforcement (August 2026)
|
|
273
|
+
|
|
274
|
+
The target moved from reader disapproval to ranking. LinkedIn added a "Seems
|
|
275
|
+
like AI slop" report control; reporting hides the post for that reader and
|
|
276
|
+
trains LinkedIn's classifiers, flagged posts lose algorithmic reach beyond the
|
|
277
|
+
author's own network, and repeat authors are notified privately in analytics.
|
|
278
|
+
LinkedIn retired its "enhance your post" generator for a proofreader. Snapchat
|
|
279
|
+
made wholly AI-generated video ineligible for Spotlight recommendation, ranking
|
|
280
|
+
human-made above synthetic even with a disclosure label
|
|
281
|
+
(Forbes, 1 August 2026).
|
|
282
|
+
|
|
283
|
+
Two consequences for this skill's design.
|
|
284
|
+
|
|
285
|
+
**The adversary is a reader-report classifier, not a word list.** LinkedIn's
|
|
286
|
+
system is trained on what readers flag. That is a moving target defined by
|
|
287
|
+
perception, which is an argument for the reflect loop — learning from what
|
|
288
|
+
writers actually strike tracks the same perception — and against betting the
|
|
289
|
+
detector on any fixed lexicon.
|
|
290
|
+
|
|
291
|
+
**Reach is the cost of a false negative.** Before this, shipping a sloppy draft
|
|
292
|
+
cost credibility with the people who noticed. Now it costs distribution to
|
|
293
|
+
everyone. That raises the value of catching the structural tells, which is where
|
|
294
|
+
a lexicon-only tool fails: the discrimination corpus contains a post scoring
|
|
295
|
+
38.6 with zero weighted tells, caught on rhythm and shape alone.
|
|
296
|
+
|
|
297
|
+
It also raises the cost of *over*-correction, since a rewrite that strips a
|
|
298
|
+
writer's voice to pass a meter is a worse outcome than the tell it removed.
|
|
299
|
+
That trade is why `references/overcorrection.md` exists and why the gate reports
|
|
300
|
+
what it did not measure.
|
|
301
|
+
|
|
302
|
+
## The host-model probe: token predictability without a shipped model
|
|
303
|
+
|
|
304
|
+
The strongest signal in the table (feature 1) is the one a lexical linter cannot
|
|
305
|
+
reach: whether a *model* finds the text predictable. DetectGPT and Binoculars read
|
|
306
|
+
it from token log-probabilities. Zero Slop cannot — it ships no model, and it runs
|
|
307
|
+
across harnesses where the model is Claude, whose API exposes no logprobs at all. So
|
|
308
|
+
the predictability channel computes the same thing by generation instead of
|
|
309
|
+
probability: it masks a spread of content words and asks the model *already running
|
|
310
|
+
the skill* to guess each from context alone, then scores how often the guess lands on
|
|
311
|
+
the word the author used (`scripts/predictability.py`). Machine text is easy to guess;
|
|
312
|
+
human word choice is not.
|
|
313
|
+
|
|
314
|
+
This is the DetectGPT insight — perturb, then ask the model how expected the original
|
|
315
|
+
was — reduced to a cloze the host model can answer with generation alone. It works in
|
|
316
|
+
any harness because it needs no logprobs and no bundled model; the scaffold (probe
|
|
317
|
+
selection, scoring) is deterministic and offline, and only the guessing step needs the
|
|
318
|
+
model, which the agent already is. A live check separates the extremes as expected: an
|
|
319
|
+
AI-slop paragraph scored 66.7 predictability against 41.7 for a human bug-report of the
|
|
320
|
+
same length. It is reported beside the surface score, never fused into it, so the
|
|
321
|
+
0–100 score stays traceable to spans and this stays a second, independent opinion.
|
|
322
|
+
|
|
323
|
+
The honest limit is calibration: the band cut-offs are set from small samples, not the
|
|
324
|
+
tens of thousands of labelled documents the detection papers use, so read it as a
|
|
325
|
+
corroborating signal, not a verdict — the same discipline the stylometric channels
|
|
326
|
+
below are still waiting on.
|
|
327
|
+
|
|
328
|
+
## The roadmap channels, and why they are not shipped yet
|
|
329
|
+
|
|
330
|
+
Current work on interpretable detection converges on a small set of
|
|
331
|
+
content-independent stylometric features, which is the direction this scorer
|
|
332
|
+
should grow rather than adding more regex patterns.
|
|
333
|
+
|
|
334
|
+
- **Function-word bigram frequency and average sentence length** are the two
|
|
335
|
+
highest-importance features in a lightweight interpretable detector, and the
|
|
336
|
+
paper reports 95–97% accuracy from them alone (NEULIF, arXiv:2511.21744).
|
|
337
|
+
- A systematic cross-domain analysis finds the same family — function words,
|
|
338
|
+
punctuation, linguistic diversity — carries most of the interpretable signal
|
|
339
|
+
(arXiv:2606.04177).
|
|
340
|
+
- These are attractive here because they are blind to specific wording, so
|
|
341
|
+
unlike the pattern meter they cannot be defeated by swapping synonyms, and
|
|
342
|
+
they degrade gracefully as models change.
|
|
343
|
+
|
|
344
|
+
They are **not shipped**, and the reason is a measured negative result rather
|
|
345
|
+
than a plan. Computing normalised sentence-opener entropy and function-word
|
|
346
|
+
bigram entropy over the labelled discrimination corpus separates human from
|
|
347
|
+
slop by essentially nothing (0.976 vs 0.986 opener entropy on twelve samples),
|
|
348
|
+
with slop scoring marginally *higher*. That is not evidence the features fail;
|
|
349
|
+
it is evidence the corpus is far too small to calibrate them — the cited work
|
|
350
|
+
derives its thresholds from tens of thousands of documents. Adding an
|
|
351
|
+
uncalibrated channel that shows no signal on the only data available would be
|
|
352
|
+
exactly the hand-tuned rule this project tries to avoid.
|
|
353
|
+
|
|
354
|
+
So the dependency is explicit: the labelled corpus (RAID arXiv:2405.07940, HC3,
|
|
355
|
+
M4, AuTextification) comes first, the stylometric channels are calibrated
|
|
356
|
+
against it second, and only then do they join the meter. Until then the honest
|
|
357
|
+
statement is that the scorer is a lexical-and-structural linter with a
|
|
358
|
+
research-backed roadmap, not a stylometric classifier.
|
|
359
|
+
|
|
360
|
+
## Why register repair beats detection (2025-26)
|
|
361
|
+
|
|
362
|
+
Two current results reframe what a slop tool should even try to do.
|
|
363
|
+
|
|
364
|
+
**Detection is losing an arms race.** Adversarial paraphrasing — an LLM
|
|
365
|
+
rewriting AI text under the guidance of a detector — cuts detector true-positive
|
|
366
|
+
rates by 87-99% and transfers across neural, watermark and zero-shot detectors
|
|
367
|
+
(Adversarial Paraphrasing, arXiv:2506.07001; StealthRL, arXiv:2602.08934).
|
|
368
|
+
Anything that competes on catching a machine is chasing a target the literature
|
|
369
|
+
is actively defeating. Zero Slop does not compete there: it repairs the register
|
|
370
|
+
rather than classifying authorship, which is the half that stays useful when
|
|
371
|
+
detection fails.
|
|
372
|
+
|
|
373
|
+
**Slop is a training-supply pollutant, not just an aesthetic one.** Model
|
|
374
|
+
collapse work shows machine text re-entering training corpora degrades the next
|
|
375
|
+
model, with as little as 1% synthetic data measurably harming quality
|
|
376
|
+
(arXiv:2603.11784, arXiv:2510.16657). That makes de-slopping a contribution to
|
|
377
|
+
the commons, not only to one post's reach — the cleaner the human writing that
|
|
378
|
+
survives, the less the collapse.
|
|
379
|
+
|
|
380
|
+
Together these are the argument for the product's shape: measure and rewrite the
|
|
381
|
+
author's own draft, personalise to the author's own voice, and never optimise
|
|
382
|
+
against a detector, because the detector is both defeatable and beside the point.
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Over-correction — the second failure mode
|
|
2
|
+
|
|
3
|
+
The classic humanizer failure is swapping AI-slop for a louder slop. Readers
|
|
4
|
+
clock both. Everything here is a rewrite *output* ban: never introduce these
|
|
5
|
+
into text that didn't have them.
|
|
6
|
+
|
|
7
|
+
## The edgy-slop catalogue
|
|
8
|
+
|
|
9
|
+
- **Forced contrarianism** — "Everyone says X. They're wrong." (unless the
|
|
10
|
+
source argued it)
|
|
11
|
+
- **Fake first person** — "I've seen this a hundred times", "In my
|
|
12
|
+
experience…" injected into authorless prose. Manufactured war stories are
|
|
13
|
+
fabrication, the cardinal sin.
|
|
14
|
+
- **Performed candor** — "Let's be real", "Here's the thing", "I'll be
|
|
15
|
+
honest": candor is shown, not announced.
|
|
16
|
+
- **Staccato drama** — "This matters. A lot. More than you think." Broetry
|
|
17
|
+
fragmentation is the LinkedIn variant.
|
|
18
|
+
- **Em-dash theatrics** — dashes manufacturing emphasis the content didn't
|
|
19
|
+
earn. (Yes, humanizers add these; yes, it reads as AI.)
|
|
20
|
+
- **Binary-contrast reveals** — "The answer isn't more tools. It's
|
|
21
|
+
discipline." One per piece max; injecting them is over-correction.
|
|
22
|
+
- **Manufactured stakes** — "In a world where…", "Now more than ever".
|
|
23
|
+
- **Intensifier padding as personality** — "genuinely", "honestly",
|
|
24
|
+
"literally" sprinkled for flavor.
|
|
25
|
+
- **Slang costume** — forced colloquialisms a professional author wouldn't
|
|
26
|
+
use ("chef's kiss", "hits different") unless the voice sample has them.
|
|
27
|
+
- **Manufactured informality** — forced lowercase, stray "lol", conspicuous
|
|
28
|
+
swearing, or broken grammar added to look human. Preserve these when they are
|
|
29
|
+
already part of the writer's voice; never inject them as camouflage.
|
|
30
|
+
- **Fake errors** — never inject typos or grammar mistakes to fool
|
|
31
|
+
detectors. That's adversarial evasion, not writing, and it degrades the
|
|
32
|
+
text.
|
|
33
|
+
- **Performed-writer prose** — theatrical framing of ordinary work ("we
|
|
34
|
+
hired an adversary"), epigram closers, staccato antithesis ("Not perfect.
|
|
35
|
+
Honest."), extended conceits (billing, courtroom, forensics, recipe),
|
|
36
|
+
hyperbole ("nothing on earth"), slang-cute idioms ("has receipts"), and
|
|
37
|
+
cute meta-taglines. The detection-side rows live in `tells.md` §3;
|
|
38
|
+
injecting them is the same costume-swap.
|
|
39
|
+
|
|
40
|
+
The bar is a *thinking* author, not a *loud* one.
|
|
41
|
+
|
|
42
|
+
## What NOT to flag (false-positive guard)
|
|
43
|
+
|
|
44
|
+
From Wikipedia's "ineffective indicators" plus detector-calibration
|
|
45
|
+
experience — these alone are NOT evidence of AI:
|
|
46
|
+
|
|
47
|
+
- Perfect grammar and spelling
|
|
48
|
+
- Formal or technical register where the genre demands it
|
|
49
|
+
- A transition word, an em-dash, a "however" in isolation
|
|
50
|
+
- Long sentences that earn their length
|
|
51
|
+
- Rule-of-three used once, deliberately, for rhythm
|
|
52
|
+
- Domain jargon used correctly for a domain audience
|
|
53
|
+
- Calibrated hedging in research/medical/legal writing
|
|
54
|
+
- Text merely being unsourced (check it, don't flag it)
|
|
55
|
+
|
|
56
|
+
Require corroboration. A paragraph needs multiple independent tells, or a failed
|
|
57
|
+
removal test, before it's slop.
|
|
58
|
+
|
|
59
|
+
## Signs of human writing — preserve on sight
|
|
60
|
+
|
|
61
|
+
When a draft shows these, protect them through the rewrite; deleting them is
|
|
62
|
+
damage:
|
|
63
|
+
|
|
64
|
+
- A claim someone could disagree with, stated without cover
|
|
65
|
+
- The specific odd fact ($1.1M, 4,000 users, "episode 142")
|
|
66
|
+
- Selective hedging at the edge of the author's knowledge
|
|
67
|
+
- Humor, irritation, dry asides, self-interruption
|
|
68
|
+
- Digressions that carry personality; asymmetric structure
|
|
69
|
+
- Insider references assumed, not explained
|
|
70
|
+
- The author's pet phrases and punctuation habits (voice sample rules)
|
|
71
|
+
- Mistakes of passion — a run-on in an excited passage. Leave it.
|
|
72
|
+
|
|
73
|
+
## Idempotence check
|
|
74
|
+
|
|
75
|
+
Run the finished rewrite through the scorer and this file once more. If your
|
|
76
|
+
rewrite added any catalogue item above, you traded costumes. Prefer the
|
|
77
|
+
smaller edit: the best de-slop is usually deletion of the hedge plus nothing.
|