buvard 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +41 -0
- package/LICENSE +674 -0
- package/README.md +95 -2
- package/SKILL.md +34 -0
- package/bin/buvard.js +68 -0
- package/docs/anatomie-du-slop.svg +76 -0
- package/docs/research-litreview.md +282 -0
- package/docs/research.md +68 -0
- package/lib/catalog.mjs +72 -0
- package/lib/index.mjs +4 -0
- package/lib/lint.mjs +86 -0
- package/package.json +37 -4
package/README.md
CHANGED
|
@@ -1,3 +1,96 @@
|
|
|
1
|
-
#
|
|
1
|
+
# buvard
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**L'anti-slop qui tache.** Blotting paper for AI-flavored prose: a weighted,
|
|
4
|
+
multi-score linter for French and English slop patterns. It absorbs what
|
|
5
|
+
baves — the empty openers, the inflated adjectives, the "Ce n'est pas X,
|
|
6
|
+
c'est Y" pivots, the tirets cadratins by the dozen.
|
|
7
|
+
|
|
8
|
+
One catalog, clustered from five anti-slop sources (two French skills, two
|
|
9
|
+
bilingual frameworks, one EN framework), scored by a ratified barème, and
|
|
10
|
+
tested. Zero config, by design — like black, the formatting is the tool's;
|
|
11
|
+
the words stay the author's.
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
git clone https://git.nech.pl/pln/buvard.git && cd buvard
|
|
17
|
+
sudo ln -sf "$PWD/bin/buvard.js" /usr/local/bin/buvard
|
|
18
|
+
|
|
19
|
+
buvard check posts/ # scores + grouped hits, per file
|
|
20
|
+
buvard check article.md --json # machine-readable, for pipelines
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## The barème
|
|
24
|
+
|
|
25
|
+
Score starts at 100. A lexique hit costs 5, a structure hit 8, servility or
|
|
26
|
+
meta-commentary 10. Punctuation runs on budgets: one tiret cadratin per 500
|
|
27
|
+
words, one `!` per 1000 — each excess occurrence costs 5.
|
|
28
|
+
|
|
29
|
+
- **100–90** — publish.
|
|
30
|
+
- **80–95** — real hits only; fix what reads wrong.
|
|
31
|
+
- **<80** — rewrite pass, then re-run. The score is a smoke alarm, not a grade.
|
|
32
|
+
|
|
33
|
+
Every hit is a review candidate, not a verdict. Never touch: the author's
|
|
34
|
+
ideas, data, deliberate jargon, quoted speech, or the voice.
|
|
35
|
+
|
|
36
|
+
## Scope and honesty
|
|
37
|
+
|
|
38
|
+
buvard is a **style linter**, not an AI detector. It finds statistically
|
|
39
|
+
over-represented patterns; it cannot and does not claim to detect machine
|
|
40
|
+
authorship. Its catalog is deliberately regex-shaped (high precision, low
|
|
41
|
+
recall): règle de trois, synonyme acrobatique and sentence-length monotony
|
|
42
|
+
stay manual checks. See [Science and references](#science-and-references)
|
|
43
|
+
for the evidence behind the catalog.
|
|
44
|
+
|
|
45
|
+
## Agents
|
|
46
|
+
|
|
47
|
+
Agents run buvard three ways, in order of friction:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
npx buvard check . # after npm publish — no install at all
|
|
51
|
+
node bin/buvard.js check . # from a clone — zero dependencies, Node >= 18
|
|
52
|
+
buvard check . # global symlink (see install above)
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
`AGENTS.md` carries the project handoff for any agent; `SKILL.md` packages the
|
|
56
|
+
catalog as an agent skill: load it, run the linter, apply the findings by hand.
|
|
57
|
+
Hits are review candidates, never verdicts — the agent edits prose like an
|
|
58
|
+
editor, not like a filter.
|
|
59
|
+
|
|
60
|
+
## Science and references
|
|
61
|
+
|
|
62
|
+
The catalog is data with provenance, and the barème is a ratified decision —
|
|
63
|
+
both are written down:
|
|
64
|
+
|
|
65
|
+
- **[docs/research.md](docs/research.md)** — the clustering study: five
|
|
66
|
+
anti-slop sources (fr1, fr2, étape, no-slop, ui-sk), what each contributes,
|
|
67
|
+
and how the shared patterns became the catalog.
|
|
68
|
+
- **[docs/research-litreview.md](docs/research-litreview.md)** — the
|
|
69
|
+
literature review on AI-text detection, and the **ten falsifiable
|
|
70
|
+
recommendations** that will become catalog v3. Read it before touching
|
|
71
|
+
the catalog.
|
|
72
|
+
|
|
73
|
+
Core citations, one line each — the lit review holds the full bibliography:
|
|
74
|
+
|
|
75
|
+
- Liang et al., 2023 — [GPT detectors are biased against non-native English writers](https://arxiv.org/abs/2304.02819): a 61.22% false-positive rate on TOEFL essays. The case for staying a linter.
|
|
76
|
+
- [MULTITuDE](https://arxiv.org/abs/2310.13606) — the large multilingual detection benchmark omits French; [INRIA's French ChatGPT-detection study](https://arxiv.org/abs/2306.05871) shows French is open territory.
|
|
77
|
+
- [GLTR](https://arxiv.org/abs/1906.04043), [DetectGPT](https://arxiv.org/abs/2301.11305), [Binoculars](https://arxiv.org/abs/2401.12070) — the detection toolbox; each degrades on short or paraphrased text.
|
|
78
|
+
- Sadasivan et al., 2024 — [paraphrase attacks collapse detection](https://arxiv.org/abs/2303.11156) on ~300-token passages. Why buvard never plays that game.
|
|
79
|
+
- Kobak et al., 2024 — [excess vocabulary](https://arxiv.org/abs/2403.07183) as a population-level AI marker. A cousin of our lexique.
|
|
80
|
+
- Sibling linters — [proselint](https://github.com/amperser/proselint) and [Vale](https://docs.vale.sh/) are the ancestors; [Wikipedia's "Signs of AI writing"](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing) is the shared folklore.
|
|
81
|
+
|
|
82
|
+
**Methodology, going forward (catalog v3):** every new rule must pass a
|
|
83
|
+
false-positive gate — under 2% per rule on a pre-2022 Gutenberg corpus,
|
|
84
|
+
human-written by construction. Model-generated text is used only to measure
|
|
85
|
+
sensitivity, never to admit a rule; a catalog tuned on one model's slop
|
|
86
|
+
learns that model's fingerprints.
|
|
87
|
+
|
|
88
|
+
## Skill
|
|
89
|
+
|
|
90
|
+
`SKILL.md` ships the catalog as an agent skill: load it, run the linter,
|
|
91
|
+
apply the findings by hand. The remedies are content, not code.
|
|
92
|
+
|
|
93
|
+
## License
|
|
94
|
+
|
|
95
|
+
GPL-3.0-only — see [LICENSE](LICENSE). The name: a buvard is blotting paper;
|
|
96
|
+
it exists because l'encre bave.
|
package/SKILL.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: buvard
|
|
3
|
+
description: Use when writing, editing or reviewing French or English prose that must not read as AI-generated — blog posts, docs, newsletters, READMEs — or before publishing any text an agent drafted. Runs the buvard linter and applies its catalog by hand.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# buvard — l'anti-slop qui tache
|
|
7
|
+
|
|
8
|
+
Blotting paper for AI-flavored prose. A weighted catalog of slop patterns
|
|
9
|
+
clustered from five anti-slop sources, plus punctuation budgets
|
|
10
|
+
(1 tiret cadratin per 500 words, 1 `!` per 1000).
|
|
11
|
+
|
|
12
|
+
## Run it
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
buvard check <files|dirs> # scores + grouped hits
|
|
16
|
+
buvard check . --json # machine-readable
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Apply the findings
|
|
20
|
+
|
|
21
|
+
1. Every hit is a **review candidate**, not a verdict. Read the sentence.
|
|
22
|
+
2. The remedy is always **content, not code**: cut the opener, name the
|
|
23
|
+
actor, say the second half directly, replace the adjective with the fact
|
|
24
|
+
that justifies it. Repetition of the right word beats an elegant synonym.
|
|
25
|
+
3. Never touch: the author's ideas, data, deliberate technical jargon,
|
|
26
|
+
quoted speech, or the voice. Two `!` in a joyful 137-word post is not slop.
|
|
27
|
+
4. Known regex gaps, check by eye: règle de trois, synonyme acrobatique,
|
|
28
|
+
sentence-length monotony, parataxis chains.
|
|
29
|
+
|
|
30
|
+
## Score interpretation
|
|
31
|
+
|
|
32
|
+
- 100–90: publish.
|
|
33
|
+
- 80–95: real hits only; fix what reads wrong.
|
|
34
|
+
- <80: rewrite pass, then re-run. Score is a smoke alarm, not a grade.
|
package/bin/buvard.js
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* buvard — l'anti-slop qui tache. Blotting paper for AI-flavored prose.
|
|
4
|
+
*
|
|
5
|
+
* buvard check <files|dirs> [--json] run the catalog over markdown/text
|
|
6
|
+
* buvard score <file> single-file score, machine-readable
|
|
7
|
+
*
|
|
8
|
+
* Scoring starts at 100; each hit costs its weight (lexique 5, structure 8,
|
|
9
|
+
* servile 10; punctuation charges per occurrence past the budget: one tiret
|
|
10
|
+
* cadratin per 500 words, one exclamation per 1000). Every hit is a review
|
|
11
|
+
* candidate, not a verdict — the author's voice always wins.
|
|
12
|
+
*
|
|
13
|
+
* Known gaps, deliberately manual: règle de trois, synonyme acrobatique and
|
|
14
|
+
* sentence-length variance need document-level analysis that regex fakes
|
|
15
|
+
* badly. Read the text too.
|
|
16
|
+
*/
|
|
17
|
+
|
|
18
|
+
import fs from 'fs';
|
|
19
|
+
import path from 'path';
|
|
20
|
+
import { CATALOG_VERSION, lint, bodyOf } from '../lib/index.mjs';
|
|
21
|
+
|
|
22
|
+
const TEXT_EXT = /\.(md|txt|mdx)$/i;
|
|
23
|
+
|
|
24
|
+
function collect(target, files) {
|
|
25
|
+
if (TEXT_EXT.test(target) && fs.existsSync(target)) return files.push(path.resolve(target));
|
|
26
|
+
if (fs.existsSync(target) && fs.statSync(target).isDirectory()) {
|
|
27
|
+
for (const f of fs.readdirSync(target)) {
|
|
28
|
+
const p = path.join(target, f);
|
|
29
|
+
if (fs.statSync(p).isDirectory()) collect(p, files);
|
|
30
|
+
else if (TEXT_EXT.test(f)) files.push(path.resolve(p));
|
|
31
|
+
}
|
|
32
|
+
return;
|
|
33
|
+
}
|
|
34
|
+
console.error(`buvard: no file or directory for "${target}"`);
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
function main(argv) {
|
|
38
|
+
const json = argv.includes('--json');
|
|
39
|
+
const targets = argv.filter((a) => !a.startsWith('--'));
|
|
40
|
+
const files = [];
|
|
41
|
+
if (targets.length > 0) targets.forEach((t) => collect(t, files));
|
|
42
|
+
else collect('.', files);
|
|
43
|
+
|
|
44
|
+
const results = files.map((file) => ({ file, ...lint(bodyOf(fs.readFileSync(file, 'utf8'))) }));
|
|
45
|
+
|
|
46
|
+
if (json) {
|
|
47
|
+
console.log(JSON.stringify({ buvard: CATALOG_VERSION, results }, null, 1));
|
|
48
|
+
return 0;
|
|
49
|
+
}
|
|
50
|
+
for (const r of results) {
|
|
51
|
+
const grouped = {};
|
|
52
|
+
for (const h of r.hits) (grouped[h.id] ??= []).push(h);
|
|
53
|
+
console.log(`\n== ${path.relative(process.cwd(), r.file)} — ${r.score}/100 (${r.words} mots)`);
|
|
54
|
+
for (const [id, list] of Object.entries(grouped)) {
|
|
55
|
+
console.log(` [${id}] ×${list.length} (-${list[0].w} chacun)`);
|
|
56
|
+
for (const h of list.slice(0, 4)) console.log(` ${h.at}`);
|
|
57
|
+
}
|
|
58
|
+
}
|
|
59
|
+
console.log(`\n---- buvard v${CATALOG_VERSION} ----`);
|
|
60
|
+
for (const r of results) console.log(`${String(r.score).padStart(3)}/100 ${path.relative(process.cwd(), r.file)}`);
|
|
61
|
+
const worst = results.filter((r) => r.hits.length > 0).sort((a, b) => a.score - b.score)[0];
|
|
62
|
+
if (worst) console.log(`priorité: ${path.relative(process.cwd(), worst.file)} (${worst.score}/100)`);
|
|
63
|
+
return 0;
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
if (process.argv[1] && path.resolve(process.argv[1]) === path.resolve(import.meta.url.replace('file://', ''))) {
|
|
67
|
+
process.exit(main(process.argv.slice(2)));
|
|
68
|
+
}
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 880 470" role="img" aria-labelledby="t d">
|
|
2
|
+
<title id="t">Anatomie d'une bavure</title>
|
|
3
|
+
<desc id="d">One slop sentence dissected: buvard's catalog flags a binary contrast (−8), a servile promise (−10) and a second tiret cadratin beyond budget (−5); the score falls from 100 to 77.</desc>
|
|
4
|
+
<defs>
|
|
5
|
+
<style>
|
|
6
|
+
.ink { fill:#1f2a33; }
|
|
7
|
+
.rouge{ fill:#a63d2f; }
|
|
8
|
+
.muted{ fill:#7a6f5d; }
|
|
9
|
+
.serif{ font-family: Georgia, 'Times New Roman', serif; }
|
|
10
|
+
.sans { font-family: 'Avenir Next', 'Helvetica Neue', Arial, sans-serif; }
|
|
11
|
+
.line { stroke:#1f2a33; fill:none; stroke-linecap:round; }
|
|
12
|
+
.rouged { stroke:#a63d2f; fill:none; stroke-linecap:round; }
|
|
13
|
+
</style>
|
|
14
|
+
</defs>
|
|
15
|
+
|
|
16
|
+
<!-- paper -->
|
|
17
|
+
<rect width="880" height="470" fill="#faf5ea"/>
|
|
18
|
+
|
|
19
|
+
<!-- title -->
|
|
20
|
+
<text x="90" y="62" class="sans ink" font-size="15" letter-spacing="6">ANATOMIE D'UNE BAVURE</text>
|
|
21
|
+
<path d="M90 74 q 60 4 120 1 t 122 -1" class="line" stroke-width="1.4"/>
|
|
22
|
+
<text x="790" y="62" class="sans muted" font-size="11" text-anchor="end" letter-spacing="2">buvard</text>
|
|
23
|
+
|
|
24
|
+
<!-- the sentence, segment by segment so the boxes align exactly -->
|
|
25
|
+
<g class="serif ink" font-size="21">
|
|
26
|
+
<text x="90" y="140" textLength="24" lengthAdjust="spacingAndGlyphs">« </text>
|
|
27
|
+
<text x="114" y="140" textLength="222" lengthAdjust="spacingAndGlyphs">Ce n'est pas un outil,</text>
|
|
28
|
+
<text x="336" y="140" textLength="205" lengthAdjust="spacingAndGlyphs">c'est un compagnon</text>
|
|
29
|
+
<text x="90" y="182" textLength="100" lengthAdjust="spacingAndGlyphs">de voyage</text>
|
|
30
|
+
<text x="196" y="182" textLength="26" lengthAdjust="spacingAndGlyphs"> — </text>
|
|
31
|
+
<text x="228" y="182" textLength="92" lengthAdjust="spacingAndGlyphs">et il va</text>
|
|
32
|
+
<text x="326" y="182" textLength="200" lengthAdjust="spacingAndGlyphs">vous changer la vie</text>
|
|
33
|
+
<text x="532" y="182" textLength="52" lengthAdjust="spacingAndGlyphs"> ! »</text>
|
|
34
|
+
</g>
|
|
35
|
+
|
|
36
|
+
<!-- red-pencil boxes around the three hits -->
|
|
37
|
+
<rect x="108" y="118" width="234" height="33" rx="7" class="rouged" stroke-width="1.6" stroke-dasharray="1 0"/>
|
|
38
|
+
<rect x="191" y="160" width="36" height="33" rx="7" class="rouged" stroke-width="1.6"/>
|
|
39
|
+
<rect x="320" y="160" width="212" height="33" rx="7" class="rouged" stroke-width="1.6"/>
|
|
40
|
+
|
|
41
|
+
<!-- leader lines -->
|
|
42
|
+
<path d="M225 151 Q 200 205 150 236" class="line" stroke-width="1" stroke-dasharray="3 4"/>
|
|
43
|
+
<circle cx="150" cy="236" r="2.2" class="rouge" stroke="none"/>
|
|
44
|
+
<path d="M209 193 Q 250 240 310 284" class="line" stroke-width="1" stroke-dasharray="3 4"/>
|
|
45
|
+
<circle cx="310" cy="284" r="2.2" class="rouge" stroke="none"/>
|
|
46
|
+
<path d="M426 193 Q 480 240 533 296" class="line" stroke-width="1" stroke-dasharray="3 4"/>
|
|
47
|
+
<circle cx="533" cy="296" r="2.2" class="rouge" stroke="none"/>
|
|
48
|
+
|
|
49
|
+
<!-- the three verdicts -->
|
|
50
|
+
<g class="serif">
|
|
51
|
+
<text x="90" y="266" class="rouge" font-size="17" font-weight="bold">contraste binaire <tspan class="rouge">−8</tspan></text>
|
|
52
|
+
<text x="90" y="286" class="muted" font-size="12.5" font-style="italic">« pas X, c'est Y » — la bascule creuse</text>
|
|
53
|
+
|
|
54
|
+
<text x="320" y="300" class="rouge" font-size="17" font-weight="bold">tiret cadratin <tspan class="rouge">−5</tspan></text>
|
|
55
|
+
<text x="320" y="320" class="muted" font-size="12.5" font-style="italic">budget : un seul pour 500 mots</text>
|
|
56
|
+
|
|
57
|
+
<text x="545" y="312" class="rouge" font-size="17" font-weight="bold">servile <tspan class="rouge">−10</tspan></text>
|
|
58
|
+
<text x="545" y="332" class="muted" font-size="12.5" font-style="italic">la promesse d'émerveillement</text>
|
|
59
|
+
</g>
|
|
60
|
+
|
|
61
|
+
<!-- the gauge: 0–100, threshold at 80, ink fill to 77 -->
|
|
62
|
+
<g>
|
|
63
|
+
<text x="90" y="368" class="sans ink" font-size="12" letter-spacing="3">SCORE</text>
|
|
64
|
+
<line x1="90" y1="400" x2="790" y2="400" class="line" stroke-width="1.2"/>
|
|
65
|
+
<line x1="90" y1="400" x2="646" y2="400" class="line" stroke-width="7"/>
|
|
66
|
+
<!-- threshold tick at 80 : 90 + 0.80*700 = 650 -->
|
|
67
|
+
<line x1="650" y1="390" x2="650" y2="410" class="rouged" stroke-width="1.6"/>
|
|
68
|
+
<text x="650" y="426" class="rouge sans" font-size="10.5" text-anchor="middle" letter-spacing="1">SEUIL 80</text>
|
|
69
|
+
<text x="790" y="426" class="muted sans" font-size="10.5" text-anchor="end">100</text>
|
|
70
|
+
<text x="90" y="426" class="muted sans" font-size="10.5">0</text>
|
|
71
|
+
<text x="646" y="384" class="rouge serif" font-size="26" font-weight="bold" text-anchor="end">77</text>
|
|
72
|
+
<text x="676" y="368" class="muted serif" font-size="13" font-style="italic">réécrire, puis re-passer</text>
|
|
73
|
+
</g>
|
|
74
|
+
|
|
75
|
+
<text x="440" y="458" class="sans muted" font-size="10" text-anchor="middle" letter-spacing="2">HITS ARE REVIEW CANDIDATES, NEVER VERDICTS — THE AUTHOR'S VOICE ALWAYS WINS</text>
|
|
76
|
+
</svg>
|
|
@@ -0,0 +1,282 @@
|
|
|
1
|
+
# buvard: literature review
|
|
2
|
+
|
|
3
|
+
Literature review supporting buvard, a regex + heuristic prose linter for AI-slop
|
|
4
|
+
patterns in French and English. Written 2026-11-07. Every non-obvious claim carries a
|
|
5
|
+
URL; anything we could not verify is marked [inferred] or [unverified]. This document is
|
|
6
|
+
itself written under buvard's own rules: no em-dash clause separators, no contrast
|
|
7
|
+
constructions, no rule-of-three defaults.
|
|
8
|
+
|
|
9
|
+
## 1. Statistical markers: what stylometry actually measures
|
|
10
|
+
|
|
11
|
+
Formal detection research works at the token level, with probability scores, while
|
|
12
|
+
buvard works at the surface level, with regex and counts. The two literatures agree on
|
|
13
|
+
one thing that matters for buvard: short texts defeat everything.
|
|
14
|
+
|
|
15
|
+
The canonical systems and their numbers:
|
|
16
|
+
|
|
17
|
+
| System / metric | What it measures | Reported result | Source |
|
|
18
|
+
|---|---|---|---|
|
|
19
|
+
| GLTR (token rank) | position of each token in the model's ranked vocabulary; human text draws from ranks 10-1000 far more than AI text | raised human judges' fake-text accuracy from 54% to 72% on GPT-2 output, with no training | https://arxiv.org/abs/1906.04043 |
|
|
20
|
+
| DetectGPT (curvature) | negative curvature of the log-probability surface under random perturbations | AUROC on fake-news detection rose from 0.81 to 0.95; requires logprobs from the source model itself | https://arxiv.org/abs/2301.11305 |
|
|
21
|
+
| Binoculars (cross-perplexity) | ratio of perplexities of two closely related models on the same text | over 90% recall on ChatGPT samples at 0.01% false positive rate | https://arxiv.org/abs/2401.12070 |
|
|
22
|
+
| Recursive paraphrase attack | paraphrase round-trips push text back toward the human perplexity band | detection rates collapse on ~300-token passages while text quality barely drops | https://arxiv.org/abs/2303.11156 |
|
|
23
|
+
|
|
24
|
+
Three limits recur across these papers and they define buvard's lane:
|
|
25
|
+
|
|
26
|
+
1. **Token-level scores need the model's logits.** DetectGPT needs the source model's
|
|
27
|
+
own logprobs, Binoculars needs two local models, GLTR needs the vocabulary ranking.
|
|
28
|
+
A regex linter cannot have any of these. That is a feature here: the catalog catches
|
|
29
|
+
surface evidence, and it says so.
|
|
30
|
+
2. **Short texts are unreliable for everyone.** Sadasivan et al. test on ~300-token
|
|
31
|
+
passages and show paraphrasing alone destroys detection. Any per-paragraph verdict
|
|
32
|
+
from a stylometric score on fewer than ~150 words is noise. buvard's weighted scores
|
|
33
|
+
should aggregate per document, per 500-word window, and report counts, never a
|
|
34
|
+
probability of AI authorship.
|
|
35
|
+
3. **Detectors discriminate against human writers as much as machines.** Seven commercial GPT
|
|
36
|
+
detectors misclassified on average 61.22% of 91 TOEFL essays by non-native writers
|
|
37
|
+
as AI-generated; 97.80% were flagged by at least one detector. Prompting to sound
|
|
38
|
+
"more like a native speaker" cut the false positive rate from 61.22% to 11.77%,
|
|
39
|
+
which pins the cause on word choice: common-word-heavy, low-perplexity prose reads
|
|
40
|
+
as machine. Source: https://arxiv.org/abs/2304.02819 (published in Patterns:
|
|
41
|
+
https://www.cell.com/patterns/fulltext/S2666-3899(23)00130-7). This is the single
|
|
42
|
+
strongest argument for buvard to stay a style linter and never an authorship judge.
|
|
43
|
+
|
|
44
|
+
On the specific surface metrics writers talk about:
|
|
45
|
+
|
|
46
|
+
- **Perplexity and burstiness.** GPTZero popularized "perplexity" (predictability) and
|
|
47
|
+
"burstiness" (sentence-to-sentence variance) as its two scores; the company's own
|
|
48
|
+
learning pages define burstiness as variation in sentence structure and length
|
|
49
|
+
[inferred URL: https://gptzero.me/learn/, we could not fetch it cleanly]. The
|
|
50
|
+
direction is consistent everywhere: AI text has flatter sentence-length variance.
|
|
51
|
+
Note the fairness trap: lower variance is also a hallmark of trained technical
|
|
52
|
+
writing, so buvard should flag flat rhythm as a structural note, at low weight.
|
|
53
|
+
- **Function-word rates and lexical diversity.** Liang et al.'s corpus method
|
|
54
|
+
(frequency of common tokens against expert-written reference texts) found a
|
|
55
|
+
"decrease in variation of linguistic features" in LLM-modified text and estimated
|
|
56
|
+
6.5-16.9% of AI-conference peer-review text as substantially LLM-modified.
|
|
57
|
+
Source: https://arxiv.org/abs/2403.07183. Kobak et al.'s excess-vocabulary analysis
|
|
58
|
+
of 15M PubMed abstracts (2010-2024) showed an abrupt step increase in specific style
|
|
59
|
+
words the month ChatGPT shipped, and a floor of at least 13.5% of 2024 abstracts
|
|
60
|
+
LLM-processed, up to 40% in some subcorpora. Source:
|
|
61
|
+
https://arxiv.org/abs/2406.07016 (published version in Science Advances, 2025).
|
|
62
|
+
For buvard: relative word frequency shifts are the most robust, hardest-to-avoid
|
|
63
|
+
signal, and they are exactly what a frequency-banded lexicon encodes.
|
|
64
|
+
- **Repetition metrics.** Low type-token ratio and repeated sentence openers are
|
|
65
|
+
standard stylometric evidence; the excess-vocabulary paper above is the strongest
|
|
66
|
+
published version of the repetition claim (same words, everywhere, all at once).
|
|
67
|
+
Sentence-opener repetition is also the top complaint in every practitioner catalog
|
|
68
|
+
reviewed in section 4.
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
## 2. The contrast-construction family and its cousins
|
|
72
|
+
|
|
73
|
+
**"Not X, but Y" / "Ce n'est pas X, c'est Y".** Wikipedia's "Signs of AI writing"
|
|
74
|
+
maintainer page lists the construction as a top indicator, in the forms "not just X,
|
|
75
|
+
but also Y" and "serves as / stands as / marks / functions as / operates as /
|
|
76
|
+
represents [a]": https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing. The
|
|
77
|
+
page's own caveat matters and buvard should quote it in its docs: these are "potential
|
|
78
|
+
signs", since human authors use the same styles constantly, especially in blogs and
|
|
79
|
+
list-style articles. hardikpandya's stop-slop skill independently catalogs "binary
|
|
80
|
+
contrasts" as an AI tell: https://github.com/hardikpandya/stop-slop. On the French
|
|
81
|
+
side, both anti-slop-fr skills flag structural contrast and false-agentivity frames
|
|
82
|
+
("les chiffres parlent d'eux-mêmes", "plongeons dans"): see section 3.
|
|
83
|
+
|
|
84
|
+
The craft consensus, reconstructed from these catalogs: the antithesis is classical
|
|
85
|
+
rhetoric and legitimate in moderation; the tell is frequency and sameness. LLMs reach
|
|
86
|
+
for the construction as a default compression move, so one instance per document is
|
|
87
|
+
style and five per 500 words is slop. For buvard this argues a density threshold with
|
|
88
|
+
a low per-instance penalty, not a per-hit kill.
|
|
89
|
+
|
|
90
|
+
**The rising "X, so Y" pattern.** This is the "It's a great tool, so..." hedge-plus-
|
|
91
|
+
pivot: a bland claim followed by "so" + grandiose consequence ("so you can focus on
|
|
92
|
+
what matters"). We found no published linguistic study of it [unverified]; it appears
|
|
93
|
+
in practitioner catalogs, including the 20-pattern creatoreconomy skill buvard already
|
|
94
|
+
ingests. Treat the sourcing as folklore for now: flag at low weight until a second
|
|
95
|
+
independent source appears.
|
|
96
|
+
|
|
97
|
+
**Em dash.** Wikipedia's signs page documents "Overuse of em dashes" with an explicit
|
|
98
|
+
date range, 2022 to September 2026, tying the spike to LLM output
|
|
99
|
+
(https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing). The 2025 press wave on
|
|
100
|
+
this is real but unreachable from this environment, so specific articles (Merriam-
|
|
101
|
+
Webster wordplay, Guardian comment pieces, New York Magazine's em-dash coverage) and
|
|
102
|
+
the reported Microsoft Word autocorrect incident that auto-inserted em dashes and was
|
|
103
|
+
reverted after user backlash are marked [unverified] here. buvard's own budget
|
|
104
|
+
approach (a cap per 500 words instead of a ban) matches the actual debate: writers
|
|
105
|
+
with strong opinions on em dashes are the loudest false-positive victims, and
|
|
106
|
+
hand-written English prose legitimately uses them. French differs: French typography
|
|
107
|
+
uses the tiret for dialogue and parenthetical incises far less in expository prose, so
|
|
108
|
+
the FR budget should be tighter.
|
|
109
|
+
|
|
110
|
+
**Rule of three.** The tricolon is a classical figure, and Wikipedia's signs page
|
|
111
|
+
lists "Rule of three" as an AI indicator without claiming any single tricolon is bad.
|
|
112
|
+
The practitioner consensus again is density: a consistent pattern of A, B, and C
|
|
113
|
+
triplets in consecutive paragraphs is the tell. buvard should detect the repetition
|
|
114
|
+
across a window (three or more tricolons in 500 words) rather than flag each one, and
|
|
115
|
+
exempt dialogue, lists, and comedic writing where the triple is the joke.
|
|
116
|
+
|
|
117
|
+
**Design rule extracted from this whole section.** For every rhetorical device in the
|
|
118
|
+
family (contrast, tricolon, dash, so-pivot), the literature and the catalogs converge
|
|
119
|
+
on the same shape: legitimate once, slop when uniform. Encode per-instance penalties
|
|
120
|
+
near zero and window-density penalties that bite.
|
|
121
|
+
|
|
122
|
+
|
|
123
|
+
## 3. French-specific AI tells: thinner coverage, different typography
|
|
124
|
+
|
|
125
|
+
**The practitioner layer is where FR is strongest.** The two anti-slop-fr skills are
|
|
126
|
+
the main public French catalogs:
|
|
127
|
+
|
|
128
|
+
- JulienSnsnCode/anti-slop-fr (https://github.com/JulienSnsnCode/anti-slop-fr):
|
|
129
|
+
organizes the problem into 13 lexicon categories and 13 structural patterns, flags
|
|
130
|
+
over-represented words ("mots surreprésentés"), ambiguous turns, predictable
|
|
131
|
+
structures such as "dans un monde en constante évolution", English calques
|
|
132
|
+
(« plongeons dans »), and false agentivity (« les chiffres parlent d'eux-mêmes »).
|
|
133
|
+
It grades text 1 to 10 on Franchise, Rythme, Confiance, Naturel, Densité and
|
|
134
|
+
rewrites under 35/50, which is an LLM-judge pipeline, unlike buvard's deterministic
|
|
135
|
+
scoring.
|
|
136
|
+
- captain-marketing/anti-slop-fr (https://github.com/captain-marketing/anti-slop-fr):
|
|
137
|
+
two main families, "Formules creuses" (empty hyperboles such as « révolutionnaire »)
|
|
138
|
+
and "Patterns structurels" (monotone rhythm, vague declaratives). It ships
|
|
139
|
+
identity/ voice profiles and examples/, i.e. it is a rewriting skill, again not a
|
|
140
|
+
linter.
|
|
141
|
+
|
|
142
|
+
The letape-dapres.fr writing-unslop source could not be fetched from this environment;
|
|
143
|
+
its contribution to buvard's catalog remains [unverified] pending a direct read.
|
|
144
|
+
|
|
145
|
+
**The research layer barely covers French.** MULTITuDE, the largest multilingual
|
|
146
|
+
machine-text benchmark, spans 74,081 texts in 11 languages and French is absent (ar,
|
|
147
|
+
ca, cs, de, en, es, nl, pt, ru, uk, zh): https://arxiv.org/abs/2310.13606. The main
|
|
148
|
+
French ChatGPT-detection study (INRIA, https://arxiv.org/abs/2306.05871, artifacts at
|
|
149
|
+
https://gitlab.inria.fr/wantoun/robust-chatgpt-detection) built its French data by
|
|
150
|
+
translating an English dataset, and its own conclusion is caution: in-domain accuracy
|
|
151
|
+
held, out-of-domain and adversarial text exposed the detectors. So the only formal FR
|
|
152
|
+
work is (a) translated, and (b) out-of-domain fragile. No published French excess-
|
|
153
|
+
vocabulary analysis equivalent to Kobak et al. exists as far as we could find
|
|
154
|
+
[unverified]. Conclusion: buvard's French catalog is not porting a mature literature,
|
|
155
|
+
it is the literature, which is the project's real opportunity.
|
|
156
|
+
|
|
157
|
+
**French slop vocabulary worth weighting (from the FR skills, cross-checked against
|
|
158
|
+
EN patterns by analogy [inferred]):** « plonger dans », « à l'ère de », « en constante
|
|
159
|
+
évolution », « il convient de noter », « il est important de noter », « crucial »,
|
|
160
|
+
« essentiel », « révolutionnaire », « palpitant », « intrinsèquement », « en
|
|
161
|
+
définitive », « à la fin de la journée », « les chiffres parlent d'eux-mêmes », «
|
|
162
|
+
permet de », « grâce à » used as a hedge, and the English calques « débloquer » (au
|
|
163
|
+
sens figuré), « donner un aperçu de ». These deserve frequency-band treatment like the
|
|
164
|
+
EN lexicon rather than flat bans.
|
|
165
|
+
|
|
166
|
+
**Typography is where FR detection differs structurally.** French orthography requires
|
|
167
|
+
a non-breaking (or narrow no-break) space before double punctuation (; : ! ?) and
|
|
168
|
+
inside guillemets « » : https://fr.wikipedia.org/wiki/Espace_ins%C3%A9cable (see also
|
|
169
|
+
the Lexique des règles typographiques de l'Imprimerie nationale). LLM output is
|
|
170
|
+
notoriously inconsistent here: thin or plain spaces slip in, and many LLM pipelines
|
|
171
|
+
emit straight quotes instead of « ». buvard can add mechanical FR-only rules with
|
|
172
|
+
almost zero false-positive risk: guillemet spacing, insécable before ; : ! ?, and
|
|
173
|
+
straight-quote detection. These are the FR equivalent of the em dash, except they are
|
|
174
|
+
genuinely wrong under French typographic convention, so they can carry a hard penalty
|
|
175
|
+
instead of a stylistic note.
|
|
176
|
+
|
|
177
|
+
**Coverage gap, quantified as far as the evidence allows:** English has the GLTR /
|
|
178
|
+
DetectGPT / Binoculars literature, a commercial detector market, and dozens of
|
|
179
|
+
practitioner catalogs. French has one translated-data detector study, one benchmark
|
|
180
|
+
that skips it, two practitioner skills, and typography rules. buvard should say this
|
|
181
|
+
on its README rather than pretend parity.
|
|
182
|
+
|
|
183
|
+
|
|
184
|
+
## 4. The detector and linter landscape
|
|
185
|
+
|
|
186
|
+
| Tool | Kind | What it actually does | Accuracy claims and FP profile |
|
|
187
|
+
|---|---|---|---|
|
|
188
|
+
| stop-slop (hardikpandya) | rules skill, MIT | an LLM agent skill, not a program: flags throat-clearing openers, business jargon, binary contrasts, passive voice; no benchmarks | none published; https://github.com/hardikpandya/stop-slop |
|
|
189
|
+
| humanizer (blader) | LLM skill, MIT | a rewrite skill over "26 patterns" (staging instead of stating, rhythm by rule, inflation and borrowed authority, formatting by rule); a blind test had judges prefer its rewrite 16/16 | no detection accuracy claim; it is a rewriter; https://github.com/blader/humanizer |
|
|
190
|
+
| deslop / unslop (Cursor ecosystem) | editor rules | could not verify from this environment: plausible slugs exist (cursor.directory returned 429, candidate GitHub repos 404) [unverified]; the pattern family is LLM-agent rules that strip slop from code comments and chat remnants | none |
|
|
191
|
+
| proselint | rules linter, BSD | the ancestor of this genre: a spell-checker for style, rules for cliches and errors, explicitly "aggregate knowledge about best practices in writing" | no accuracy claim; it reports rules fired, not authorship; https://github.com/amperser/proselint |
|
|
192
|
+
| Vale | rules linter, MIT | "brings code-like linting to prose": YAML-defined styles with regex and extension points, no model, explicitly not a detector | https://docs.vale.sh/ |
|
|
193
|
+
| GPTZero | ML service | perplexity and burstiness scores; claims "99.9% Accuracy" while its own text warns no detector is 100% accurate and results "should not be used to punish or as the final verdict" | https://gptzero.me/ ; the 61.22% TOEFL false-positive result (section 1) includes detectors of this class |
|
|
194
|
+
| Pangram | ML service | a "traditional language model architecture" classifier for text, image and video; claims "over 99% accurate" with a "1 in 10,000 False positive rate", "verified by researchers" at U Chicago and U Maryland | https://www.pangram.com/ ; the claim is vendor-published [inferred: independent replication details not accessible here] |
|
|
195
|
+
| Originality.ai | ML service | "Most Accurate AI Detector in third-party studies"; no specific FP rate on its own front page, no visible appeal process | https://originality.ai/ |
|
|
196
|
+
|
|
197
|
+
Reading the table as a competitive map for buvard:
|
|
198
|
+
|
|
199
|
+
1. **The rules side of the market is agent skills, not linters.** stop-slop and
|
|
200
|
+
humanizer are prompts for LLMs to rewrite text. Nobody ships a deterministic,
|
|
201
|
+
score-weighted, CI-runnable prose linter focused on AI slop. proselint and Vale
|
|
202
|
+
are deterministic but generic-style, with no slop catalog and no French. This is
|
|
203
|
+
buvard's empty seat: Vale's architecture (YAML rules, regex, severities, exit
|
|
204
|
+
codes) pointed at a maintained FR+EN slop catalog.
|
|
205
|
+
2. **ML detectors claim 99+ percent and disclaim everything.** GPTZero's own warning
|
|
206
|
+
against using results "to punish" is the confession the whole category shares, and
|
|
207
|
+
the peer-reviewed floor is the 61.22% false-positive rate on non-native writing
|
|
208
|
+
(section 1). Any tool that emits an authorship probability inherits this.
|
|
209
|
+
3. **None of the rules tools has French.** proselint is English-only; Vale's official
|
|
210
|
+
styles are English-first [inferred]; both FR skills are prompts. buvard's FR rules
|
|
211
|
+
(section 3 typography rules in particular) have no direct competitor at all.
|
|
212
|
+
4. **False-positive profile to design against.** The documented victims of AI
|
|
213
|
+
detection are non-native writers (arxiv 2304.02819) and, anecdotally across the
|
|
214
|
+
catalogs, autistic and ADHD writers whose prose is flatly organized or
|
|
215
|
+
formality-heavy [unverified, no published study found]. For a rules linter the
|
|
216
|
+
equivalent risk is harsh severity on ordinary rhetoric; buvard should default all
|
|
217
|
+
contrast-family rules to informational severity.
|
|
218
|
+
|
|
219
|
+
|
|
220
|
+
## 5. Implications for buvard
|
|
221
|
+
|
|
222
|
+
Ten recommendations, each stated so it can be falsified by a measurement:
|
|
223
|
+
|
|
224
|
+
1. **Never claim AI detection.** Position buvard as a style linter that reports
|
|
225
|
+
surface slop evidence. No authorship probability, no "percent human". Test: the
|
|
226
|
+
output schema contains counts and weighted scores only. Grounds: 61.22% FPR on
|
|
227
|
+
non-native writers (section 1) and GPTZero's own anti-punishment disclaimer
|
|
228
|
+
(section 4).
|
|
229
|
+
2. **Score the contrast family by density, not by instance.** Antithesis is
|
|
230
|
+
legitimate once and slop when uniform (section 2). Test: a pre-2022 Gutenberg
|
|
231
|
+
prose corpus should show the per-instance rule firing on classic essays at rates
|
|
232
|
+
indistinguishable from modern human prose, while the window rule separates the
|
|
233
|
+
corpora.
|
|
234
|
+
3. **Promote FR typography rules to hard penalties.** Insécable before ; : ! ?,
|
|
235
|
+
guillemet spacing, straight quotes: mechanically wrong per French typographic law
|
|
236
|
+
(fr.wikipedia.org/wiki/Espace_ins%C3%A9cable), near-zero false-positive risk, and
|
|
237
|
+
no competitor has them. Test: FP rate on the Gutenberg FR corpus should be ~0
|
|
238
|
+
after normalizing Unicode.
|
|
239
|
+
4. **Build the FR lexicon as frequency bands, not a flat list.** Tier 1 (always
|
|
240
|
+
slop: « il convient de noter », « à l'ère de », « plonger dans ») gets the full
|
|
241
|
+
weight; tier 2 (context-dependent: « crucial », « permet de ») fires only on
|
|
242
|
+
density. Grounds: the FR skills' two-tier structure (section 3) and the excess-
|
|
243
|
+
vocabulary finding that frequency shifts, not single words, are the robust signal
|
|
244
|
+
(arxiv 2406.07016).
|
|
245
|
+
5. **Cap total lexicon penalty per document.** Common-word-heavy prose is exactly
|
|
246
|
+
what detectors punish unfairly (arxiv 2304.02819), so buvard must not reproduce
|
|
247
|
+
the failure in lint form. Test: a text with 40 tier-2 hits and nothing else
|
|
248
|
+
stays below the "fail" threshold.
|
|
249
|
+
6. **Flag flat rhythm as an informational note.** Compute the coefficient of
|
|
250
|
+
variation of sentence lengths per 500-word window; flag low CV only in prose
|
|
251
|
+
genres where variance is expected (essays, narrative) and never in technical docs.
|
|
252
|
+
Thresholds are an open empirical question [inferred]; publish whatever the human
|
|
253
|
+
corpus shows.
|
|
254
|
+
7. **Keep the "X, so Y" pivot at low weight and dated folklore status.** No
|
|
255
|
+
independent source found (section 2). Test for promotion: a second practitioner
|
|
256
|
+
catalog or an n-gram frequency study showing post-2022 growth. Until then it
|
|
257
|
+
cannot be promoted above -2.
|
|
258
|
+
8. **Em-dash budgets per language.** EN 1 per 500 words (as shipped), FR tighter at
|
|
259
|
+
1 per 1000, both exempting dialogue lines. Grounds: the documented 2022-2026
|
|
260
|
+
em-dash spike is EN-centric (Wikipedia signs page), and FR prose uses tirets
|
|
261
|
+
mainly for dialogue and incises.
|
|
262
|
+
9. **Rule-of-three fires on the window, never the first tricolon.** Test: flagging
|
|
263
|
+
only at three or more tricolons per 500 words should leave the Gutenberg corpus
|
|
264
|
+
nearly untouched.
|
|
265
|
+
10. **Version the catalog like an API, with provenance on every rule.** Rules get
|
|
266
|
+
stable ids (`buvard.fr.lexique.007`), each carries `source` (URL + retrieval
|
|
267
|
+
date) and `severity`, the catalog semvers (adding a rule bumps minor, reweighting
|
|
268
|
+
bumps major since scores shift), and a machine-readable changelog. Grounds:
|
|
269
|
+
catalogs drift as LLMs change their tells; the em-dash range on the Wikipedia
|
|
270
|
+
page shows tells have lifetimes. Without provenance, a 2027 user cannot tell a
|
|
271
|
+
measured pattern from a 2024 hunch. This review's [unverified] marks are exactly
|
|
272
|
+
the metadata such a field would have made mandatory.
|
|
273
|
+
|
|
274
|
+
**Summary of the position.** The statistical literature says detection itself is
|
|
275
|
+
unreliable, biased against non-native writers, and defeated by paraphrasing; the
|
|
276
|
+
commercial market sells 99% claims with disclaimers attached; the rules layer of the
|
|
277
|
+
market is prompt skills with no determinism, no scores, and no French. A deterministic,
|
|
278
|
+
score-weighted, provenance-tracked FR+EN style linter with a public false-positive
|
|
279
|
+
test corpus occupies ground that nobody currently holds, and its honest claim ("your
|
|
280
|
+
text reads like the slop everyone complains about") is the only claim in this space
|
|
281
|
+
that survives contact with the evidence.
|
|
282
|
+
|