humanizer-lint 0.2.3 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +556 -470
- package/SKILL.md +132 -116
- package/package.json +47 -25
- package/scripts/analyze.mjs +87 -408
- package/scripts/core/analyzer-core.mjs +352 -0
- package/scripts/utils/metrics-core.mjs +84 -0
- package/scripts/utils/metrics.mjs +28 -73
package/README.md
CHANGED
|
@@ -1,470 +1,556 @@
|
|
|
1
|
-
<div align="center">
|
|
2
|
-
|
|
3
|
-
# humanizer-lint
|
|
4
|
-
|
|
5
|
-
### Deterministic prose and Markdown style linting for Indonesian and English
|
|
6
|
-
|
|
7
|
-
Detect repetitive LLM phrasing, uniform sentence rhythm, structural clichés, and voice-profile drift without sending text to an external service.
|
|
8
|
-
|
|
9
|
-
[](https://github.com/yanstd888/humanizer-lint/releases)
|
|
10
|
-
[](https://github.com/yanstd888/humanizer-lint/releases)
|
|
10
|
+
[](https://www.npmjs.com/package/humanizer-lint)
|
|
11
|
+
[](https://github.com/yanstd888/humanizer-lint/actions/workflows/ci.yml)
|
|
12
|
+
[](package.json)
|
|
13
|
+
[](https://www.skills.sh/yanstd888/humanizer-lint/humanizer-lint/security/snyk)
|
|
14
|
+
[](https://www.skills.sh/yanstd888/humanizer-lint/humanizer-lint/security/socket)
|
|
15
|
+
[](https://www.skills.sh/yanstd888/humanizer-lint/humanizer-lint/security/agent-trust-hub)
|
|
16
|
+
[](https://humanizer.aiclub.ai.id/)
|
|
17
|
+
[](LICENSE)
|
|
18
|
+
[](https://nodejs.org/)
|
|
19
|
+
[](https://www.skills.sh/yanstd888/humanizer-lint/humanizer-lint)
|
|
20
|
+
[](https://github.com/yanstd888/humanizer-lint/stargazers)
|
|
21
|
+
|
|
22
|
+
[What it does](#what-it-does) | [Try in your browser](#try-in-your-browser) | [Install](#install) | [Quick start](#quick-start) | [CLI reference](#cli-reference) | [Detectors](#detectors-and-violation-types) | [Agent skill](#use-as-an-agent-skill) | [FAQ](#frequently-asked-questions)
|
|
23
|
+
|
|
24
|
+
</div>
|
|
25
|
+
|
|
26
|
+
**humanizer-lint is an open-source natural-text and Markdown prose linter that deterministically identifies style patterns often associated with AI-generated writing in Indonesian and English; it is a style validator, not an authorship detector.**
|
|
27
|
+
|
|
28
|
+
```sh
|
|
29
|
+
npx skills add yanstd888/humanizer-lint
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Use the command above to install the agent skill. For one-off local linting, run the published `humanizer-lint` CLI below.
|
|
33
|
+
|
|
34
|
+
## Try in your browser
|
|
35
|
+
|
|
36
|
+
Use the official demo: [humanizer.aiclub.ai.id](https://humanizer.aiclub.ai.id).
|
|
37
|
+
|
|
38
|
+
[humanizer-lint.pages.dev](https://humanizer-lint.pages.dev) remains available as a mirror.
|
|
39
|
+
|
|
40
|
+
Open a deterministic sample view directly:
|
|
41
|
+
|
|
42
|
+
| Sample | URL | Expected result |
|
|
43
|
+
|---|---|---|
|
|
44
|
+
| AI sample (ID) | [open](https://humanizer.aiclub.ai.id/?sample=ai-id) | `FAIL`, score `< 65` |
|
|
45
|
+
| Human sample (ID) | [open](https://humanizer.aiclub.ai.id/?sample=human-id) | `PASS`, score `>= 88` |
|
|
46
|
+
| AI sample (EN) | [open](https://humanizer.aiclub.ai.id/?sample=ai-en) | `FAIL`, score `< 65` |
|
|
47
|
+
| Human sample (EN) | [open](https://humanizer.aiclub.ai.id/?sample=human-en) | `PASS`, score `>= 88` |
|
|
48
|
+
|
|
49
|
+
Use `?lang=en` or `?lang=id` to open the playground chrome in a chosen language for that visit; it does not overwrite a saved browser preference.
|
|
50
|
+
|
|
51
|
+
It works locally with zero runtime dependencies. Instead of rewriting text through another language model, it reports paragraph- and sentence-level violations with specific correction instructions. Use it to lint LLM-generated documentation, remove robotic AI phrasing from Markdown, check burstiness and lexical diversity, or add a deterministic prose gate to an agent workflow.
|
|
52
|
+
|
|
53
|
+
It can serve as a CLI text humanizer guardrail, automated writing style linter, or CI/CD documentation linter. The bundled rules flag overused AI clichés such as `delve` and `tapestry` without rewriting the source.
|
|
54
|
+
|
|
55
|
+
## What it does
|
|
56
|
+
|
|
57
|
+
`humanizer-lint` analyzes prose after removing or neutralizing common non-prose Markdown structures. It then combines lexical rules, sentence-length statistics, structural checks, Indonesian morphology checks, and an optional voice profile into a score from 0 to 100.
|
|
58
|
+
|
|
59
|
+
The project includes:
|
|
60
|
+
|
|
61
|
+
- 86 reviewable lexical and structural regex rules: 42 Indonesian and 44 English.
|
|
62
|
+
- Sentence-length burstiness, adjacent rhythm delta, short/long sentence ratios, and adaptive MATTR.
|
|
63
|
+
- Repetition checks based on paragraph symmetry, repeated sentence starters, and compression ratio.
|
|
64
|
+
- Indonesian nominalization, passive-voice, loanword, and translation-style opening checks.
|
|
65
|
+
- Optional voice profiles generated from one to five writing samples.
|
|
66
|
+
- Human-readable terminal output and pure JSON output.
|
|
67
|
+
- Standard exit codes for agents, scripts, and CI jobs.
|
|
68
|
+
|
|
69
|
+
It does not estimate perplexity, classify who wrote the text, rewrite prose automatically, scan directories, or provide SARIF output.
|
|
70
|
+
|
|
71
|
+
## Terminal demo
|
|
72
|
+
|
|
73
|
+
Given this input:
|
|
74
|
+
|
|
75
|
+
```text
|
|
76
|
+
In today's fast-paced world, teams delve into a robust landscape. Moreover, this pivotal approach fosters seamless collaboration across every project. In conclusion, the solution unlocks valuable insights for everyone.
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Run the analyzer:
|
|
80
|
+
|
|
81
|
+
```sh
|
|
82
|
+
humanizer-lint --file draft.md --lang en
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The current CLI reports:
|
|
86
|
+
|
|
87
|
+
```text
|
|
88
|
+
Status: FAIL
|
|
89
|
+
Score: 39
|
|
90
|
+
Language: en
|
|
91
|
+
|
|
92
|
+
Metrics:
|
|
93
|
+
wordCount: 29
|
|
94
|
+
sentenceCount: 3
|
|
95
|
+
meanSentenceLength: 9.6667
|
|
96
|
+
stdDev: 0.4714
|
|
97
|
+
burstinessCV: 0.0488
|
|
98
|
+
adjacentDelta: 0.5
|
|
99
|
+
shortSentenceRatio: 0
|
|
100
|
+
longSentenceRatio: 0
|
|
101
|
+
mattr: 1
|
|
102
|
+
compressionRatio: 0.6972
|
|
103
|
+
nominalizationRatio: 0
|
|
104
|
+
passiveRatio: 0
|
|
105
|
+
voiceProfileDeviation: 0.5319
|
|
106
|
+
|
|
107
|
+
Violations:
|
|
108
|
+
SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the exact action or subject instead.
|
|
109
|
+
SLOP_PHRASE | Paragraph 1, Sentence 2 | Describe the actual consequence without inflated significance.
|
|
110
|
+
SLOP_PHRASE | Paragraph 1, Sentence 2 | Name the concrete behavior or outcome.
|
|
111
|
+
SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the field or conditions directly.
|
|
112
|
+
SLOP_PHRASE | Paragraph 1, Sentence 3 | Name what becomes possible and how.
|
|
113
|
+
SLOP_PHRASE | Paragraph 1, Sentence 2 | Connect the sentences by their actual relationship.
|
|
114
|
+
SLOP_PHRASE | Paragraph 1, Sentence 3 | End on the final concrete point.
|
|
115
|
+
SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the current condition precisely.
|
|
116
|
+
SLOP_PHRASE | Paragraph 1, Sentence 2 | Describe the missing step, delay, or transition precisely.
|
|
117
|
+
SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the failure mode or capacity the system handles.
|
|
118
|
+
SLOP_PHRASE | Paragraph 1, Sentence 3 | State the finding itself.
|
|
119
|
+
MONOTONOUS_RHYTHM | Paragraph 1, Sentences 1-3 | Sentence lengths are uniform (10, 10, 9 words). Split one sentence into a short statement or merge two into a longer clause.
|
|
120
|
+
LOW_BURSTINESS | Whole text | Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it.
|
|
121
|
+
LOW_ADJACENT_DELTA | Whole text | Change neighboring sentence lengths by splitting or merging only where the content allows it.
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
A failed analysis exits with code `1`. A passing fixture returns:
|
|
125
|
+
|
|
126
|
+
```text
|
|
127
|
+
Status: PASS
|
|
128
|
+
Score: 100
|
|
129
|
+
Language: en
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
and exits with code `0`.
|
|
133
|
+
|
|
134
|
+
## Static `SKILL.md` Humanizers vs. `humanizer-lint` (Closed-Loop CLI)
|
|
135
|
+
|
|
136
|
+
A static writing prompt can ask a model to vary prose, but it cannot verify the finished draft. `humanizer-lint` takes the smaller, deterministic approach: inspect the exact text after drafting, make only flagged revisions, then check again locally.
|
|
137
|
+
|
|
138
|
+
| Capability | Static `SKILL.md` humanizer prompt | `humanizer-lint` closed-loop CLI |
|
|
139
|
+
|---|---|---|
|
|
140
|
+
| Checks the completed draft | No | Yes, exact input is measured |
|
|
141
|
+
| Measures sentence rhythm | Prompt guidance only | CV, adjacent delta, short/long distribution |
|
|
142
|
+
| Measures vocabulary repetition | Prompt guidance only | Adaptive MATTR and repetition checks |
|
|
143
|
+
| Flags configured AI clichés | Model-dependent | Rule IDs and locations |
|
|
144
|
+
| Preserves source text | Depends on rewrite | Yes, analysis only |
|
|
145
|
+
| Runs offline after installation | Depends on agent/model | Yes |
|
|
146
|
+
| Supports EN and ID lexical rules | Prompt-dependent | Yes |
|
|
147
|
+
| Produces JSON and exit codes | Usually no | Yes |
|
|
148
|
+
| Proves human authorship | No | No |
|
|
149
|
+
|
|
150
|
+
If you are looking for an AI writing detector linter, a ChatGPT tone remover CLI, or an open-source alternative to AI humanizers, note the boundary: this tool detects configured style signals and rhythm problems. It does not claim to determine whether a human or model wrote the text.
|
|
151
|
+
|
|
152
|
+
## Install
|
|
153
|
+
|
|
154
|
+
### CLI
|
|
155
|
+
|
|
156
|
+
Run the published CLI without a global install:
|
|
157
|
+
|
|
158
|
+
```sh
|
|
159
|
+
npx humanizer-lint --file tulisan.md --lang id
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Or install it globally:
|
|
163
|
+
|
|
164
|
+
```sh
|
|
165
|
+
npm install -g humanizer-lint
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
### Install as an agent skill
|
|
169
|
+
|
|
170
|
+
The recommended installation for coding agents is through the skills CLI:
|
|
171
|
+
|
|
172
|
+
```sh
|
|
173
|
+
npx skills add yanstd888/humanizer-lint
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
The public listing, installation command, and install count are available on [skills.sh](https://skills.sh/yanstd888/humanizer-lint).
|
|
177
|
+
|
|
178
|
+
### Development (from source)
|
|
179
|
+
|
|
180
|
+
Clone the repository and use `npm link` only when developing the local source:
|
|
181
|
+
|
|
182
|
+
```sh
|
|
183
|
+
git clone https://github.com/yanstd888/humanizer-lint.git
|
|
184
|
+
cd humanizer-lint
|
|
185
|
+
npm test
|
|
186
|
+
npm link
|
|
187
|
+
humanizer-lint --help
|
|
188
|
+
humanizer-fingerprint --help
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
No dependency installation or build step is required.
|
|
192
|
+
|
|
193
|
+
## Quick start
|
|
194
|
+
|
|
195
|
+
Analyze one file:
|
|
196
|
+
|
|
197
|
+
```sh
|
|
198
|
+
humanizer-lint --file draft.md --lang auto
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Return machine-readable JSON:
|
|
202
|
+
|
|
203
|
+
```sh
|
|
204
|
+
humanizer-lint --file draft.md --lang id --json
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
Analyze stdin without creating a temporary file:
|
|
208
|
+
|
|
209
|
+
```sh
|
|
210
|
+
cat draft.md | humanizer-lint --stdin --lang auto --json
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
Use a custom voice profile:
|
|
214
|
+
|
|
215
|
+
```sh
|
|
216
|
+
humanizer-lint --file draft.md --lang en --profile profiles/custom-voice.json --json
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
Without `npm link`, replace `humanizer-lint` with `node scripts/analyze.mjs` and replace `humanizer-fingerprint` with `node scripts/fingerprint.mjs`.
|
|
220
|
+
|
|
221
|
+
## Contoh / Example
|
|
222
|
+
|
|
223
|
+
Skor mengukur kecocokan dengan sinyal gaya yang diperiksa oleh linter: skor lebih tinggi berarti lebih sedikit pelanggaran yang ditemukan. Ini bukan skor probabilitas penulis AI atau manusia.
|
|
224
|
+
|
|
225
|
+
### Before
|
|
226
|
+
|
|
227
|
+
```text
|
|
228
|
+
Di era digital saat ini, penggunaan teknologi menjadi sangat penting bagi setiap bisnis. Tidak hanya itu, solusi ini juga memberikan manfaat yang sangat besar bagi semua pihak. Dengan demikian, organisasi dapat meningkatkan efisiensi secara signifikan dalam berbagai aspek. Pada akhirnya, pendekatan ini membuka peluang baru yang lebih optimal bagi masa depan.
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
Output dari `tests/fixtures/ai-before-id.md`:
|
|
232
|
+
|
|
233
|
+
```text
|
|
234
|
+
Status: FAIL
|
|
235
|
+
Score: 38
|
|
236
|
+
Language: id
|
|
237
|
+
|
|
238
|
+
Violations:
|
|
239
|
+
SLOP_PHRASE | Paragraph 1, Sentence 1 | Mulai langsung dari subjek atau fakta konkret.
|
|
240
|
+
SLOP_PHRASE | Paragraph 1, Sentence 3 | Sebutkan aspek yang relevan.
|
|
241
|
+
SLOP_PHRASE | Paragraph 1, Sentence 4 | Tutup langsung dengan akibat atau keputusan akhir.
|
|
242
|
+
MONOTONOUS_RHYTHM | Paragraph 1, Sentences 1-3 | Sentence lengths are uniform (13, 14, 11 words).
|
|
243
|
+
LOW_BURSTINESS | Whole text | Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it.
|
|
244
|
+
LOW_ADJACENT_DELTA | Whole text | Change neighboring sentence lengths by splitting or merging only where the content allows it.
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
### After
|
|
248
|
+
|
|
249
|
+
```text
|
|
250
|
+
Pemilik toko memakai aplikasi stok untuk melihat barang yang habis pada hari yang sama. Semua melihat angka yang sama. Saat satu produk menipis, petugas memesan ulang sebelum rak kosong dan mencatat alasan setiap perubahan. Laporan harian menunjukkan barang mana yang laku, barang mana yang tertahan, dan kapan petugas harus menghubungi pemasok lagi agar toko tidak kehabisan barang yang dicari pelanggan. Pemilik toko kini membeli barang berdasarkan catatan, bukan tebakan.
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
Output dari `tests/fixtures/human-after-id.md`:
|
|
254
|
+
|
|
255
|
+
```text
|
|
256
|
+
Status: PASS
|
|
257
|
+
Score: 100
|
|
258
|
+
Language: id
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
Untuk contoh ini, skor naik dari **38** menjadi **100** setelah frasa template dihilangkan, subjek dibuat konkret, dan panjang kalimat divariasikan.
|
|
262
|
+
|
|
263
|
+
## CLI reference
|
|
264
|
+
|
|
265
|
+
### Analyzer
|
|
266
|
+
|
|
267
|
+
```text
|
|
268
|
+
humanizer-lint (--file <path> | --stdin) [--lang id|en|auto] [--profile <path>] [--json]
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
| Option | Required | Description |
|
|
272
|
+
|---|---:|---|
|
|
273
|
+
| `--file <path>` | One input required | Read one UTF-8 text or Markdown file. |
|
|
274
|
+
| `--stdin` | One input required | Read the complete document from standard input. |
|
|
275
|
+
| `--lang id|en|auto` | No | Select Indonesian, English, or function-word auto-detection. Default: `auto`. |
|
|
276
|
+
| `--profile <path>` | No | Load a JSON voice profile. |
|
|
277
|
+
| `--json` | No | Print JSON only. Without it, print the terminal report. |
|
|
278
|
+
| `--help` | No | Print command usage. |
|
|
279
|
+
|
|
280
|
+
Exactly one of `--file` or `--stdin` is required.
|
|
281
|
+
|
|
282
|
+
### Voice fingerprint
|
|
283
|
+
|
|
284
|
+
```text
|
|
285
|
+
humanizer-fingerprint --input <file-or-dir> --output <profile.json> [--name <name>] [--lang id|en|auto]
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
| Option | Required | Description |
|
|
289
|
+
|---|---:|---|
|
|
290
|
+
| `--input <path>` | Yes | Read one file or up to five `.md` and `.txt` files from one directory. |
|
|
291
|
+
| `--output <path>` | Yes | Write the generated profile atomically. |
|
|
292
|
+
| `--name <name>` | No | Set the profile name. Default: `custom-voice`. |
|
|
293
|
+
| `--lang id|en|auto` | No | Record the intended language. Default: `auto`. |
|
|
294
|
+
| `--help` | No | Print command usage. |
|
|
295
|
+
|
|
296
|
+
### Exit codes
|
|
297
|
+
|
|
298
|
+
| Code | Meaning |
|
|
299
|
+
|---:|---|
|
|
300
|
+
| `0` | `PASS`, or `SKIP` for text too short for statistical scoring. |
|
|
301
|
+
| `1` | Analysis completed and returned `FAIL`. |
|
|
302
|
+
| `2` | Invalid arguments, unreadable input, or malformed profile. |
|
|
303
|
+
|
|
304
|
+
## Detectors and violation types
|
|
305
|
+
|
|
306
|
+
The JSON rule catalogs contain stable IDs. Runtime statistical checks use named violation types.
|
|
307
|
+
|
|
308
|
+
| Violation type or rule range | Language | What it detects |
|
|
309
|
+
|---|---|---|
|
|
310
|
+
| `ID001`-`ID040`, `EN001`-`EN040` / `SLOP_PHRASE` | Indonesian and English | Configured clichés, generic transitions, inflated wording, and repeated LLM vocabulary such as `delve`, `tapestry`, `testament`, and their Indonesian equivalents. |
|
|
311
|
+
| `ID041`-`ID042` / `DEFINITIONAL_OPENING` | Indonesian | Formulaic paragraph openings and rigid definition templates. |
|
|
312
|
+
| `EN041`-`EN044` / `STRUCTURAL_CLICHE` | English | Participial tails, binary formulas, repeated double-hyphens, and similar structural templates. |
|
|
313
|
+
| `MONOTONOUS_RHYTHM` | Both | Three adjacent sentences whose lengths differ by at most three words. |
|
|
314
|
+
| `LOW_BURSTINESS` | Both | Sentence-length coefficient of variation below the configured threshold. |
|
|
315
|
+
| `MISSING_SENTENCE_RANGE` | Both | Text with at least eight sentences that lacks enough short or long sentences. |
|
|
316
|
+
| `LOW_ADJACENT_DELTA` | Both | Consecutive sentence lengths that change too little. |
|
|
317
|
+
| `LOW_LEXICAL_DIVERSITY` | Both | Adaptive MATTR below the language threshold. |
|
|
318
|
+
| `LOW_ENTROPY_REPETITION` | Both | Low DEFLATE compression ratio in text of at least 150 words. |
|
|
319
|
+
| `SYMMETRIC_PARAGRAPHS` | Both | Three consecutive paragraphs with the same sentence count. |
|
|
320
|
+
| `REPETITIVE_STARTERS` | Both | A paragraph with at least three sentences beginning with the same token. |
|
|
321
|
+
| `EXCESSIVE_NOMINALIZATION` | Indonesian | Nominalization density above the configured threshold after whitelist filtering. |
|
|
322
|
+
| `PASSIVE_BLOATING` | Indonesian | Passive morphology density or consecutive translation-style subordinate openings. |
|
|
323
|
+
| `VOICE_PROFILE_DEVIATION` | Both | Custom-profile mean sentence length that differs from the target by more than 30 percent. |
|
|
324
|
+
|
|
325
|
+
## Metrics and scoring
|
|
326
|
+
|
|
327
|
+
| Metric | Definition |
|
|
328
|
+
|---|---|
|
|
329
|
+
| `burstinessCV` | Population standard deviation divided by mean sentence length. |
|
|
330
|
+
| `adjacentDelta` | Mean absolute difference between neighboring sentence lengths. |
|
|
331
|
+
| `mattr` | Moving-average type-token ratio over an adaptive 10- to 50-token window. |
|
|
332
|
+
| `compressionRatio` | Raw DEFLATE byte length divided by original UTF-8 byte length. |
|
|
333
|
+
| `shortSentenceRatio` | Share of sentences containing six words or fewer. |
|
|
334
|
+
| `longSentenceRatio` | Share of sentences containing 26 words or more. |
|
|
335
|
+
| `voiceProfileDeviation` | Mean relative deviation from available profile targets. It is not an authorship probability. |
|
|
336
|
+
|
|
337
|
+
The score starts at 100 and applies these caps:
|
|
338
|
+
|
|
339
|
+
| Category | Maximum deduction |
|
|
340
|
+
|---|---:|
|
|
341
|
+
| Burstiness and sentence-range checks | 25 |
|
|
342
|
+
| Monotonous rhythm windows | 20 |
|
|
343
|
+
| Lexical and structural clichés | 30 |
|
|
344
|
+
| Indonesian morphology | 15 |
|
|
345
|
+
| Vocabulary, entropy, and custom-profile deviation combined | 10 |
|
|
346
|
+
|
|
347
|
+
A normal-length document passes only when the score is at least 80, no `SLOP_PHRASE` or `DEFINITIONAL_OPENING` remains, burstiness and adjacent-delta gates pass, and the sentence-range requirement passes when applicable.
|
|
348
|
+
|
|
349
|
+
Documents below 20 words or three sentences return `SKIP`. Lexical slop phrases are still reported.
|
|
350
|
+
|
|
351
|
+
## Markdown sanitization
|
|
352
|
+
|
|
353
|
+
Before tokenization, the analyzer:
|
|
354
|
+
|
|
355
|
+
- removes YAML frontmatter, fenced code blocks, headings, and Markdown tables;
|
|
356
|
+
- keeps Markdown link anchor text while removing the URL;
|
|
357
|
+
- replaces inline code and bare URLs with `TOKEN_ENTITY`;
|
|
358
|
+
- removes short or unfinished list fragments while keeping complete list prose;
|
|
359
|
+
- removes blockquote markers while preserving the quoted prose;
|
|
360
|
+
- preserves paragraph boundaries for locations and structural checks.
|
|
361
|
+
|
|
362
|
+
This is deterministic sanitization, not a full Markdown or MDX abstract syntax tree parser. HTML blocks and arbitrary MDX components do not receive special parsing.
|
|
363
|
+
|
|
364
|
+
## Voice profiles
|
|
365
|
+
|
|
366
|
+
Generate a profile from one to five samples:
|
|
367
|
+
|
|
368
|
+
```sh
|
|
369
|
+
humanizer-fingerprint \
|
|
370
|
+
--input writing-samples \
|
|
371
|
+
--output profiles/custom-voice.json \
|
|
372
|
+
--name editorial-voice \
|
|
373
|
+
--lang en
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
The profile records mean sentence length, standard deviation, burstiness, adjacent delta, MATTR, average paragraph sentence count, and punctuation rates. Supplying it with `--profile` adds `voiceProfileDeviation`; a mean sentence-length drift above 30 percent creates a five-point profile penalty inside the 10-point vocabulary and entropy cap.
|
|
377
|
+
|
|
378
|
+
## JSON output
|
|
379
|
+
|
|
380
|
+
`--json` writes a single JSON object to stdout. The stable top-level fields are:
|
|
381
|
+
|
|
382
|
+
```json
|
|
383
|
+
{
|
|
384
|
+
"status": "FAIL",
|
|
385
|
+
"score": 39,
|
|
386
|
+
"language": "en",
|
|
387
|
+
"metrics": {
|
|
388
|
+
"wordCount": 29,
|
|
389
|
+
"sentenceCount": 3,
|
|
390
|
+
"burstinessCV": 0.0488,
|
|
391
|
+
"adjacentDelta": 0.5,
|
|
392
|
+
"mattr": 1
|
|
393
|
+
},
|
|
394
|
+
"violations": [
|
|
395
|
+
{
|
|
396
|
+
"type": "LOW_BURSTINESS",
|
|
397
|
+
"location": "Whole text",
|
|
398
|
+
"actual": 0.0488,
|
|
399
|
+
"target": 0.4,
|
|
400
|
+
"fixInstruction": "Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it."
|
|
401
|
+
}
|
|
402
|
+
]
|
|
403
|
+
}
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
The real result may contain additional metric properties and violations.
|
|
407
|
+
|
|
408
|
+
## Use as an agent skill
|
|
409
|
+
|
|
410
|
+
After installation, an agent can use the skill as a closed verification loop:
|
|
411
|
+
|
|
412
|
+
1. Draft or edit Indonesian or English prose without changing supported facts.
|
|
413
|
+
2. Run the analyzer against the exact complete draft.
|
|
414
|
+
3. Read every violation and edit only the affected prose or named whole-document gate.
|
|
415
|
+
4. Re-run the analyzer, stopping after three iterations.
|
|
416
|
+
5. Return only a `PASS`, or an honestly labeled short-text `SKIP` with no unresolved lexical violations.
|
|
417
|
+
|
|
418
|
+
The skill is indexed on skills.sh and its current listing passes the platform's Snyk security audit.
|
|
419
|
+
|
|
420
|
+
## Used by
|
|
421
|
+
|
|
422
|
+
The skills.sh badge at the top of this README is the public source for anonymous installation activity. GitHub's native `Used by` sidebar is generated automatically from its dependency graph; repository owners cannot set that count manually.
|
|
423
|
+
|
|
424
|
+
Because the CLI is published on npm, npm downloads and downstream package manifests are the relevant distribution signals. The skills.sh listing separately represents agent-skill installation activity.
|
|
425
|
+
|
|
426
|
+
## CI and repository automation
|
|
427
|
+
|
|
428
|
+
This repository keeps `.github` intentionally small. The included `.github/workflows/ci.yml` workflow runs the test suite and package dry-run on supported Node.js versions. A CI badge is useful because it reflects an actual workflow; issue templates, Dependabot, and release automation are omitted until the repository needs them.
|
|
429
|
+
|
|
430
|
+
To use the analyzer in another GitHub Actions workflow without an npm release, check out this repository into a subdirectory and run the script directly:
|
|
431
|
+
|
|
432
|
+
```yaml
|
|
433
|
+
name: prose-check
|
|
434
|
+
|
|
435
|
+
on:
|
|
436
|
+
pull_request:
|
|
437
|
+
paths:
|
|
438
|
+
- "**/*.md"
|
|
439
|
+
- "**/*.txt"
|
|
440
|
+
|
|
441
|
+
jobs:
|
|
442
|
+
lint-readme:
|
|
443
|
+
runs-on: ubuntu-latest
|
|
444
|
+
steps:
|
|
445
|
+
- uses: actions/checkout@v4
|
|
446
|
+
- uses: actions/checkout@v4
|
|
447
|
+
with:
|
|
448
|
+
repository: yanstd888/humanizer-lint
|
|
449
|
+
path: .humanizer-lint
|
|
450
|
+
- uses: actions/setup-node@v4
|
|
451
|
+
with:
|
|
452
|
+
node-version: "22"
|
|
453
|
+
- run: node .humanizer-lint/scripts/analyze.mjs --file README.md --lang auto --json
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
The analyzer accepts one file per invocation. Invoke it once for each document you want to gate.
|
|
457
|
+
|
|
458
|
+
## Security properties
|
|
459
|
+
|
|
460
|
+
Production scripts use only Node.js built-ins and have no runtime dependencies. The test suite rejects production use of `child_process`, `eval`, dynamic `Function` construction, and `fetch`. The fingerprint writer writes to a temporary file and renames it into place.
|
|
461
|
+
|
|
462
|
+
Text remains local unless the surrounding agent or workflow sends it elsewhere.
|
|
463
|
+
|
|
464
|
+
## Frequently asked questions
|
|
465
|
+
|
|
466
|
+
<details>
|
|
467
|
+
<summary><strong>Is humanizer-lint an AI content detector?</strong></summary>
|
|
468
|
+
|
|
469
|
+
No. It detects configured lexical, structural, rhythm, and morphology signals. A pass or failure does not prove who wrote the text.
|
|
470
|
+
|
|
471
|
+
</details>
|
|
472
|
+
|
|
473
|
+
<details>
|
|
474
|
+
<summary><strong>Does humanizer-lint rewrite or auto-fix prose?</strong></summary>
|
|
475
|
+
|
|
476
|
+
No. It returns correction instructions but does not modify the source file. This prevents silent factual or formatting changes.
|
|
477
|
+
|
|
478
|
+
</details>
|
|
479
|
+
|
|
480
|
+
<details>
|
|
481
|
+
<summary><strong>Can it remove robotic AI phrasing from Markdown?</strong></summary>
|
|
482
|
+
|
|
483
|
+
It can identify the exact paragraph or sentence that matches a configured phrase or style gate. A human or coding agent then makes the smallest supported edit and runs the analyzer again.
|
|
484
|
+
|
|
485
|
+
</details>
|
|
486
|
+
|
|
487
|
+
<details>
|
|
488
|
+
<summary><strong>Can I install it from npm?</strong></summary>
|
|
489
|
+
|
|
490
|
+
Yes. Run `npx humanizer-lint --file draft.md --lang auto` for a one-off command, or install it globally with `npm install -g humanizer-lint`. Install the agent skill separately with `npx skills add yanstd888/humanizer-lint`.
|
|
491
|
+
|
|
492
|
+
</details>
|
|
493
|
+
|
|
494
|
+
<details>
|
|
495
|
+
<summary><strong>Which languages are supported?</strong></summary>
|
|
496
|
+
|
|
497
|
+
The configured lexical and language-detection paths support Indonesian and English. Do not treat the current statistical thresholds as validated defaults for other languages.
|
|
498
|
+
|
|
499
|
+
</details>
|
|
500
|
+
|
|
501
|
+
<details>
|
|
502
|
+
<summary><strong>Does it send text to an API?</strong></summary>
|
|
503
|
+
|
|
504
|
+
No. Production code has no network call. The skills installer and Git operations need network access only to download the repository.
|
|
505
|
+
|
|
506
|
+
</details>
|
|
507
|
+
|
|
508
|
+
<details>
|
|
509
|
+
<summary><strong>Why did a short document return SKIP?</strong></summary>
|
|
510
|
+
|
|
511
|
+
Documents below 20 words or three sentences do not contain enough material for the configured statistical gates. Lexical slop phrases are still checked.
|
|
512
|
+
|
|
513
|
+
</details>
|
|
514
|
+
|
|
515
|
+
## Contributing
|
|
516
|
+
|
|
517
|
+
Contributions should remain deterministic, zero-dependency, and covered by a focused regression test.
|
|
518
|
+
|
|
519
|
+
1. Fork the repository.
|
|
520
|
+
2. Create a focused branch.
|
|
521
|
+
3. Add or update one minimal test for non-trivial behavior.
|
|
522
|
+
4. Run `npm test` and `npm pack --dry-run`.
|
|
523
|
+
5. Open a pull request that explains the user-visible effect.
|
|
524
|
+
|
|
525
|
+
Every repository change updates the package and skill version, then publishes a matching Git tag and GitHub release after verification.
|
|
526
|
+
|
|
527
|
+
## Project layout
|
|
528
|
+
|
|
529
|
+
```text
|
|
530
|
+
.github/
|
|
531
|
+
workflows/
|
|
532
|
+
ci.yml
|
|
533
|
+
LICENSE
|
|
534
|
+
README.md
|
|
535
|
+
SKILL.md
|
|
536
|
+
package.json
|
|
537
|
+
profiles/
|
|
538
|
+
default-human.json
|
|
539
|
+
rules/
|
|
540
|
+
en-slop.json
|
|
541
|
+
id-slop.json
|
|
542
|
+
scripts/
|
|
543
|
+
analyze.mjs
|
|
544
|
+
fingerprint.mjs
|
|
545
|
+
utils/
|
|
546
|
+
metrics.mjs
|
|
547
|
+
sanitizer.mjs
|
|
548
|
+
tokenizer.mjs
|
|
549
|
+
tests/
|
|
550
|
+
fixtures/
|
|
551
|
+
test-runner.mjs
|
|
552
|
+
```
|
|
553
|
+
|
|
554
|
+
## License
|
|
555
|
+
|
|
556
|
+
Licensed under the [MIT License](LICENSE).
|