humanizer-lint 0.2.3 → 0.2.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,470 +1,529 @@
1
- <div align="center">
2
-
3
- # humanizer-lint
4
-
5
- ### Deterministic prose and Markdown style linting for Indonesian and English
6
-
7
- Detect repetitive LLM phrasing, uniform sentence rhythm, structural clichés, and voice-profile drift without sending text to an external service.
8
-
9
- [![GitHub Release](https://img.shields.io/github/v/release/yanstd888/humanizer-lint?style=flat-square&logo=github)](https://github.com/yanstd888/humanizer-lint/releases)
10
- [![CI](https://img.shields.io/github/actions/workflow/status/yanstd888/humanizer-lint/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/yanstd888/humanizer-lint/actions/workflows/ci.yml)
11
- [![License](https://img.shields.io/github/license/yanstd888/humanizer-lint?style=flat-square)](LICENSE)
12
- [![Node.js](https://img.shields.io/badge/Node.js-%3E%3D18.3-339933?style=flat-square&logo=nodedotjs&logoColor=white)](https://nodejs.org/)
13
- [![skills.sh](https://skills.sh/b/yanstd888/humanizer-lint)](https://skills.sh/yanstd888/humanizer-lint)
14
- [![GitHub Stars](https://img.shields.io/github/stars/yanstd888/humanizer-lint?style=flat-square)](https://github.com/yanstd888/humanizer-lint/stargazers)
15
-
16
- [What it does](#what-it-does) | [Quick start](#quick-start) | [CLI reference](#cli-reference) | [Detectors](#detectors-and-violation-types) | [Agent skill](#use-as-an-agent-skill) | [FAQ](#frequently-asked-questions)
17
-
18
- </div>
19
-
20
- **humanizer-lint is an open-source natural-text and Markdown prose linter that deterministically identifies style patterns often associated with AI-generated writing in Indonesian and English; it is a style validator, not an authorship detector.**
21
-
22
- It works locally with zero runtime dependencies. Instead of rewriting text through another language model, it reports paragraph- and sentence-level violations with specific correction instructions. Use it to lint LLM-generated documentation, remove robotic AI phrasing from Markdown, check burstiness and lexical diversity, or add a deterministic prose gate to an agent workflow.
23
-
24
- It can serve as a CLI text humanizer guardrail, automated writing style linter, or CI/CD documentation linter. The bundled rules flag overused AI clichés such as `delve` and `tapestry` without rewriting the source.
25
-
26
- ## What it does
27
-
28
- `humanizer-lint` analyzes prose after removing or neutralizing common non-prose Markdown structures. It then combines lexical rules, sentence-length statistics, structural checks, Indonesian morphology checks, and an optional voice profile into a score from 0 to 100.
29
-
30
- The project includes:
31
-
32
- - 86 reviewable lexical and structural regex rules: 42 Indonesian and 44 English.
33
- - Sentence-length burstiness, adjacent rhythm delta, short/long sentence ratios, and adaptive MATTR.
34
- - Repetition checks based on paragraph symmetry, repeated sentence starters, and compression ratio.
35
- - Indonesian nominalization, passive-voice, loanword, and translation-style opening checks.
36
- - Optional voice profiles generated from one to five writing samples.
37
- - Human-readable terminal output and pure JSON output.
38
- - Standard exit codes for agents, scripts, and CI jobs.
39
-
40
- It does not estimate perplexity, classify who wrote the text, rewrite prose automatically, scan directories, or provide SARIF output.
41
-
42
- ## Terminal demo
43
-
44
- Given this input:
45
-
46
- ```text
47
- In today's fast-paced world, teams delve into a robust landscape. Moreover, this pivotal approach fosters seamless collaboration across every project. In conclusion, the solution unlocks valuable insights for everyone.
48
- ```
49
-
50
- Run the analyzer:
51
-
52
- ```sh
53
- humanizer-lint --file draft.md --lang en
54
- ```
55
-
56
- The current CLI reports:
57
-
58
- ```text
59
- Status: FAIL
60
- Score: 39
61
- Language: en
62
-
63
- Metrics:
64
- wordCount: 29
65
- sentenceCount: 3
66
- meanSentenceLength: 9.6667
67
- stdDev: 0.4714
68
- burstinessCV: 0.0488
69
- adjacentDelta: 0.5
70
- shortSentenceRatio: 0
71
- longSentenceRatio: 0
72
- mattr: 1
73
- compressionRatio: 0.6972
74
- nominalizationRatio: 0
75
- passiveRatio: 0
76
- voiceProfileDeviation: 0.5319
77
-
78
- Violations:
79
- SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the exact action or subject instead.
80
- SLOP_PHRASE | Paragraph 1, Sentence 2 | Describe the actual consequence without inflated significance.
81
- SLOP_PHRASE | Paragraph 1, Sentence 2 | Name the concrete behavior or outcome.
82
- SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the field or conditions directly.
83
- SLOP_PHRASE | Paragraph 1, Sentence 3 | Name what becomes possible and how.
84
- SLOP_PHRASE | Paragraph 1, Sentence 2 | Connect the sentences by their actual relationship.
85
- SLOP_PHRASE | Paragraph 1, Sentence 3 | End on the final concrete point.
86
- SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the current condition precisely.
87
- SLOP_PHRASE | Paragraph 1, Sentence 2 | Describe the missing step, delay, or transition precisely.
88
- SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the failure mode or capacity the system handles.
89
- SLOP_PHRASE | Paragraph 1, Sentence 3 | State the finding itself.
90
- MONOTONOUS_RHYTHM | Paragraph 1, Sentences 1-3 | Sentence lengths are uniform (10, 10, 9 words). Split one sentence into a short statement or merge two into a longer clause.
91
- LOW_BURSTINESS | Whole text | Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it.
92
- LOW_ADJACENT_DELTA | Whole text | Change neighboring sentence lengths by splitting or merging only where the content allows it.
93
- ```
94
-
95
- A failed analysis exits with code `1`. A passing fixture returns:
96
-
97
- ```text
98
- Status: PASS
99
- Score: 100
100
- Language: en
101
- ```
102
-
103
- and exits with code `0`.
104
-
105
- ## Why humanizer-lint
106
-
107
- Many AI humanizer services send text to another model and return a rewritten version. That can alter facts, quotations, formatting, or technical meaning. `humanizer-lint` takes the smaller and more predictable approach: it identifies the prose that needs attention and leaves the edit under your control.
108
-
109
- | Capability | humanizer-lint | Web rewriting service | General grammar checker |
110
- |---|---|---|---|
111
- | Detects configured LLM clichés | Yes, with rule IDs and locations | Usually rewrites them | Usually not |
112
- | Measures sentence rhythm | Yes | Usually not exposed | Limited |
113
- | Preserves source text | Yes, analysis only | No, returns a rewrite | Depends on product |
114
- | Handles common Markdown structures | Yes, through deterministic sanitization | Product-dependent | Product-dependent |
115
- | Runs offline after installation | Yes | Usually no | Product-dependent |
116
- | Supports Indonesian and English lexical rules | Yes | Product-dependent | Product-dependent |
117
- | Produces JSON | Yes | Product-dependent | Product-dependent |
118
- | Proves human authorship | No | No | No |
119
-
120
- If you are looking for an AI writing detector linter, a ChatGPT tone remover CLI, or an open-source alternative to AI humanizers, note the boundary: this tool detects configured style signals and rhythm problems. It does not claim to determine whether a human or model wrote the text.
121
-
122
- ## Installation
123
-
124
- ### Install as an agent skill
125
-
126
- The recommended installation for coding agents is through the skills CLI:
127
-
128
- ```sh
129
- npx skills add yanstd888/humanizer-lint --skill humanizer-lint
130
- ```
131
-
132
- The public listing, installation command, and install count are available on [skills.sh](https://skills.sh/yanstd888/humanizer-lint).
133
-
134
- ### Run the CLI from source
135
-
136
- The CLI package is not currently published to npm. Clone the repository and either run the scripts directly or use `npm link` to expose the two local binaries:
137
-
138
- ```sh
139
- git clone https://github.com/yanstd888/humanizer-lint.git
140
- cd humanizer-lint
141
- npm test
142
- npm link
143
- humanizer-lint --help
144
- humanizer-fingerprint --help
145
- ```
146
-
147
- No dependency installation or build step is required.
148
-
149
- ## Quick start
150
-
151
- Analyze one file:
152
-
153
- ```sh
154
- humanizer-lint --file draft.md --lang auto
155
- ```
156
-
157
- Return machine-readable JSON:
158
-
159
- ```sh
160
- humanizer-lint --file draft.md --lang id --json
161
- ```
162
-
163
- Analyze stdin without creating a temporary file:
164
-
165
- ```sh
166
- cat draft.md | humanizer-lint --stdin --lang auto --json
167
- ```
168
-
169
- Use a custom voice profile:
170
-
171
- ```sh
172
- humanizer-lint --file draft.md --lang en --profile profiles/custom-voice.json --json
173
- ```
174
-
175
- Without `npm link`, replace `humanizer-lint` with `node scripts/analyze.mjs` and replace `humanizer-fingerprint` with `node scripts/fingerprint.mjs`.
176
-
177
- ## CLI reference
178
-
179
- ### Analyzer
180
-
181
- ```text
182
- humanizer-lint (--file <path> | --stdin) [--lang id|en|auto] [--profile <path>] [--json]
183
- ```
184
-
185
- | Option | Required | Description |
186
- |---|---:|---|
187
- | `--file <path>` | One input required | Read one UTF-8 text or Markdown file. |
188
- | `--stdin` | One input required | Read the complete document from standard input. |
189
- | `--lang id|en|auto` | No | Select Indonesian, English, or function-word auto-detection. Default: `auto`. |
190
- | `--profile <path>` | No | Load a JSON voice profile. |
191
- | `--json` | No | Print JSON only. Without it, print the terminal report. |
192
- | `--help` | No | Print command usage. |
193
-
194
- Exactly one of `--file` or `--stdin` is required.
195
-
196
- ### Voice fingerprint
197
-
198
- ```text
199
- humanizer-fingerprint --input <file-or-dir> --output <profile.json> [--name <name>] [--lang id|en|auto]
200
- ```
201
-
202
- | Option | Required | Description |
203
- |---|---:|---|
204
- | `--input <path>` | Yes | Read one file or up to five `.md` and `.txt` files from one directory. |
205
- | `--output <path>` | Yes | Write the generated profile atomically. |
206
- | `--name <name>` | No | Set the profile name. Default: `custom-voice`. |
207
- | `--lang id|en|auto` | No | Record the intended language. Default: `auto`. |
208
- | `--help` | No | Print command usage. |
209
-
210
- ### Exit codes
211
-
212
- | Code | Meaning |
213
- |---:|---|
214
- | `0` | `PASS`, or `SKIP` for text too short for statistical scoring. |
215
- | `1` | Analysis completed and returned `FAIL`. |
216
- | `2` | Invalid arguments, unreadable input, or malformed profile. |
217
-
218
- ## Detectors and violation types
219
-
220
- The JSON rule catalogs contain stable IDs. Runtime statistical checks use named violation types.
221
-
222
- | Violation type or rule range | Language | What it detects |
223
- |---|---|---|
224
- | `ID001`-`ID040`, `EN001`-`EN040` / `SLOP_PHRASE` | Indonesian and English | Configured clichés, generic transitions, inflated wording, and repeated LLM vocabulary such as `delve`, `tapestry`, `testament`, and their Indonesian equivalents. |
225
- | `ID041`-`ID042` / `DEFINITIONAL_OPENING` | Indonesian | Formulaic paragraph openings and rigid definition templates. |
226
- | `EN041`-`EN044` / `STRUCTURAL_CLICHE` | English | Participial tails, binary formulas, repeated double-hyphens, and similar structural templates. |
227
- | `MONOTONOUS_RHYTHM` | Both | Three adjacent sentences whose lengths differ by at most three words. |
228
- | `LOW_BURSTINESS` | Both | Sentence-length coefficient of variation below the configured threshold. |
229
- | `MISSING_SENTENCE_RANGE` | Both | Text with at least eight sentences that lacks enough short or long sentences. |
230
- | `LOW_ADJACENT_DELTA` | Both | Consecutive sentence lengths that change too little. |
231
- | `LOW_LEXICAL_DIVERSITY` | Both | Adaptive MATTR below the language threshold. |
232
- | `LOW_ENTROPY_REPETITION` | Both | Low DEFLATE compression ratio in text of at least 150 words. |
233
- | `SYMMETRIC_PARAGRAPHS` | Both | Three consecutive paragraphs with the same sentence count. |
234
- | `REPETITIVE_STARTERS` | Both | A paragraph with at least three sentences beginning with the same token. |
235
- | `EXCESSIVE_NOMINALIZATION` | Indonesian | Nominalization density above the configured threshold after whitelist filtering. |
236
- | `PASSIVE_BLOATING` | Indonesian | Passive morphology density or consecutive translation-style subordinate openings. |
237
- | `VOICE_PROFILE_DEVIATION` | Both | Custom-profile mean sentence length that differs from the target by more than 30 percent. |
238
-
239
- ## Metrics and scoring
240
-
241
- | Metric | Definition |
242
- |---|---|
243
- | `burstinessCV` | Population standard deviation divided by mean sentence length. |
244
- | `adjacentDelta` | Mean absolute difference between neighboring sentence lengths. |
245
- | `mattr` | Moving-average type-token ratio over an adaptive 10- to 50-token window. |
246
- | `compressionRatio` | Raw DEFLATE byte length divided by original UTF-8 byte length. |
247
- | `shortSentenceRatio` | Share of sentences containing six words or fewer. |
248
- | `longSentenceRatio` | Share of sentences containing 26 words or more. |
249
- | `voiceProfileDeviation` | Mean relative deviation from available profile targets. It is not an authorship probability. |
250
-
251
- The score starts at 100 and applies these caps:
252
-
253
- | Category | Maximum deduction |
254
- |---|---:|
255
- | Burstiness and sentence-range checks | 25 |
256
- | Monotonous rhythm windows | 20 |
257
- | Lexical and structural clichés | 30 |
258
- | Indonesian morphology | 15 |
259
- | Vocabulary, entropy, and custom-profile deviation combined | 10 |
260
-
261
- A normal-length document passes only when the score is at least 80, no `SLOP_PHRASE` or `DEFINITIONAL_OPENING` remains, burstiness and adjacent-delta gates pass, and the sentence-range requirement passes when applicable.
262
-
263
- Documents below 20 words or three sentences return `SKIP`. Lexical slop phrases are still reported.
264
-
265
- ## Markdown sanitization
266
-
267
- Before tokenization, the analyzer:
268
-
269
- - removes YAML frontmatter, fenced code blocks, headings, and Markdown tables;
270
- - keeps Markdown link anchor text while removing the URL;
271
- - replaces inline code and bare URLs with `TOKEN_ENTITY`;
272
- - removes short or unfinished list fragments while keeping complete list prose;
273
- - removes blockquote markers while preserving the quoted prose;
274
- - preserves paragraph boundaries for locations and structural checks.
275
-
276
- This is deterministic sanitization, not a full Markdown or MDX abstract syntax tree parser. HTML blocks and arbitrary MDX components do not receive special parsing.
277
-
278
- ## Voice profiles
279
-
280
- Generate a profile from one to five samples:
281
-
282
- ```sh
283
- humanizer-fingerprint \
284
- --input writing-samples \
285
- --output profiles/custom-voice.json \
286
- --name editorial-voice \
287
- --lang en
288
- ```
289
-
290
- The profile records mean sentence length, standard deviation, burstiness, adjacent delta, MATTR, average paragraph sentence count, and punctuation rates. Supplying it with `--profile` adds `voiceProfileDeviation`; a mean sentence-length drift above 30 percent creates a five-point profile penalty inside the 10-point vocabulary and entropy cap.
291
-
292
- ## JSON output
293
-
294
- `--json` writes a single JSON object to stdout. The stable top-level fields are:
295
-
296
- ```json
297
- {
298
- "status": "FAIL",
299
- "score": 39,
300
- "language": "en",
301
- "metrics": {
302
- "wordCount": 29,
303
- "sentenceCount": 3,
304
- "burstinessCV": 0.0488,
305
- "adjacentDelta": 0.5,
306
- "mattr": 1
307
- },
308
- "violations": [
309
- {
310
- "type": "LOW_BURSTINESS",
311
- "location": "Whole text",
312
- "actual": 0.0488,
313
- "target": 0.4,
314
- "fixInstruction": "Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it."
315
- }
316
- ]
317
- }
318
- ```
319
-
320
- The real result may contain additional metric properties and violations.
321
-
322
- ## Use as an agent skill
323
-
324
- After installation, an agent can use the skill as a closed verification loop:
325
-
326
- 1. Draft or edit Indonesian or English prose without changing supported facts.
327
- 2. Run the analyzer against the exact complete draft.
328
- 3. Read every violation and edit only the affected prose or named whole-document gate.
329
- 4. Re-run the analyzer, stopping after three iterations.
330
- 5. Return only a `PASS`, or an honestly labeled short-text `SKIP` with no unresolved lexical violations.
331
-
332
- The skill is indexed on skills.sh and its current listing passes the platform's Snyk security audit.
333
-
334
- ## Used by
335
-
336
- The skills.sh badge at the top of this README is the public source for anonymous installation activity. GitHub's native `Used by` sidebar is generated automatically from its dependency graph; repository owners cannot set that count manually.
337
-
338
- Because the CLI is not published to npm, skill installations are represented by skills.sh rather than GitHub package dependents. The GitHub panel will appear when downstream repositories declare a dependency that GitHub can detect.
339
-
340
- ## CI and repository automation
341
-
342
- This repository keeps `.github` intentionally small. The included `.github/workflows/ci.yml` workflow runs the test suite and package dry-run on supported Node.js versions. A CI badge is useful because it reflects an actual workflow; issue templates, Dependabot, and release automation are omitted until the repository needs them.
343
-
344
- To use the analyzer in another GitHub Actions workflow without an npm release, check out this repository into a subdirectory and run the script directly:
345
-
346
- ```yaml
347
- name: prose-check
348
-
349
- on:
350
- pull_request:
351
- paths:
352
- - "**/*.md"
353
- - "**/*.txt"
354
-
355
- jobs:
356
- lint-readme:
357
- runs-on: ubuntu-latest
358
- steps:
359
- - uses: actions/checkout@v4
360
- - uses: actions/checkout@v4
361
- with:
362
- repository: yanstd888/humanizer-lint
363
- path: .humanizer-lint
364
- - uses: actions/setup-node@v4
365
- with:
366
- node-version: "22"
367
- - run: node .humanizer-lint/scripts/analyze.mjs --file README.md --lang auto --json
368
- ```
369
-
370
- The analyzer accepts one file per invocation. Invoke it once for each document you want to gate.
371
-
372
- ## Security properties
373
-
374
- Production scripts use only Node.js built-ins and have no runtime dependencies. The test suite rejects production use of `child_process`, `eval`, dynamic `Function` construction, and `fetch`. The fingerprint writer writes to a temporary file and renames it into place.
375
-
376
- Text remains local unless the surrounding agent or workflow sends it elsewhere.
377
-
378
- ## Frequently asked questions
379
-
380
- <details>
381
- <summary><strong>Is humanizer-lint an AI content detector?</strong></summary>
382
-
383
- No. It detects configured lexical, structural, rhythm, and morphology signals. A pass or failure does not prove who wrote the text.
384
-
385
- </details>
386
-
387
- <details>
388
- <summary><strong>Does humanizer-lint rewrite or auto-fix prose?</strong></summary>
389
-
390
- No. It returns correction instructions but does not modify the source file. This prevents silent factual or formatting changes.
391
-
392
- </details>
393
-
394
- <details>
395
- <summary><strong>Can it remove robotic AI phrasing from Markdown?</strong></summary>
396
-
397
- It can identify the exact paragraph or sentence that matches a configured phrase or style gate. A human or coding agent then makes the smallest supported edit and runs the analyzer again.
398
-
399
- </details>
400
-
401
- <details>
402
- <summary><strong>Can I install it from npm?</strong></summary>
403
-
404
- Not currently. Install the agent skill with `npx skills add yanstd888/humanizer-lint --skill humanizer-lint`, or clone the repository and use `npm link` for the local CLI binaries.
405
-
406
- </details>
407
-
408
- <details>
409
- <summary><strong>Which languages are supported?</strong></summary>
410
-
411
- The configured lexical and language-detection paths support Indonesian and English. Do not treat the current statistical thresholds as validated defaults for other languages.
412
-
413
- </details>
414
-
415
- <details>
416
- <summary><strong>Does it send text to an API?</strong></summary>
417
-
418
- No. Production code has no network call. The skills installer and Git operations need network access only to download the repository.
419
-
420
- </details>
421
-
422
- <details>
423
- <summary><strong>Why did a short document return SKIP?</strong></summary>
424
-
425
- Documents below 20 words or three sentences do not contain enough material for the configured statistical gates. Lexical slop phrases are still checked.
426
-
427
- </details>
428
-
429
- ## Contributing
430
-
431
- Contributions should remain deterministic, zero-dependency, and covered by a focused regression test.
432
-
433
- 1. Fork the repository.
434
- 2. Create a focused branch.
435
- 3. Add or update one minimal test for non-trivial behavior.
436
- 4. Run `npm test` and `npm pack --dry-run`.
437
- 5. Open a pull request that explains the user-visible effect.
438
-
439
- Every repository change updates the package and skill version, then publishes a matching Git tag and GitHub release after verification.
440
-
441
- ## Project layout
442
-
443
- ```text
444
- .github/
445
- workflows/
446
- ci.yml
447
- LICENSE
448
- README.md
449
- SKILL.md
450
- package.json
451
- profiles/
452
- default-human.json
453
- rules/
454
- en-slop.json
455
- id-slop.json
456
- scripts/
457
- analyze.mjs
458
- fingerprint.mjs
459
- utils/
460
- metrics.mjs
461
- sanitizer.mjs
462
- tokenizer.mjs
463
- tests/
464
- fixtures/
465
- test-runner.mjs
466
- ```
467
-
468
- ## License
469
-
470
- Licensed under the [MIT License](LICENSE).
1
+ <div align="center">
2
+
3
+ # humanizer-lint
4
+
5
+ ### Deterministic prose and Markdown style linting for Indonesian and English
6
+
7
+ Detect repetitive LLM phrasing, uniform sentence rhythm, structural clichés, and voice-profile drift without sending text to an external service.
8
+
9
+ [![GitHub Release](https://img.shields.io/github/v/release/yanstd888/humanizer-lint?style=flat-square&logo=github)](https://github.com/yanstd888/humanizer-lint/releases)
10
+ [![npm](https://img.shields.io/npm/v/humanizer-lint)](https://www.npmjs.com/package/humanizer-lint)
11
+ [![CI](https://img.shields.io/github/actions/workflow/status/yanstd888/humanizer-lint/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/yanstd888/humanizer-lint/actions/workflows/ci.yml)
12
+ [![License](https://img.shields.io/github/license/yanstd888/humanizer-lint?style=flat-square)](LICENSE)
13
+ [![Node.js](https://img.shields.io/badge/Node.js-%3E%3D18.3-339933?style=flat-square&logo=nodedotjs&logoColor=white)](https://nodejs.org/)
14
+ [![skills.sh](https://skills.sh/b/yanstd888/humanizer-lint)](https://skills.sh/yanstd888/humanizer-lint)
15
+ [![GitHub Stars](https://img.shields.io/github/stars/yanstd888/humanizer-lint?style=flat-square)](https://github.com/yanstd888/humanizer-lint/stargazers)
16
+
17
+ [What it does](#what-it-does) | [Install](#install) | [Quick start](#quick-start) | [CLI reference](#cli-reference) | [Detectors](#detectors-and-violation-types) | [Agent skill](#use-as-an-agent-skill) | [FAQ](#frequently-asked-questions)
18
+
19
+ </div>
20
+
21
+ **humanizer-lint is an open-source natural-text and Markdown prose linter that deterministically identifies style patterns often associated with AI-generated writing in Indonesian and English; it is a style validator, not an authorship detector.**
22
+
23
+ Try the static browser demo: [humanizer-lint.pages.dev](https://humanizer-lint.pages.dev).
24
+
25
+ It works locally with zero runtime dependencies. Instead of rewriting text through another language model, it reports paragraph- and sentence-level violations with specific correction instructions. Use it to lint LLM-generated documentation, remove robotic AI phrasing from Markdown, check burstiness and lexical diversity, or add a deterministic prose gate to an agent workflow.
26
+
27
+ It can serve as a CLI text humanizer guardrail, automated writing style linter, or CI/CD documentation linter. The bundled rules flag overused AI clichés such as `delve` and `tapestry` without rewriting the source.
28
+
29
+ ## What it does
30
+
31
+ `humanizer-lint` analyzes prose after removing or neutralizing common non-prose Markdown structures. It then combines lexical rules, sentence-length statistics, structural checks, Indonesian morphology checks, and an optional voice profile into a score from 0 to 100.
32
+
33
+ The project includes:
34
+
35
+ - 86 reviewable lexical and structural regex rules: 42 Indonesian and 44 English.
36
+ - Sentence-length burstiness, adjacent rhythm delta, short/long sentence ratios, and adaptive MATTR.
37
+ - Repetition checks based on paragraph symmetry, repeated sentence starters, and compression ratio.
38
+ - Indonesian nominalization, passive-voice, loanword, and translation-style opening checks.
39
+ - Optional voice profiles generated from one to five writing samples.
40
+ - Human-readable terminal output and pure JSON output.
41
+ - Standard exit codes for agents, scripts, and CI jobs.
42
+
43
+ It does not estimate perplexity, classify who wrote the text, rewrite prose automatically, scan directories, or provide SARIF output.
44
+
45
+ ## Terminal demo
46
+
47
+ Given this input:
48
+
49
+ ```text
50
+ In today's fast-paced world, teams delve into a robust landscape. Moreover, this pivotal approach fosters seamless collaboration across every project. In conclusion, the solution unlocks valuable insights for everyone.
51
+ ```
52
+
53
+ Run the analyzer:
54
+
55
+ ```sh
56
+ humanizer-lint --file draft.md --lang en
57
+ ```
58
+
59
+ The current CLI reports:
60
+
61
+ ```text
62
+ Status: FAIL
63
+ Score: 39
64
+ Language: en
65
+
66
+ Metrics:
67
+ wordCount: 29
68
+ sentenceCount: 3
69
+ meanSentenceLength: 9.6667
70
+ stdDev: 0.4714
71
+ burstinessCV: 0.0488
72
+ adjacentDelta: 0.5
73
+ shortSentenceRatio: 0
74
+ longSentenceRatio: 0
75
+ mattr: 1
76
+ compressionRatio: 0.6972
77
+ nominalizationRatio: 0
78
+ passiveRatio: 0
79
+ voiceProfileDeviation: 0.5319
80
+
81
+ Violations:
82
+ SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the exact action or subject instead.
83
+ SLOP_PHRASE | Paragraph 1, Sentence 2 | Describe the actual consequence without inflated significance.
84
+ SLOP_PHRASE | Paragraph 1, Sentence 2 | Name the concrete behavior or outcome.
85
+ SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the field or conditions directly.
86
+ SLOP_PHRASE | Paragraph 1, Sentence 3 | Name what becomes possible and how.
87
+ SLOP_PHRASE | Paragraph 1, Sentence 2 | Connect the sentences by their actual relationship.
88
+ SLOP_PHRASE | Paragraph 1, Sentence 3 | End on the final concrete point.
89
+ SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the current condition precisely.
90
+ SLOP_PHRASE | Paragraph 1, Sentence 2 | Describe the missing step, delay, or transition precisely.
91
+ SLOP_PHRASE | Paragraph 1, Sentence 1 | Name the failure mode or capacity the system handles.
92
+ SLOP_PHRASE | Paragraph 1, Sentence 3 | State the finding itself.
93
+ MONOTONOUS_RHYTHM | Paragraph 1, Sentences 1-3 | Sentence lengths are uniform (10, 10, 9 words). Split one sentence into a short statement or merge two into a longer clause.
94
+ LOW_BURSTINESS | Whole text | Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it.
95
+ LOW_ADJACENT_DELTA | Whole text | Change neighboring sentence lengths by splitting or merging only where the content allows it.
96
+ ```
97
+
98
+ A failed analysis exits with code `1`. A passing fixture returns:
99
+
100
+ ```text
101
+ Status: PASS
102
+ Score: 100
103
+ Language: en
104
+ ```
105
+
106
+ and exits with code `0`.
107
+
108
+ ## Why humanizer-lint
109
+
110
+ Many AI humanizer services send text to another model and return a rewritten version. That can alter facts, quotations, formatting, or technical meaning. `humanizer-lint` takes the smaller and more predictable approach: it identifies the prose that needs attention and leaves the edit under your control.
111
+
112
+ | Capability | humanizer-lint | Web rewriting service | General grammar checker |
113
+ |---|---|---|---|
114
+ | Detects configured LLM clichés | Yes, with rule IDs and locations | Usually rewrites them | Usually not |
115
+ | Measures sentence rhythm | Yes | Usually not exposed | Limited |
116
+ | Preserves source text | Yes, analysis only | No, returns a rewrite | Depends on product |
117
+ | Handles common Markdown structures | Yes, through deterministic sanitization | Product-dependent | Product-dependent |
118
+ | Runs offline after installation | Yes | Usually no | Product-dependent |
119
+ | Supports Indonesian and English lexical rules | Yes | Product-dependent | Product-dependent |
120
+ | Produces JSON | Yes | Product-dependent | Product-dependent |
121
+ | Proves human authorship | No | No | No |
122
+
123
+ If you are looking for an AI writing detector linter, a ChatGPT tone remover CLI, or an open-source alternative to AI humanizers, note the boundary: this tool detects configured style signals and rhythm problems. It does not claim to determine whether a human or model wrote the text.
124
+
125
+ ## Install
126
+
127
+ ### CLI
128
+
129
+ Run the published CLI without a global install:
130
+
131
+ ```sh
132
+ npx humanizer-lint --file tulisan.md --lang id
133
+ ```
134
+
135
+ Or install it globally:
136
+
137
+ ```sh
138
+ npm install -g humanizer-lint
139
+ ```
140
+
141
+ ### Install as an agent skill
142
+
143
+ The recommended installation for coding agents is through the skills CLI:
144
+
145
+ ```sh
146
+ npx skills add yanstd888/humanizer-lint --skill humanizer-lint
147
+ ```
148
+
149
+ The public listing, installation command, and install count are available on [skills.sh](https://skills.sh/yanstd888/humanizer-lint).
150
+
151
+ ### Development (from source)
152
+
153
+ Clone the repository and use `npm link` only when developing the local source:
154
+
155
+ ```sh
156
+ git clone https://github.com/yanstd888/humanizer-lint.git
157
+ cd humanizer-lint
158
+ npm test
159
+ npm link
160
+ humanizer-lint --help
161
+ humanizer-fingerprint --help
162
+ ```
163
+
164
+ No dependency installation or build step is required.
165
+
166
+ ## Quick start
167
+
168
+ Analyze one file:
169
+
170
+ ```sh
171
+ humanizer-lint --file draft.md --lang auto
172
+ ```
173
+
174
+ Return machine-readable JSON:
175
+
176
+ ```sh
177
+ humanizer-lint --file draft.md --lang id --json
178
+ ```
179
+
180
+ Analyze stdin without creating a temporary file:
181
+
182
+ ```sh
183
+ cat draft.md | humanizer-lint --stdin --lang auto --json
184
+ ```
185
+
186
+ Use a custom voice profile:
187
+
188
+ ```sh
189
+ humanizer-lint --file draft.md --lang en --profile profiles/custom-voice.json --json
190
+ ```
191
+
192
+ Without `npm link`, replace `humanizer-lint` with `node scripts/analyze.mjs` and replace `humanizer-fingerprint` with `node scripts/fingerprint.mjs`.
193
+
194
+ ## Contoh / Example
195
+
196
+ Skor mengukur kecocokan dengan sinyal gaya yang diperiksa oleh linter: skor lebih tinggi berarti lebih sedikit pelanggaran yang ditemukan. Ini bukan skor probabilitas penulis AI atau manusia.
197
+
198
+ ### Before
199
+
200
+ ```text
201
+ Di era digital saat ini, penggunaan teknologi menjadi sangat penting bagi setiap bisnis. Tidak hanya itu, solusi ini juga memberikan manfaat yang sangat besar bagi semua pihak. Dengan demikian, organisasi dapat meningkatkan efisiensi secara signifikan dalam berbagai aspek. Pada akhirnya, pendekatan ini membuka peluang baru yang lebih optimal bagi masa depan.
202
+ ```
203
+
204
+ Output dari `tests/fixtures/ai-before-id.md`:
205
+
206
+ ```text
207
+ Status: FAIL
208
+ Score: 38
209
+ Language: id
210
+
211
+ Violations:
212
+ SLOP_PHRASE | Paragraph 1, Sentence 1 | Mulai langsung dari subjek atau fakta konkret.
213
+ SLOP_PHRASE | Paragraph 1, Sentence 3 | Sebutkan aspek yang relevan.
214
+ SLOP_PHRASE | Paragraph 1, Sentence 4 | Tutup langsung dengan akibat atau keputusan akhir.
215
+ MONOTONOUS_RHYTHM | Paragraph 1, Sentences 1-3 | Sentence lengths are uniform (13, 14, 11 words).
216
+ LOW_BURSTINESS | Whole text | Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it.
217
+ LOW_ADJACENT_DELTA | Whole text | Change neighboring sentence lengths by splitting or merging only where the content allows it.
218
+ ```
219
+
220
+ ### After
221
+
222
+ ```text
223
+ Pemilik toko memakai aplikasi stok untuk melihat barang yang habis pada hari yang sama. Semua melihat angka yang sama. Saat satu produk menipis, petugas memesan ulang sebelum rak kosong dan mencatat alasan setiap perubahan. Laporan harian menunjukkan barang mana yang laku, barang mana yang tertahan, dan kapan petugas harus menghubungi pemasok lagi agar toko tidak kehabisan barang yang dicari pelanggan. Pemilik toko kini membeli barang berdasarkan catatan, bukan tebakan.
224
+ ```
225
+
226
+ Output dari `tests/fixtures/human-after-id.md`:
227
+
228
+ ```text
229
+ Status: PASS
230
+ Score: 100
231
+ Language: id
232
+ ```
233
+
234
+ Untuk contoh ini, skor naik dari **38** menjadi **100** setelah frasa template dihilangkan, subjek dibuat konkret, dan panjang kalimat divariasikan.
235
+
236
+ ## CLI reference
237
+
238
+ ### Analyzer
239
+
240
+ ```text
241
+ humanizer-lint (--file <path> | --stdin) [--lang id|en|auto] [--profile <path>] [--json]
242
+ ```
243
+
244
+ | Option | Required | Description |
245
+ |---|---:|---|
246
+ | `--file <path>` | One input required | Read one UTF-8 text or Markdown file. |
247
+ | `--stdin` | One input required | Read the complete document from standard input. |
248
+ | `--lang id|en|auto` | No | Select Indonesian, English, or function-word auto-detection. Default: `auto`. |
249
+ | `--profile <path>` | No | Load a JSON voice profile. |
250
+ | `--json` | No | Print JSON only. Without it, print the terminal report. |
251
+ | `--help` | No | Print command usage. |
252
+
253
+ Exactly one of `--file` or `--stdin` is required.
254
+
255
+ ### Voice fingerprint
256
+
257
+ ```text
258
+ humanizer-fingerprint --input <file-or-dir> --output <profile.json> [--name <name>] [--lang id|en|auto]
259
+ ```
260
+
261
+ | Option | Required | Description |
262
+ |---|---:|---|
263
+ | `--input <path>` | Yes | Read one file or up to five `.md` and `.txt` files from one directory. |
264
+ | `--output <path>` | Yes | Write the generated profile atomically. |
265
+ | `--name <name>` | No | Set the profile name. Default: `custom-voice`. |
266
+ | `--lang id|en|auto` | No | Record the intended language. Default: `auto`. |
267
+ | `--help` | No | Print command usage. |
268
+
269
+ ### Exit codes
270
+
271
+ | Code | Meaning |
272
+ |---:|---|
273
+ | `0` | `PASS`, or `SKIP` for text too short for statistical scoring. |
274
+ | `1` | Analysis completed and returned `FAIL`. |
275
+ | `2` | Invalid arguments, unreadable input, or malformed profile. |
276
+
277
+ ## Detectors and violation types
278
+
279
+ The JSON rule catalogs contain stable IDs. Runtime statistical checks use named violation types.
280
+
281
+ | Violation type or rule range | Language | What it detects |
282
+ |---|---|---|
283
+ | `ID001`-`ID040`, `EN001`-`EN040` / `SLOP_PHRASE` | Indonesian and English | Configured clichés, generic transitions, inflated wording, and repeated LLM vocabulary such as `delve`, `tapestry`, `testament`, and their Indonesian equivalents. |
284
+ | `ID041`-`ID042` / `DEFINITIONAL_OPENING` | Indonesian | Formulaic paragraph openings and rigid definition templates. |
285
+ | `EN041`-`EN044` / `STRUCTURAL_CLICHE` | English | Participial tails, binary formulas, repeated double-hyphens, and similar structural templates. |
286
+ | `MONOTONOUS_RHYTHM` | Both | Three adjacent sentences whose lengths differ by at most three words. |
287
+ | `LOW_BURSTINESS` | Both | Sentence-length coefficient of variation below the configured threshold. |
288
+ | `MISSING_SENTENCE_RANGE` | Both | Text with at least eight sentences that lacks enough short or long sentences. |
289
+ | `LOW_ADJACENT_DELTA` | Both | Consecutive sentence lengths that change too little. |
290
+ | `LOW_LEXICAL_DIVERSITY` | Both | Adaptive MATTR below the language threshold. |
291
+ | `LOW_ENTROPY_REPETITION` | Both | Low DEFLATE compression ratio in text of at least 150 words. |
292
+ | `SYMMETRIC_PARAGRAPHS` | Both | Three consecutive paragraphs with the same sentence count. |
293
+ | `REPETITIVE_STARTERS` | Both | A paragraph with at least three sentences beginning with the same token. |
294
+ | `EXCESSIVE_NOMINALIZATION` | Indonesian | Nominalization density above the configured threshold after whitelist filtering. |
295
+ | `PASSIVE_BLOATING` | Indonesian | Passive morphology density or consecutive translation-style subordinate openings. |
296
+ | `VOICE_PROFILE_DEVIATION` | Both | Custom-profile mean sentence length that differs from the target by more than 30 percent. |
297
+
298
+ ## Metrics and scoring
299
+
300
+ | Metric | Definition |
301
+ |---|---|
302
+ | `burstinessCV` | Population standard deviation divided by mean sentence length. |
303
+ | `adjacentDelta` | Mean absolute difference between neighboring sentence lengths. |
304
+ | `mattr` | Moving-average type-token ratio over an adaptive 10- to 50-token window. |
305
+ | `compressionRatio` | Raw DEFLATE byte length divided by original UTF-8 byte length. |
306
+ | `shortSentenceRatio` | Share of sentences containing six words or fewer. |
307
+ | `longSentenceRatio` | Share of sentences containing 26 words or more. |
308
+ | `voiceProfileDeviation` | Mean relative deviation from available profile targets. It is not an authorship probability. |
309
+
310
+ The score starts at 100 and applies these caps:
311
+
312
+ | Category | Maximum deduction |
313
+ |---|---:|
314
+ | Burstiness and sentence-range checks | 25 |
315
+ | Monotonous rhythm windows | 20 |
316
+ | Lexical and structural clichés | 30 |
317
+ | Indonesian morphology | 15 |
318
+ | Vocabulary, entropy, and custom-profile deviation combined | 10 |
319
+
320
+ A normal-length document passes only when the score is at least 80, no `SLOP_PHRASE` or `DEFINITIONAL_OPENING` remains, burstiness and adjacent-delta gates pass, and the sentence-range requirement passes when applicable.
321
+
322
+ Documents below 20 words or three sentences return `SKIP`. Lexical slop phrases are still reported.
323
+
324
+ ## Markdown sanitization
325
+
326
+ Before tokenization, the analyzer:
327
+
328
+ - removes YAML frontmatter, fenced code blocks, headings, and Markdown tables;
329
+ - keeps Markdown link anchor text while removing the URL;
330
+ - replaces inline code and bare URLs with `TOKEN_ENTITY`;
331
+ - removes short or unfinished list fragments while keeping complete list prose;
332
+ - removes blockquote markers while preserving the quoted prose;
333
+ - preserves paragraph boundaries for locations and structural checks.
334
+
335
+ This is deterministic sanitization, not a full Markdown or MDX abstract syntax tree parser. HTML blocks and arbitrary MDX components do not receive special parsing.
336
+
337
+ ## Voice profiles
338
+
339
+ Generate a profile from one to five samples:
340
+
341
+ ```sh
342
+ humanizer-fingerprint \
343
+ --input writing-samples \
344
+ --output profiles/custom-voice.json \
345
+ --name editorial-voice \
346
+ --lang en
347
+ ```
348
+
349
+ The profile records mean sentence length, standard deviation, burstiness, adjacent delta, MATTR, average paragraph sentence count, and punctuation rates. Supplying it with `--profile` adds `voiceProfileDeviation`; a mean sentence-length drift above 30 percent creates a five-point profile penalty inside the 10-point vocabulary and entropy cap.
350
+
351
+ ## JSON output
352
+
353
+ `--json` writes a single JSON object to stdout. The stable top-level fields are:
354
+
355
+ ```json
356
+ {
357
+ "status": "FAIL",
358
+ "score": 39,
359
+ "language": "en",
360
+ "metrics": {
361
+ "wordCount": 29,
362
+ "sentenceCount": 3,
363
+ "burstinessCV": 0.0488,
364
+ "adjacentDelta": 0.5,
365
+ "mattr": 1
366
+ },
367
+ "violations": [
368
+ {
369
+ "type": "LOW_BURSTINESS",
370
+ "location": "Whole text",
371
+ "actual": 0.0488,
372
+ "target": 0.4,
373
+ "fixInstruction": "Vary sentence lengths by splitting one uniform sentence and merging another where the meaning supports it."
374
+ }
375
+ ]
376
+ }
377
+ ```
378
+
379
+ The real result may contain additional metric properties and violations.
380
+
381
+ ## Use as an agent skill
382
+
383
+ After installation, an agent can use the skill as a closed verification loop:
384
+
385
+ 1. Draft or edit Indonesian or English prose without changing supported facts.
386
+ 2. Run the analyzer against the exact complete draft.
387
+ 3. Read every violation and edit only the affected prose or named whole-document gate.
388
+ 4. Re-run the analyzer, stopping after three iterations.
389
+ 5. Return only a `PASS`, or an honestly labeled short-text `SKIP` with no unresolved lexical violations.
390
+
391
+ The skill is indexed on skills.sh and its current listing passes the platform's Snyk security audit.
392
+
393
+ ## Used by
394
+
395
+ The skills.sh badge at the top of this README is the public source for anonymous installation activity. GitHub's native `Used by` sidebar is generated automatically from its dependency graph; repository owners cannot set that count manually.
396
+
397
+ Because the CLI is not published to npm, skill installations are represented by skills.sh rather than GitHub package dependents. The GitHub panel will appear when downstream repositories declare a dependency that GitHub can detect.
398
+
399
+ ## CI and repository automation
400
+
401
+ This repository keeps `.github` intentionally small. The included `.github/workflows/ci.yml` workflow runs the test suite and package dry-run on supported Node.js versions. A CI badge is useful because it reflects an actual workflow; issue templates, Dependabot, and release automation are omitted until the repository needs them.
402
+
403
+ To use the analyzer in another GitHub Actions workflow without an npm release, check out this repository into a subdirectory and run the script directly:
404
+
405
+ ```yaml
406
+ name: prose-check
407
+
408
+ on:
409
+ pull_request:
410
+ paths:
411
+ - "**/*.md"
412
+ - "**/*.txt"
413
+
414
+ jobs:
415
+ lint-readme:
416
+ runs-on: ubuntu-latest
417
+ steps:
418
+ - uses: actions/checkout@v4
419
+ - uses: actions/checkout@v4
420
+ with:
421
+ repository: yanstd888/humanizer-lint
422
+ path: .humanizer-lint
423
+ - uses: actions/setup-node@v4
424
+ with:
425
+ node-version: "22"
426
+ - run: node .humanizer-lint/scripts/analyze.mjs --file README.md --lang auto --json
427
+ ```
428
+
429
+ The analyzer accepts one file per invocation. Invoke it once for each document you want to gate.
430
+
431
+ ## Security properties
432
+
433
+ Production scripts use only Node.js built-ins and have no runtime dependencies. The test suite rejects production use of `child_process`, `eval`, dynamic `Function` construction, and `fetch`. The fingerprint writer writes to a temporary file and renames it into place.
434
+
435
+ Text remains local unless the surrounding agent or workflow sends it elsewhere.
436
+
437
+ ## Frequently asked questions
438
+
439
+ <details>
440
+ <summary><strong>Is humanizer-lint an AI content detector?</strong></summary>
441
+
442
+ No. It detects configured lexical, structural, rhythm, and morphology signals. A pass or failure does not prove who wrote the text.
443
+
444
+ </details>
445
+
446
+ <details>
447
+ <summary><strong>Does humanizer-lint rewrite or auto-fix prose?</strong></summary>
448
+
449
+ No. It returns correction instructions but does not modify the source file. This prevents silent factual or formatting changes.
450
+
451
+ </details>
452
+
453
+ <details>
454
+ <summary><strong>Can it remove robotic AI phrasing from Markdown?</strong></summary>
455
+
456
+ It can identify the exact paragraph or sentence that matches a configured phrase or style gate. A human or coding agent then makes the smallest supported edit and runs the analyzer again.
457
+
458
+ </details>
459
+
460
+ <details>
461
+ <summary><strong>Can I install it from npm?</strong></summary>
462
+
463
+ Not currently. Install the agent skill with `npx skills add yanstd888/humanizer-lint --skill humanizer-lint`, or clone the repository and use `npm link` for the local CLI binaries.
464
+
465
+ </details>
466
+
467
+ <details>
468
+ <summary><strong>Which languages are supported?</strong></summary>
469
+
470
+ The configured lexical and language-detection paths support Indonesian and English. Do not treat the current statistical thresholds as validated defaults for other languages.
471
+
472
+ </details>
473
+
474
+ <details>
475
+ <summary><strong>Does it send text to an API?</strong></summary>
476
+
477
+ No. Production code has no network call. The skills installer and Git operations need network access only to download the repository.
478
+
479
+ </details>
480
+
481
+ <details>
482
+ <summary><strong>Why did a short document return SKIP?</strong></summary>
483
+
484
+ Documents below 20 words or three sentences do not contain enough material for the configured statistical gates. Lexical slop phrases are still checked.
485
+
486
+ </details>
487
+
488
+ ## Contributing
489
+
490
+ Contributions should remain deterministic, zero-dependency, and covered by a focused regression test.
491
+
492
+ 1. Fork the repository.
493
+ 2. Create a focused branch.
494
+ 3. Add or update one minimal test for non-trivial behavior.
495
+ 4. Run `npm test` and `npm pack --dry-run`.
496
+ 5. Open a pull request that explains the user-visible effect.
497
+
498
+ Every repository change updates the package and skill version, then publishes a matching Git tag and GitHub release after verification.
499
+
500
+ ## Project layout
501
+
502
+ ```text
503
+ .github/
504
+ workflows/
505
+ ci.yml
506
+ LICENSE
507
+ README.md
508
+ SKILL.md
509
+ package.json
510
+ profiles/
511
+ default-human.json
512
+ rules/
513
+ en-slop.json
514
+ id-slop.json
515
+ scripts/
516
+ analyze.mjs
517
+ fingerprint.mjs
518
+ utils/
519
+ metrics.mjs
520
+ sanitizer.mjs
521
+ tokenizer.mjs
522
+ tests/
523
+ fixtures/
524
+ test-runner.mjs
525
+ ```
526
+
527
+ ## License
528
+
529
+ Licensed under the [MIT License](LICENSE).