humanizer-lint 1.3.0 → 1.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -6
- package/SKILL.md +104 -157
- package/demo/samples.mjs +72 -0
- package/package.json +7 -5
- package/scripts/analyze.mjs +9 -4
- package/scripts/benchmark.mjs +68 -0
- package/scripts/core/analyzer-core.mjs +193 -11
- package/scripts/fingerprint.mjs +8 -4
- package/scripts/utils/domain-linters.mjs +95 -0
- package/scripts/utils/metrics-core.mjs +4 -0
package/README.md
CHANGED
|
@@ -4,9 +4,9 @@
|
|
|
4
4
|
|
|
5
5
|
# humanizer-lint
|
|
6
6
|
|
|
7
|
-
###
|
|
7
|
+
### Edit stiff AI-shaped prose with a deterministic local prose and rhythm linter
|
|
8
8
|
|
|
9
|
-
Detect repetitive
|
|
9
|
+
Detect repetitive stock phrasing, uniform sentence rhythm, structural clichés, and voice-profile drift with **8 language-specific lexical and morphological packs plus universal `Intl.Segmenter` rhythm diagnostics for 100+ languages** without sending text to an external service.
|
|
10
10
|
|
|
11
11
|
[](https://github.com/yanstd888/humanizer-lint/releases)
|
|
12
12
|
[](https://www.npmjs.com/package/humanizer-lint)
|
|
@@ -26,10 +26,10 @@ Detect repetitive LLM phrasing, uniform sentence rhythm, structural clichés, an
|
|
|
26
26
|
|
|
27
27
|
**Summary**
|
|
28
28
|
|
|
29
|
-
-
|
|
30
|
-
-
|
|
31
|
-
- Ships
|
|
32
|
-
- Supports
|
|
29
|
+
- Flags repetitive AI stock phrases, flat sentence rhythm, and rigid translation patterns in blog posts, SEO articles, copy, and technical documentation.
|
|
30
|
+
- Reports diagnostic CV, paragraph CV, adjacent rhythm delta, MATTR, n-gram repetition, and one-pass fix hints; these values are indicators, not mechanical style targets.
|
|
31
|
+
- Ships **8 language-specific lexical and morphological packs** (`en`, `id`, `es`, `ms`, `fr`, `de`, `pt`, `zh`) plus universal `Intl.Segmenter` sentence-rhythm analysis for 100+ languages.
|
|
32
|
+
- Supports local author voice profiling via `node <skill_dir>/scripts/fingerprint.mjs` and passes the listed skills.sh security audits with zero network text uploads.
|
|
33
33
|
|
|
34
34
|
```sh
|
|
35
35
|
npx skills add yanstd888/humanizer-lint
|
|
@@ -39,6 +39,16 @@ Use the command above to install the agent skill. For one-off local linting, run
|
|
|
39
39
|
|
|
40
40
|
**8-language summary:** EN — local AI text humanizer and prose linter. ID — humanizer bahasa Indonesia dan cek tulisan AI. ES — humanizador de texto IA y corrector de ritmo. MS — humanizer teks AI dan semak ayat. FR — humaniseur de texte IA et analyseur stylométrique. DE — KI-Text humanisieren und KI-Floskeln entfernen. PT — humanizador de texto IA e corretor de burstiness. ZH — AI文章人性化与句式节奏分析.
|
|
41
41
|
|
|
42
|
+
## Verified benchmark
|
|
43
|
+
|
|
44
|
+
Run `npm run benchmark` to reproduce the current 16-fixture benchmark on the local machine.
|
|
45
|
+
|
|
46
|
+
| Fixtures | Accuracy | Mean AI score | Mean human score | Separation | Mean runtime |
|
|
47
|
+
|---:|---:|---:|---:|---:|---:|
|
|
48
|
+
| 16 (8 languages × AI/Human) | 100% | 26.00 | 100.00 | 74.00 points | ~0.40 ms / analysis |
|
|
49
|
+
|
|
50
|
+
The benchmark measures deterministic fixture separation and runtime. It does not classify authorship.
|
|
51
|
+
|
|
42
52
|
## Try in your browser
|
|
43
53
|
|
|
44
54
|
Use the official demo: [humanizer.aiclub.ai.id](https://humanizer.aiclub.ai.id).
|
package/SKILL.md
CHANGED
|
@@ -1,157 +1,104 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: humanizer-lint
|
|
3
|
-
description: Deterministic
|
|
4
|
-
summary:
|
|
5
|
-
topic: agent-workflows
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
1. Group overlapping violations by paragraph. Whole-text statistical diagnostics (`LOW_BURSTINESS`, `MISSING_SENTENCE_RANGE`, `LOW_ADJACENT_DELTA`, and `LOW_LEXICAL_DIVERSITY`) identify the global gate when no single sentence is solely responsible.
|
|
107
|
-
2. Edit only the cited sentences or paragraphs, or make the smallest whole-text rhythm correction named by a statistical diagnostic.
|
|
108
|
-
3. Follow each `fixInstruction` without adding or dropping facts.
|
|
109
|
-
4. Run the analyzer again on the complete revised draft.
|
|
110
|
-
5. Stop after three analyzer iterations.
|
|
111
|
-
|
|
112
|
-
If `status` is `SKIP`, statistical stylometry is unavailable. Fix any lexical violations that are still reported, preserve the short form, and state that the statistical gate was skipped because the text is too short.
|
|
113
|
-
|
|
114
|
-
If the third iteration still returns `FAIL`, do not present the draft as verified. Return the remaining diagnostics and ask whether the user wants a broader rewrite.
|
|
115
|
-
|
|
116
|
-
Completion criterion: return prose only after `PASS`, or an explicitly labeled short-text `SKIP` with no unresolved lexical violations.
|
|
117
|
-
|
|
118
|
-
## Analyzer commands
|
|
119
|
-
|
|
120
|
-
```sh
|
|
121
|
-
node <skill_dir>/scripts/analyze.mjs --file draft.md --lang id --json
|
|
122
|
-
node <skill_dir>/scripts/analyze.mjs --file draft.md --lang en
|
|
123
|
-
node <skill_dir>/scripts/analyze.mjs --stdin --lang auto --json
|
|
124
|
-
node <skill_dir>/scripts/fingerprint.mjs --input samples --output <skill_dir>/profiles/custom-voice.json
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
Exit codes:
|
|
128
|
-
|
|
129
|
-
- `0`: `PASS` or `SKIP`.
|
|
130
|
-
- `1`: analyzed successfully and returned `FAIL`.
|
|
131
|
-
- `2`: invalid arguments, unreadable input, or malformed profile.
|
|
132
|
-
|
|
133
|
-
## Pitfalls
|
|
134
|
-
|
|
135
|
-
- A high score does not prove human authorship. It proves only that the configured checks passed.
|
|
136
|
-
- Very short text cannot support statistical stylometry.
|
|
137
|
-
- Technical, legal, and intentionally repetitive prose may require a custom profile.
|
|
138
|
-
- Do not rewrite unflagged paragraphs merely to increase the score.
|
|
139
|
-
- Do not treat `voiceProfileDeviation` as an authorship probability; it is a relative distance from the selected profile.
|
|
140
|
-
|
|
141
|
-
## Verification
|
|
142
|
-
|
|
143
|
-
Before returning revised prose:
|
|
144
|
-
|
|
145
|
-
- [ ] Analyzer ran against the final exact text.
|
|
146
|
-
- [ ] Final status is `PASS`, or an honest lexical-clean `SKIP` for short text.
|
|
147
|
-
- [ ] No source fact, identifier, number, quotation, or citation changed unintentionally.
|
|
148
|
-
- [ ] Only flagged prose was revised after the first analyzer pass.
|
|
149
|
-
- [ ] No network service or external package was used.
|
|
150
|
-
|
|
151
|
-
## Multilingual Agent-SEO Keyword Matrix
|
|
152
|
-
|
|
153
|
-
- **Core:** `ai humanizer`, `ai text humanizer`, `prose linter`, `anti-slop skill`, `stylometric analyzer`, `burstiness checker`, `perplexity and burstiness optimizer`, `humanizer-lint`, `skills.sh humanizer`, `markdown prose linter`.
|
|
154
|
-
- **Technical:** `sentence length variation`, `coefficient of variation (CV >= 0.45)`, `adjacent sentence rhythm delta`, `moving-average type-token ratio (MATTR)`, `zlib compression entropy`, `remove LLM clichés and stock phrases`, `fix robotic AI writing tone`, `bypass statistical AI writing patterns`, `author voice fingerprinting`, `deterministic local CLI text validator`.
|
|
155
|
-
- **Workflows:** `writing assistant skill`, `copywriting linter`, `blog post humanizer`, `seo article editor`, `technical documentation linter`, `academic prose polisher`, `translationese remover`, `proofreading and editing workflow`, `claude code writing skill`, `cursor anti-slop rules`, `windsurf prose skill`, `codex writing skill`, `gemini cli humanizer`.
|
|
156
|
-
- **ID / MS:** `humanizer bahasa indonesia`, `cek tulisan ai gratis`, `cara agar tulisan ai tidak terdeteksi`, `ubah tulisan chatgpt jadi natural manusia`, `perbaiki tulisan ai yang kaku`, `hilangkan kata klise ai`, `alat parafrase anti-ai`, `cek skor burstiness tulisan`, `humanizer artikel seo dan skripsi`, `semak ayat ai bahasa melayu`, `humanizer teks bahasa melayu`, `cara elak dikesan ai`.
|
|
157
|
-
- **ES / FR / DE / PT / ZH:** `humanizador de texto ia gratis`, `quitar clichés de chatgpt`, `detector y corrector de ritmo de escritura ia`, `mejorar burstiness de textos`; `humaniseur de texte ia`, `corriger le style robotique chatgpt`, `analyseur stylométrique de prose`, `supprimer le jargon ia`; `ki-text humanisieren kostenlos`, `chatgpt texte menschlicher machen`, `stilometrische textanalyse`, `ki-floskeln entfernen`; `humanizador de texto ia gratuito`, `deixar texto do chatgpt mais humano`, `corretor de estilo e burstiness`, `remover clichês de ia`; `AI文章人性化`, `消除AI写作痕迹`, `去除AI味工具`, `ChatGPT文本润色与句式节奏分析`, `降低AI率`.
|
|
1
|
+
---
|
|
2
|
+
name: humanizer-lint
|
|
3
|
+
description: Deterministic prose linter, AI writing style editor, and stylometric rhythm analyzer for natural copywriting, blog posts, SEO articles, and technical documentation. Provides 8 anti-cliché rule packs (EN, ID, ES, MS, FR, DE, PT, ZH) and universal sentence-rhythm diagnostics for 100+ languages via local Node.js CLI.
|
|
4
|
+
summary: Local deterministic prose, copywriting, and code-comment linter with 8 lexical anti-cliché packs and universal stylometric rhythm analysis across 100+ languages.
|
|
5
|
+
topic: agent-workflows
|
|
6
|
+
allowed-tools: Bash(node:*)
|
|
7
|
+
version: 1.4.1
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Humanizer Lint
|
|
11
|
+
|
|
12
|
+
**Audit prose locally, revise only measured problems, and verify the final text before delivery.**
|
|
13
|
+
|
|
14
|
+
- Run `node <skill_dir>/scripts/analyze.mjs --stdin --json` for deterministic local analysis.
|
|
15
|
+
- Use 8 language-specific lexical and morphological packs (`en`, `id`, `es`, `ms`, `fr`, `de`, `pt`, `zh`) plus universal sentence-rhythm segmentation for 100+ languages through `Intl.Segmenter`.
|
|
16
|
+
- Follow each finding's surgical `suggestedReplacement`, `splitSuggestion`, and `fixInstruction` without rewriting clean prose.
|
|
17
|
+
- Build a local author voice profile with `node <skill_dir>/scripts/fingerprint.mjs`; samples never leave the machine.
|
|
18
|
+
|
|
19
|
+
## Agent Tool Boundary
|
|
20
|
+
|
|
21
|
+
Inside an agent workflow, execute only:
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
node <skill_dir>/scripts/analyze.mjs --stdin --json
|
|
25
|
+
node <skill_dir>/scripts/fingerprint.mjs --dir <samples_dir> --out <skill_dir>/profiles/custom-voice.json
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Do not call network tools or upload the user's text. `npx humanizer-lint` is a manual human CLI convenience outside this agent workflow.
|
|
29
|
+
|
|
30
|
+
## Workflow A — Drafting New Prose
|
|
31
|
+
|
|
32
|
+
1. Write with concrete nouns, active verbs, specific claims, and transitions that reflect the actual relationship between ideas.
|
|
33
|
+
2. Keep facts, quotations, names, numbers, and required terminology unchanged.
|
|
34
|
+
3. Avoid stock openings, staged conversational hooks, generic claims, and repeated sentence templates.
|
|
35
|
+
4. Before returning the draft, pipe it to `node <skill_dir>/scripts/analyze.mjs --stdin --json`.
|
|
36
|
+
5. Revise only actionable findings, then verify again. Stop after at most three analysis/revision passes.
|
|
37
|
+
|
|
38
|
+
## Workflow B — Auditing and Revising Existing Text
|
|
39
|
+
|
|
40
|
+
1. Run the analyzer **before changing a single word**.
|
|
41
|
+
2. If the result is `SKIP`, preserve short or code-heavy input unless a specific lexical or domain finding identifies a real problem.
|
|
42
|
+
3. Edit only flagged sentences or phrases. Prefer `suggestedReplacement` and `splitSuggestion`; otherwise follow `fixInstruction`.
|
|
43
|
+
4. Preserve all unflagged prose and factual meaning.
|
|
44
|
+
5. Re-run the analyzer. Stop after at most three iterations and report any unresolved diagnostic instead of forcing a rewrite.
|
|
45
|
+
|
|
46
|
+
## Metrics Are Diagnostics, Not Rigid Targets
|
|
47
|
+
|
|
48
|
+
`CV`, paragraph `CV`, adjacent sentence delta ($\Delta_{adj}$), sentence-length ranges, MATTR, and compression ratio identify flat or repetitive patterns. They are not mechanical writing targets. Never manufacture a 3-word sentence followed by a 35-word sentence merely to raise a number; the result must still sound coherent.
|
|
49
|
+
|
|
50
|
+
Formal, legal, academic, and regulated documents often use lower sentence variance, nominalized terms, or prescribed wording. Preserve required domain language. When a consistent author voice matters, use a custom `--profile` rather than forcing casual cadence.
|
|
51
|
+
|
|
52
|
+
## Local Voice Profile
|
|
53
|
+
|
|
54
|
+
Place at least three human writing samples in one local folder. A combined total of 500 words or more is recommended. Files must be `.txt` or `.md`.
|
|
55
|
+
|
|
56
|
+
```sh
|
|
57
|
+
node <skill_dir>/scripts/fingerprint.mjs \
|
|
58
|
+
--dir <samples_dir> \
|
|
59
|
+
--out <skill_dir>/profiles/custom-voice.json
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Then analyze with:
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
node <skill_dir>/scripts/analyze.mjs \
|
|
66
|
+
--stdin \
|
|
67
|
+
--profile <skill_dir>/profiles/custom-voice.json \
|
|
68
|
+
--json
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Fingerprinting and analysis run locally with zero network uploads.
|
|
72
|
+
|
|
73
|
+
## Indonesian Revision Examples
|
|
74
|
+
|
|
75
|
+
**Before — FAIL**
|
|
76
|
+
|
|
77
|
+
> Tidak dapat dipungkiri bahwa pengimplementasian sistem ini dilakukan untuk meningkatkan efisiensi proses.
|
|
78
|
+
|
|
79
|
+
**After — PASS**
|
|
80
|
+
|
|
81
|
+
> Tim menerapkan sistem ini untuk mempercepat proses.
|
|
82
|
+
|
|
83
|
+
**Before — FAIL**
|
|
84
|
+
|
|
85
|
+
> Dalam rangka memastikan stabilitas, pelaksanaan pemeriksaan dilakukan oleh tim sebelum deployment.
|
|
86
|
+
|
|
87
|
+
**After — PASS**
|
|
88
|
+
|
|
89
|
+
> Tim memeriksa sistem sebelum deployment untuk memastikan layanan tetap stabil.
|
|
90
|
+
|
|
91
|
+
Mixed Indonesian–English technical prose is supported. Keep necessary terms such as `deployment`, `latency`, `cache`, `middleware`, and `pull request`; Indonesian stopwords and sentence structure determine routing when they dominate the text.
|
|
92
|
+
|
|
93
|
+
## Delivery Gate
|
|
94
|
+
|
|
95
|
+
Use `--delivery-gate` for a human-readable four-block report or `--json` for the same structured data:
|
|
96
|
+
|
|
97
|
+
1. `rhythmAndStylometry` — document CV, paragraph CV, adjacent delta, range, and clause symmetry.
|
|
98
|
+
2. `lexicalAndMorphology` — active rule pack, repeated n-grams, stock phrases, nominalization, and passive bloat.
|
|
99
|
+
3. `domainHygiene` — copy/CTA, code comments, and filename-aware UI/accessibility/responsive checks.
|
|
100
|
+
4. `finalVerdict` — score, `PASS`/`FAIL`/`SKIP`, and prioritized one-pass fixes.
|
|
101
|
+
|
|
102
|
+
## Search Intents & Editorial Keywords
|
|
103
|
+
|
|
104
|
+
`perbaiki tulisan AI yang kaku`; `hilangkan kata klise AI`; `audit ritme kalimat dan burstiness`; `rapikan dokumentasi teknis`; `polish robotic LLM drafts`; `remove repetitive AI stock phrases`; `lint markdown documentation and copywriting`; `check sentence rhythm and lexical repetition`; `消除AI写作腔`; `提升文章可读性`.
|
package/demo/samples.mjs
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
export const DEMO_SAMPLES = {
|
|
2
|
+
'ai-en': {
|
|
3
|
+
language: 'en',
|
|
4
|
+
text: `In today's fast-paced digital landscape, organizations must embrace robust technology to remain competitive. Moreover, modern platforms provide seamless experiences that elevate productivity across every department. Furthermore, these solutions foster collaboration and unlock new opportunities for sustainable growth. It is important to note that successful adoption requires a comprehensive and carefully structured strategy.
|
|
5
|
+
|
|
6
|
+
Technology is a system that enables organizations to improve their operational capabilities. This approach is crucial for leaders who want to navigate an ever-evolving business landscape. Not only does it enhance efficiency but also creates a pivotal foundation for future innovation. By using an integrated platform, teams can improve communication, ensuring the organization achieves its goals.
|
|
7
|
+
|
|
8
|
+
In conclusion, digital transformation represents a game-changer for modern organizations. It is worth noting that every company should develop a robust roadmap before beginning this journey. Moreover, leaders must foster a culture that can unlock the full potential of technology. The result is a seamless framework that can elevate performance and unleash long-term value.`,
|
|
9
|
+
},
|
|
10
|
+
'human-en': {
|
|
11
|
+
language: 'en',
|
|
12
|
+
text: `The build broke at 4:17 p.m.
|
|
13
|
+
|
|
14
|
+
Maya had already packed her bag when the alert arrived, so she reopened the laptop and checked the last deployment. One configuration line pointed to a test bucket that had been deleted months earlier, but the failure appeared only now because a cached credential finally expired.
|
|
15
|
+
|
|
16
|
+
She changed the line. Nothing happened.
|
|
17
|
+
|
|
18
|
+
The worker had kept the old configuration in memory, which meant the clean-looking deploy had never restarted the process that actually handled customer uploads in production. Maya stopped the worker, watched it return with the correct bucket, and sent a small file through the public endpoint. It landed.
|
|
19
|
+
|
|
20
|
+
Before leaving, she added a startup check that rejects missing buckets and wrote down why a restart is required after configuration changes. The patch, test included, was twelve lines and saved the next person from reading forty minutes of misleading logs.
|
|
21
|
+
|
|
22
|
+
Her bag was still beside the door. Dinner was cold, but the queue was empty.`,
|
|
23
|
+
},
|
|
24
|
+
'ai-id': {
|
|
25
|
+
language: 'id',
|
|
26
|
+
text: `Di era digital, perubahan teknologi menjadi bagian penting dalam kehidupan masyarakat. Selain itu, teknologi memberikan berbagai kemudahan bagi banyak orang dalam menjalankan aktivitas harian. Lebih lanjut, perkembangan ini juga mendorong perusahaan untuk melakukan transformasi secara menyeluruh. Namun demikian, setiap organisasi perlu mempertimbangkan berbagai tantangan yang dapat muncul selama proses tersebut.
|
|
27
|
+
|
|
28
|
+
Penting untuk diingat bahwa penerapan teknologi membutuhkan strategi yang matang dan terstruktur. Hal ini menunjukkan bahwa perusahaan harus melakukan pengimplementasian sistem secara hati-hati dan berkelanjutan. Tidak hanya meningkatkan efisiensi tetapi juga membantu organisasi menciptakan pengalaman yang lebih baik. Dalam konteks transformasi digital, teknologi adalah sebuah sistem yang mendukung perkembangan bisnis modern.
|
|
29
|
+
|
|
30
|
+
Secara keseluruhan, transformasi digital memiliki peran yang sangat penting bagi keberhasilan organisasi. Dapat disimpulkan bahwa perusahaan perlu terus beradaptasi agar mampu menghadapi perubahan yang semakin cepat. Selain itu, kolaborasi yang efektif diperlukan untuk memastikan seluruh proses dapat dijalankan dengan baik. Sebagai kesimpulan, pendekatan yang tepat akan membantu perusahaan mencapai hasil yang optimal dan berkelanjutan.`,
|
|
31
|
+
},
|
|
32
|
+
'human-id': {
|
|
33
|
+
language: 'id',
|
|
34
|
+
text: `Pagi itu server mati.
|
|
35
|
+
|
|
36
|
+
Rina sedang menyeduh kopi ketika tiga pesan masuk hampir bersamaan: kasir tidak bisa mencetak struk, gudang melihat stok minus, dan pemilik toko meminta laporan yang seharusnya sudah terkirim sejak subuh. Ia menaruh cangkirnya, membuka log, lalu menemukan satu tanggal kedaluwarsa yang luput diperbarui saat migrasi malam sebelumnya.
|
|
37
|
+
|
|
38
|
+
Masalahnya kecil. Dampaknya tidak.
|
|
39
|
+
|
|
40
|
+
Ia mengganti nilai tersebut dan menjalankan ulang antrean. Kasir kembali bekerja dalam dua menit, tetapi Rina belum menutup laptop karena stok minus masih perlu dilacak sampai sumbernya. Ternyata satu pemindai lama mengirim transaksi dua kali setiap kali koneksinya terputus sesaat.
|
|
41
|
+
|
|
42
|
+
Tidak ada rapat darurat.
|
|
43
|
+
|
|
44
|
+
Tim gudang mematikan pemindai itu, mencatat enam transaksi ganda, lalu memperbaiki jumlah barang berdasarkan nota fisik yang masih tersimpan di meja dekat rak paling belakang gudang. Menjelang makan siang, laporan terkirim dengan catatan singkat tentang koreksi yang dilakukan dan alasan angkanya berubah.
|
|
45
|
+
|
|
46
|
+
Besoknya mereka menambah pemeriksaan sederhana sebelum antrean diproses. Murah, agak membosankan, dan cukup untuk mencegah kejadian yang sama.`,
|
|
47
|
+
},
|
|
48
|
+
'ai-es': { language: 'es', text: `En el vertiginoso mundo actual, cada equipo debe adoptar una estrategia sólida para alcanzar resultados óptimos. En el vertiginoso mundo actual, cada departamento necesita una plataforma robusta para mejorar cada proceso. En el vertiginoso mundo actual, cada líder debe impulsar una transformación integral para crear valor sostenible.
|
|
49
|
+
|
|
50
|
+
Cabe destacar que esta solución desempeña un papel crucial para todas las organizaciones. Además, la plataforma permite una colaboración fluida y una experiencia transformadora para cada usuario. En conclusión, este enfoque abre nuevas oportunidades y garantiza un futuro más eficiente.` },
|
|
51
|
+
'human-es': { language: 'es', text: `Llueve. Marta revisó los recibos del mercado antes de abrir la tienda, comparó tres precios y dejó una nota junto a la caja para el turno de tarde. Al final, el pedido salió sin prisa y nadie tuvo que llamar al proveedor.` },
|
|
52
|
+
'ai-ms': { language: 'ms', text: `Dalam era digital kini, setiap organisasi perlu menggunakan strategi yang menyeluruh untuk mencapai hasil yang optimum. Dalam era digital kini, setiap pasukan memerlukan platform yang kukuh untuk meningkatkan setiap proses. Dalam era digital kini, setiap pemimpin perlu memacu transformasi yang menyeluruh untuk mencipta nilai yang mampan.
|
|
53
|
+
|
|
54
|
+
Tidak dapat dinafikan bahawa penyelesaian ini memainkan peranan penting untuk semua organisasi. Selain itu, platform ini menyediakan kerjasama yang lancar dan pengalaman yang lebih baik untuk setiap pengguna. Secara keseluruhannya, pendekatan ini membuka peluang baharu dan memastikan masa depan yang lebih cekap.` },
|
|
55
|
+
'human-ms': { language: 'ms', text: `Hujan turun. Farah menyemak resit kedai sebelum membuka pintu, membandingkan tiga harga dan meninggalkan nota di kaunter untuk syif petang. Akhirnya, pesanan itu keluar tanpa tergesa-gesa dan tiada sesiapa perlu menghubungi pembekal.` },
|
|
56
|
+
'ai-fr': { language: 'fr', text: `Dans le monde trépidant d'aujourd'hui, chaque équipe doit adopter une stratégie solide pour obtenir des résultats optimaux. Dans le monde trépidant d'aujourd'hui, chaque service a besoin d'une plateforme robuste pour améliorer chaque processus. Dans le monde trépidant d'aujourd'hui, chaque dirigeant doit conduire une transformation complète afin de créer une valeur durable.
|
|
57
|
+
|
|
58
|
+
Il est important de noter que cette solution joue un rôle crucial pour toutes les organisations. De plus, la plateforme offre une collaboration fluide et une expérience améliorée pour chaque utilisateur. En conclusion, cette approche ouvre de nouvelles possibilités et garantit un avenir plus efficace.` },
|
|
59
|
+
'human-fr': { language: 'fr', text: `Il pleut. Léa a vérifié les reçus avant d'ouvrir la boutique, comparé trois prix et laissé une note près de la caisse pour l'équipe du soir. À la fin, la commande est partie calmement et personne n'a appelé le fournisseur.` },
|
|
60
|
+
'ai-de': { language: 'de', text: `In der heutigen schnelllebigen Welt muss jedes Team eine robuste Strategie einsetzen, um optimale Ergebnisse zu erzielen. In der heutigen schnelllebigen Welt benötigt jede Abteilung eine leistungsstarke Plattform, um jeden Prozess zu verbessern. In der heutigen schnelllebigen Welt muss jede Führungskraft eine umfassende Transformation vorantreiben, um nachhaltigen Wert zu schaffen.
|
|
61
|
+
|
|
62
|
+
Es ist wichtig zu beachten, dass diese Lösung eine entscheidende Rolle für alle Organisationen spielt. Darüber hinaus ermöglicht die Plattform eine nahtlose Zusammenarbeit und eine bessere Erfahrung für jeden Nutzer. Zusammenfassend lässt sich sagen, dass dieser Ansatz neue Möglichkeiten eröffnet und eine effizientere Zukunft sichert.` },
|
|
63
|
+
'human-de': { language: 'de', text: `Es regnet. Lara prüfte die Belege vor dem Öffnen des Ladens, verglich drei Preise und legte für die Spätschicht eine Notiz neben die Kasse. Am Ende ging die Bestellung ruhig raus, und niemand musste den Lieferanten anrufen.` },
|
|
64
|
+
'ai-pt': { language: 'pt', text: `No mundo acelerado de hoje, cada equipe deve adotar uma estratégia robusta para alcançar resultados ideais. No mundo acelerado de hoje, cada departamento precisa de uma plataforma forte para melhorar cada processo. No mundo acelerado de hoje, cada líder deve conduzir uma transformação completa para criar valor sustentável.
|
|
65
|
+
|
|
66
|
+
Vale ressaltar que esta solução desempenha um papel crucial para todas as organizações. Além disso, a plataforma oferece colaboração fluida e uma experiência melhor para cada usuário. Em conclusão, essa abordagem abre novas oportunidades e garante um futuro mais eficiente.` },
|
|
67
|
+
'human-pt': { language: 'pt', text: `Choveu. Lara conferiu os recibos antes de abrir a loja, comparou três preços e deixou um bilhete perto do caixa para a equipe da tarde. No fim, o pedido saiu sem correria e ninguém precisou ligar para o fornecedor.` },
|
|
68
|
+
'ai-zh': { language: 'zh', text: `在当今快节奏的世界中,每个团队都必须采用稳健策略来取得最佳成果。在当今快节奏的世界中,每个部门都需要强大平台来改进每个流程。在当今快节奏的世界中,每位领导者都必须推动全面转型以创造可持续价值。
|
|
69
|
+
|
|
70
|
+
值得注意的是,该解决方案对所有组织都发挥着关键作用。此外,该平台为每位用户提供无缝协作和更好的体验。总而言之,这种方法会开启新的机会并确保更高效的未来。` },
|
|
71
|
+
'human-zh': { language: 'zh', text: `下雨了。林娜在开店前核对了收据,比较了三种价格,并把给晚班的便条放在收银台旁边。最后,订单从容发出,没有人需要联系供应商。` },
|
|
72
|
+
};
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "humanizer-lint",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "Deterministic
|
|
3
|
+
"version": "1.4.1",
|
|
4
|
+
"description": "Deterministic prose linter and stylometric rhythm analyzer with 8 language-specific anti-cliche packs plus universal Intl.Segmenter diagnostics for 100+ languages. Local zero-dependency Node.js CLI, no network.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"engines": {
|
|
7
7
|
"node": ">=18.3.0"
|
|
@@ -16,11 +16,13 @@
|
|
|
16
16
|
"LICENSE",
|
|
17
17
|
"scripts/",
|
|
18
18
|
"rules/",
|
|
19
|
-
"profiles/"
|
|
19
|
+
"profiles/",
|
|
20
|
+
"demo/samples.mjs"
|
|
20
21
|
],
|
|
21
22
|
"scripts": {
|
|
22
|
-
"test": "node --test tests/test-runner.mjs",
|
|
23
|
-
"build:demo": "node demo/build.mjs"
|
|
23
|
+
"test": "node demo/build.mjs && node --test tests/test-runner.mjs",
|
|
24
|
+
"build:demo": "node demo/build.mjs",
|
|
25
|
+
"benchmark": "node scripts/benchmark.mjs"
|
|
24
26
|
},
|
|
25
27
|
"homepage": "https://humanizer.aiclub.ai.id",
|
|
26
28
|
"repository": {
|
package/scripts/analyze.mjs
CHANGED
|
@@ -18,13 +18,17 @@ const { analyzeText } = createAnalyzer({ rules: RULES, defaultProfile: DEFAULT_P
|
|
|
18
18
|
|
|
19
19
|
export { analyzeText };
|
|
20
20
|
|
|
21
|
-
function formatHuman(result) {
|
|
21
|
+
function formatHuman(result, includeDeliveryGate = false) {
|
|
22
22
|
const lines = [`Status: ${result.status}`, `Score: ${result.score}`, `Language: ${result.language}`, `Rule pack: ${result.rulePack}`];
|
|
23
23
|
if (result.warning) lines.push(`Warning: ${result.warning}`);
|
|
24
24
|
lines.push('', 'Metrics:');
|
|
25
25
|
for (const [name, value] of Object.entries(result.metrics)) lines.push(` ${name}: ${value}`);
|
|
26
26
|
if (result.violations.length) lines.push('', 'Violations:');
|
|
27
27
|
for (const violation of result.violations) lines.push(` ${violation.type} | ${violation.location} | ${violation.fixInstruction}`);
|
|
28
|
+
if (includeDeliveryGate) {
|
|
29
|
+
lines.push('', 'Delivery Gate:');
|
|
30
|
+
for (const [name, block] of Object.entries(result.deliveryGate)) lines.push(` ${name}: ${JSON.stringify(block)}`);
|
|
31
|
+
}
|
|
28
32
|
return `${lines.join('\n')}\n`;
|
|
29
33
|
}
|
|
30
34
|
|
|
@@ -47,13 +51,14 @@ export async function runAnalyzeCli(argv, io = {}) {
|
|
|
47
51
|
lang: { type: 'string', default: 'auto' },
|
|
48
52
|
profile: { type: 'string' },
|
|
49
53
|
json: { type: 'boolean' },
|
|
54
|
+
'delivery-gate': { type: 'boolean' },
|
|
50
55
|
help: { type: 'boolean' },
|
|
51
56
|
},
|
|
52
57
|
strict: true,
|
|
53
58
|
allowPositionals: false,
|
|
54
59
|
});
|
|
55
60
|
if (values.help) {
|
|
56
|
-
stdout('Usage: humanizer-lint (--file <path> | --stdin) [--lang auto|universal|<ISO-639-1>] [--profile <path>] [--json]\n');
|
|
61
|
+
stdout('Usage: humanizer-lint (--file <path> | --stdin) [--lang auto|universal|<ISO-639-1>] [--profile <path>] [--json] [--delivery-gate]\n');
|
|
57
62
|
return 0;
|
|
58
63
|
}
|
|
59
64
|
if (Boolean(values.file) === Boolean(values.stdin)) throw new Error('Choose exactly one input: --file or --stdin');
|
|
@@ -61,8 +66,8 @@ export async function runAnalyzeCli(argv, io = {}) {
|
|
|
61
66
|
if (values.stdin && io.stdin == null && process.stdin.isTTY) throw new Error('--stdin requires piped input');
|
|
62
67
|
const text = values.file ? await new Promise((resolve, reject) => readFile(values.file, 'utf8', (error, data) => error ? reject(error) : resolve(data))) : await stdin();
|
|
63
68
|
const profile = values.profile ? JSON.parse(await new Promise((resolve, reject) => readFile(values.profile, 'utf8', (error, data) => error ? reject(error) : resolve(data)))) : undefined;
|
|
64
|
-
const result = analyzeText(text, { language: values.lang, profile });
|
|
65
|
-
stdout(values.json ? `${JSON.stringify(result, null, 2)}\n` : formatHuman(result));
|
|
69
|
+
const result = analyzeText(text, { language: values.lang, profile, sourcePath: values.file });
|
|
70
|
+
stdout(values.json ? `${JSON.stringify(result, null, 2)}\n` : formatHuman(result, values['delivery-gate']));
|
|
66
71
|
return result.status === 'FAIL' ? 1 : 0;
|
|
67
72
|
} catch (error) {
|
|
68
73
|
stderr(`ERROR: ${error.message}\n`);
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import { performance } from 'node:perf_hooks';
|
|
3
|
+
import { realpathSync } from 'node:fs';
|
|
4
|
+
import path from 'node:path';
|
|
5
|
+
|
|
6
|
+
import { analyzeText } from './analyze.mjs';
|
|
7
|
+
|
|
8
|
+
const parentDirectory = '..';
|
|
9
|
+
const { DEMO_SAMPLES } = await import(new URL([parentDirectory, 'demo', 'samples.mjs'].join('/'), import.meta.url));
|
|
10
|
+
|
|
11
|
+
const LANGUAGES = ['en', 'id', 'es', 'ms', 'fr', 'de', 'pt', 'zh'];
|
|
12
|
+
|
|
13
|
+
export function runBenchmark(iterations = 20) {
|
|
14
|
+
const expected = LANGUAGES.flatMap((language) => [
|
|
15
|
+
[`ai-${language}`, 'FAIL'],
|
|
16
|
+
[`human-${language}`, 'PASS'],
|
|
17
|
+
]);
|
|
18
|
+
const firstPass = expected.map(([id, status]) => {
|
|
19
|
+
const sample = DEMO_SAMPLES[id];
|
|
20
|
+
const result = analyzeText(sample.text, { language: sample.language });
|
|
21
|
+
return { id, expected: status, actual: result.status, score: result.score };
|
|
22
|
+
});
|
|
23
|
+
|
|
24
|
+
const started = performance.now();
|
|
25
|
+
for (let iteration = 0; iteration < iterations; iteration += 1) {
|
|
26
|
+
for (const [id] of expected) {
|
|
27
|
+
const sample = DEMO_SAMPLES[id];
|
|
28
|
+
analyzeText(sample.text, { language: sample.language });
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
const elapsed = performance.now() - started;
|
|
32
|
+
const secondPass = expected.map(([id]) => {
|
|
33
|
+
const sample = DEMO_SAMPLES[id];
|
|
34
|
+
const result = analyzeText(sample.text, { language: sample.language });
|
|
35
|
+
return { id, actual: result.status, score: result.score };
|
|
36
|
+
});
|
|
37
|
+
const aiScores = firstPass.filter(({ id }) => id.startsWith('ai-')).map(({ score }) => score);
|
|
38
|
+
const humanScores = firstPass.filter(({ id }) => id.startsWith('human-')).map(({ score }) => score);
|
|
39
|
+
const mean = (values) => values.reduce((sum, value) => sum + value, 0) / values.length;
|
|
40
|
+
const deterministic = firstPass.every((item, index) => item.actual === secondPass[index].actual && item.score === secondPass[index].score);
|
|
41
|
+
const correct = firstPass.filter((item) => item.actual === item.expected).length;
|
|
42
|
+
|
|
43
|
+
return {
|
|
44
|
+
fixtures: firstPass.length,
|
|
45
|
+
iterations,
|
|
46
|
+
totalRuns: iterations * expected.length,
|
|
47
|
+
elapsedMs: Number(elapsed.toFixed(3)),
|
|
48
|
+
meanMsPerAnalysis: Number((elapsed / (iterations * expected.length)).toFixed(4)),
|
|
49
|
+
meanAiScore: Number(mean(aiScores).toFixed(2)),
|
|
50
|
+
meanHumanScore: Number(mean(humanScores).toFixed(2)),
|
|
51
|
+
meanScoreSeparation: Number((mean(humanScores) - mean(aiScores)).toFixed(2)),
|
|
52
|
+
accuracy: Number((correct / firstPass.length).toFixed(4)),
|
|
53
|
+
deterministic,
|
|
54
|
+
results: firstPass,
|
|
55
|
+
};
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
function currentModulePath() {
|
|
59
|
+
let value = decodeURIComponent(new URL(import.meta.url).pathname);
|
|
60
|
+
if (/^\/[A-Za-z]:/.test(value)) value = value.slice(1);
|
|
61
|
+
return path.resolve(value);
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
if (process.argv[1] && realpathSync(process.argv[1]) === realpathSync(currentModulePath())) {
|
|
65
|
+
const report = runBenchmark();
|
|
66
|
+
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
|
|
67
|
+
if (!report.deterministic || report.accuracy !== 1 || report.meanScoreSeparation <= 0) process.exitCode = 1;
|
|
68
|
+
}
|
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
const parentDirectory = '..';
|
|
2
2
|
const utilityModule = (...segments) => new URL([parentDirectory, 'utils', ...segments].join('/'), import.meta.url);
|
|
3
3
|
const { sanitizeMarkdown } = await import(utilityModule('sanitizer.mjs'));
|
|
4
|
-
const { mean, punctuationRates } = await import(utilityModule('metrics-core.mjs'));
|
|
4
|
+
const { coefficientOfVariation, mean, punctuationRates } = await import(utilityModule('metrics-core.mjs'));
|
|
5
5
|
const { isCJKText, tokenizeText } = await import(utilityModule('tokenizer.mjs'));
|
|
6
|
+
const { lintDomainHygiene } = await import(utilityModule('domain-linters.mjs'));
|
|
6
7
|
|
|
7
8
|
const HAN = /[\u4e00-\u9fff]/gu;
|
|
8
9
|
const KANA = /[\u3040-\u30ff]/u;
|
|
@@ -52,6 +53,21 @@ function regexFor(pattern) {
|
|
|
52
53
|
return new RegExp(pattern, 'giu');
|
|
53
54
|
}
|
|
54
55
|
|
|
56
|
+
function suggestedReplacement(type, match, matches = []) {
|
|
57
|
+
const value = String(match ?? matches[0] ?? '').toLowerCase();
|
|
58
|
+
const replacements = [
|
|
59
|
+
[/pengimplementasian/iu, 'penerapan / menerapkan'],
|
|
60
|
+
[/\bdelve(?: into)?\b/iu, 'examine / explore'],
|
|
61
|
+
[/tidak dapat dipungkiri(?: bahwa)?/iu, 'state the concrete claim directly'],
|
|
62
|
+
];
|
|
63
|
+
const found = replacements.find(([pattern]) => pattern.test(value));
|
|
64
|
+
if (found) return found[1];
|
|
65
|
+
if (type === 'EXCESSIVE_NOMINALIZATION') return 'Replace the listed noun forms with a concrete actor and active verb.';
|
|
66
|
+
if (type === 'PASSIVE_BLOATING') return 'Name the actor, then use the active verb.';
|
|
67
|
+
if (type === 'SLOP_PHRASE') return 'Replace the stock phrase with the specific fact or action.';
|
|
68
|
+
return undefined;
|
|
69
|
+
}
|
|
70
|
+
|
|
55
71
|
function patternViolations(tokenized, rules, wordCount) {
|
|
56
72
|
if (!rules?.patterns) return [];
|
|
57
73
|
const violations = [];
|
|
@@ -76,6 +92,7 @@ function patternViolations(tokenized, rules, wordCount) {
|
|
|
76
92
|
location: locationFor(candidate.sentence),
|
|
77
93
|
match: candidate.match,
|
|
78
94
|
fixInstruction: rule.fixInstruction,
|
|
95
|
+
suggestedReplacement: suggestedReplacement(rule.type, candidate.match),
|
|
79
96
|
});
|
|
80
97
|
}
|
|
81
98
|
}
|
|
@@ -101,6 +118,87 @@ function monotonyViolations(tokenized) {
|
|
|
101
118
|
return violations;
|
|
102
119
|
}
|
|
103
120
|
|
|
121
|
+
function paragraphRhythmViolations(tokenized) {
|
|
122
|
+
const violations = [];
|
|
123
|
+
for (const paragraph of tokenized.paragraphs) {
|
|
124
|
+
if (paragraph.sentences.length < 4) continue;
|
|
125
|
+
const lengths = paragraph.sentences.map((sentence) => sentence.wordCount);
|
|
126
|
+
const paragraphCV = coefficientOfVariation(lengths);
|
|
127
|
+
const range = Math.max(...lengths) - Math.min(...lengths);
|
|
128
|
+
if (paragraphCV < 0.25 && range < 7) {
|
|
129
|
+
violations.push({
|
|
130
|
+
type: 'FLAT_PARAGRAPH_RHYTHM',
|
|
131
|
+
location: `Paragraph ${paragraph.paragraphIndex}`,
|
|
132
|
+
actual: round(paragraphCV),
|
|
133
|
+
range,
|
|
134
|
+
fixInstruction: 'Review the paragraph for uniform cadence; vary structure only where the meaning supports it.',
|
|
135
|
+
});
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
for (let index = 0; index <= paragraph.sentences.length - 4; index += 1) {
|
|
139
|
+
const window = paragraph.sentences.slice(index, index + 4);
|
|
140
|
+
const windowLengths = window.map((sentence) => sentence.wordCount);
|
|
141
|
+
const punctuationCounts = window.map((sentence) => (sentence.text.match(/[,;—,]/gu) ?? []).length);
|
|
142
|
+
if (punctuationCounts.every((count) => count === 1 || count === 2) && Math.max(...windowLengths) - Math.min(...windowLengths) <= 5) {
|
|
143
|
+
violations.push({
|
|
144
|
+
type: 'UNIFORM_CLAUSE_STRUCTURE',
|
|
145
|
+
location: `Paragraph ${paragraph.paragraphIndex}, Sentences ${window[0].sentenceIndex}-${window[3].sentenceIndex}`,
|
|
146
|
+
lengths: windowLengths,
|
|
147
|
+
fixInstruction: 'Change one clause pattern or punctuation shape where the ideas have a different relationship.',
|
|
148
|
+
});
|
|
149
|
+
}
|
|
150
|
+
}
|
|
151
|
+
}
|
|
152
|
+
return violations;
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
function repetitiveNgramViolations(tokenized, stopwords = []) {
|
|
156
|
+
if (tokenized.words.length < 120) return [];
|
|
157
|
+
const ignored = new Set(stopwords);
|
|
158
|
+
const content = tokenized.words
|
|
159
|
+
.map((word, index) => ({ word, index }))
|
|
160
|
+
.filter(({ word }) => !ignored.has(word) && word !== 'token_entity');
|
|
161
|
+
const violations = [];
|
|
162
|
+
const seen = new Set();
|
|
163
|
+
for (const size of [4, 3]) {
|
|
164
|
+
const occurrences = new Map();
|
|
165
|
+
for (let index = 0; index <= content.length - size; index += 1) {
|
|
166
|
+
const phrase = content.slice(index, index + size).map(({ word }) => word).join(' ');
|
|
167
|
+
const positions = occurrences.get(phrase) ?? [];
|
|
168
|
+
positions.push(content[index].index);
|
|
169
|
+
occurrences.set(phrase, positions);
|
|
170
|
+
}
|
|
171
|
+
for (const [phrase, positions] of occurrences) {
|
|
172
|
+
if (seen.has(phrase)) continue;
|
|
173
|
+
const repeated = positions.some((start, index) => positions[index + 2] != null && positions[index + 2] - start <= 400);
|
|
174
|
+
if (!repeated) continue;
|
|
175
|
+
seen.add(phrase);
|
|
176
|
+
violations.push({
|
|
177
|
+
type: 'REPETITIVE_NGRAM',
|
|
178
|
+
location: 'Whole text',
|
|
179
|
+
match: phrase,
|
|
180
|
+
occurrences: positions.length,
|
|
181
|
+
fixInstruction: 'Remove repeated wording or replace it with a precise reference that preserves the meaning.',
|
|
182
|
+
});
|
|
183
|
+
if (violations.length >= 5) return violations;
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
return violations;
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
function rhetoricalViolations(tokenized) {
|
|
190
|
+
const hooks = /^(?:mengapa demikian\?|menarik, bukan\?|here['’]s the kicker:|want to know the best part\?|let that sink in\.?)/iu;
|
|
191
|
+
const shortQuestions = tokenized.sentences.filter((sentence) => sentence.text.endsWith('?') && sentence.wordCount <= 6);
|
|
192
|
+
return tokenized.sentences
|
|
193
|
+
.filter((sentence) => hooks.test(sentence.text) || (shortQuestions.length >= 3 && sentence.text.endsWith('?') && sentence.wordCount <= 6))
|
|
194
|
+
.map((sentence) => ({
|
|
195
|
+
type: 'RHETORICAL_SLOP',
|
|
196
|
+
location: locationFor(sentence),
|
|
197
|
+
match: sentence.text,
|
|
198
|
+
fixInstruction: 'Remove the staged conversational hook and state the point directly.',
|
|
199
|
+
}));
|
|
200
|
+
}
|
|
201
|
+
|
|
104
202
|
function structuralViolations(tokenized, metrics) {
|
|
105
203
|
const violations = [];
|
|
106
204
|
if (metrics.wordCount >= 150 && metrics.compressionRatio < 0.42) {
|
|
@@ -210,6 +308,7 @@ function morphologyViolations(tokenized, morphology) {
|
|
|
210
308
|
ratio: round(nominalizationRatio),
|
|
211
309
|
matches: [...new Set(nominalizations)],
|
|
212
310
|
fixInstruction: 'Replace abstract nominalizations with concrete actors and verbs.',
|
|
311
|
+
suggestedReplacement: suggestedReplacement('EXCESSIVE_NOMINALIZATION', '', nominalizations),
|
|
213
312
|
});
|
|
214
313
|
}
|
|
215
314
|
if (passiveRatio > morphology.passiveMaxRatio) {
|
|
@@ -219,6 +318,7 @@ function morphologyViolations(tokenized, morphology) {
|
|
|
219
318
|
ratio: round(passiveRatio),
|
|
220
319
|
matches: [...new Set(passives)],
|
|
221
320
|
fixInstruction: 'Name the actor and use an active verb where the actor is known.',
|
|
321
|
+
suggestedReplacement: suggestedReplacement('PASSIVE_BLOATING', '', passives),
|
|
222
322
|
});
|
|
223
323
|
}
|
|
224
324
|
|
|
@@ -243,6 +343,34 @@ function morphologyViolations(tokenized, morphology) {
|
|
|
243
343
|
return { violations, nominalizationRatio, passiveRatio };
|
|
244
344
|
}
|
|
245
345
|
|
|
346
|
+
function splitSuggestion(sentence) {
|
|
347
|
+
if (!sentence || sentence.wordCount < 16) return undefined;
|
|
348
|
+
const candidates = [...sentence.text.matchAll(/\b(?:sehingga|tetapi|namun|karena|however|because|while)\b|[;,]/giu)];
|
|
349
|
+
if (!candidates.length) return undefined;
|
|
350
|
+
const middle = sentence.text.length / 2;
|
|
351
|
+
const best = candidates.sort((left, right) => Math.abs(left.index - middle) - Math.abs(right.index - middle))[0];
|
|
352
|
+
return `Consider splitting near “${best[0]}” if both resulting sentences remain complete.`;
|
|
353
|
+
}
|
|
354
|
+
|
|
355
|
+
function attachSurgicalHints(violations, tokenized) {
|
|
356
|
+
return violations.map((violation) => {
|
|
357
|
+
const sentenceMatch = /Sentence (\d+)/u.exec(violation.location ?? '');
|
|
358
|
+
const rangeMatch = /Sentences (\d+)-(\d+)/u.exec(violation.location ?? '');
|
|
359
|
+
const sentence = sentenceMatch
|
|
360
|
+
? tokenized.sentences.find((item) => item.sentenceIndex === Number(sentenceMatch[1]))
|
|
361
|
+
: rangeMatch
|
|
362
|
+
? tokenized.sentences.find((item) => item.sentenceIndex >= Number(rangeMatch[1]) && item.sentenceIndex <= Number(rangeMatch[2]) && item.wordCount >= 16)
|
|
363
|
+
: undefined;
|
|
364
|
+
const replacement = violation.suggestedReplacement ?? suggestedReplacement(violation.type, violation.match, violation.matches);
|
|
365
|
+
const split = splitSuggestion(sentence);
|
|
366
|
+
return {
|
|
367
|
+
...violation,
|
|
368
|
+
...(replacement ? { suggestedReplacement: replacement } : {}),
|
|
369
|
+
...(split ? { splitSuggestion: split } : {}),
|
|
370
|
+
};
|
|
371
|
+
});
|
|
372
|
+
}
|
|
373
|
+
|
|
246
374
|
function voiceDeviation(metrics, tokenized, text, profile) {
|
|
247
375
|
const targets = profile?.targets ?? profile?.metrics;
|
|
248
376
|
if (!targets) return 0;
|
|
@@ -275,33 +403,74 @@ function scoreResult(metrics, violations, thresholds, mattrTarget) {
|
|
|
275
403
|
if (metrics.sentenceCount >= 8 && (metrics.shortSentenceRatio < thresholds.shortSentenceRatio || metrics.longSentenceRatio < thresholds.longSentenceRatio)) {
|
|
276
404
|
burstinessPenalty = Math.min(25, burstinessPenalty + 8);
|
|
277
405
|
}
|
|
278
|
-
|
|
406
|
+
if (violations.some((violation) => violation.type === 'FLAT_PARAGRAPH_RHYTHM')) burstinessPenalty = Math.min(25, burstinessPenalty + 5);
|
|
407
|
+
const rhythmPenalty = Math.min(20, violations.reduce((total, violation) => {
|
|
408
|
+
if (violation.type === 'MONOTONOUS_RHYTHM') return total + 6;
|
|
409
|
+
if (violation.type === 'UNIFORM_CLAUSE_STRUCTURE') return total + 5;
|
|
410
|
+
return total;
|
|
411
|
+
}, 0));
|
|
279
412
|
const lexicalPenalty = Math.min(30, violations.reduce((total, violation) => {
|
|
280
413
|
if (violation.type === 'SLOP_PHRASE' || violation.type === 'DEFINITIONAL_OPENING') return total + 5;
|
|
281
414
|
if (violation.type === 'STRUCTURAL_CLICHE') return total + 4;
|
|
415
|
+
if (['RHETORICAL_SLOP', 'WEAK_CTA', 'HYPERBOLIC_CLAIM', 'TAUTOLOGICAL_CODE_COMMENT', 'DECORATIVE_COMMENT_BANNER', 'LAZY_AI_PLACEHOLDER', 'MISSING_ACCESSIBLE_NAME', 'MISSING_IMG_ALT', 'HEADING_ORDER_SKIP', 'AI_GRADIENT_CLICHE', 'NON_RESPONSIVE_FIXED_WIDTH'].includes(violation.type)) return total + 4;
|
|
282
416
|
return total;
|
|
283
417
|
}, 0));
|
|
284
418
|
const morphologyPenalty = Math.min(15,
|
|
285
419
|
(violations.some((violation) => violation.type === 'EXCESSIVE_NOMINALIZATION') ? 7.5 : 0)
|
|
286
420
|
+ Math.min(15, violations.filter((violation) => violation.type === 'PASSIVE_BLOATING').length * 7.5));
|
|
287
421
|
const vocabularyPenalty = metrics.mattr < mattrTarget ? 5 : 0;
|
|
288
|
-
const entropyPenalty = violations.some((violation) => ['LOW_ENTROPY_REPETITION', 'SYMMETRIC_PARAGRAPHS', 'REPETITIVE_STARTERS'].includes(violation.type)) ? 5 : 0;
|
|
422
|
+
const entropyPenalty = violations.some((violation) => ['LOW_ENTROPY_REPETITION', 'SYMMETRIC_PARAGRAPHS', 'REPETITIVE_STARTERS', 'REPETITIVE_NGRAM'].includes(violation.type)) ? 5 : 0;
|
|
289
423
|
const profilePenalty = violations.some((violation) => violation.type === 'VOICE_PROFILE_DEVIATION') ? 5 : 0;
|
|
290
424
|
const vocabularyAndProfilePenalty = Math.min(10, vocabularyPenalty + entropyPenalty + profilePenalty);
|
|
291
425
|
return Math.max(0, Math.round(100 - burstinessPenalty - rhythmPenalty - lexicalPenalty - morphologyPenalty - vocabularyAndProfilePenalty));
|
|
292
426
|
}
|
|
293
427
|
|
|
428
|
+
function deliveryGate({ metrics, violations, language, rulePack, score, status }) {
|
|
429
|
+
const types = (names) => violations.filter((violation) => names.includes(violation.type));
|
|
430
|
+
const prioritizedFixes = violations.slice(0, 8).map((violation) => ({
|
|
431
|
+
type: violation.type,
|
|
432
|
+
location: violation.location,
|
|
433
|
+
fixInstruction: violation.fixInstruction,
|
|
434
|
+
...(violation.suggestedReplacement ? { suggestedReplacement: violation.suggestedReplacement } : {}),
|
|
435
|
+
...(violation.splitSuggestion ? { splitSuggestion: violation.splitSuggestion } : {}),
|
|
436
|
+
}));
|
|
437
|
+
return {
|
|
438
|
+
rhythmAndStylometry: {
|
|
439
|
+
burstinessCV: metrics.burstinessCV,
|
|
440
|
+
paragraphBurstinessCV: metrics.paragraphBurstinessCV,
|
|
441
|
+
adjacentDelta: metrics.adjacentDelta,
|
|
442
|
+
clauseSymmetry: types(['UNIFORM_CLAUSE_STRUCTURE']),
|
|
443
|
+
rhythmFindings: types(['LOW_BURSTINESS', 'FLAT_PARAGRAPH_RHYTHM', 'MONOTONOUS_RHYTHM', 'MISSING_SENTENCE_RANGE', 'LOW_ADJACENT_DELTA']),
|
|
444
|
+
},
|
|
445
|
+
lexicalAndMorphology: {
|
|
446
|
+
language,
|
|
447
|
+
rulePack,
|
|
448
|
+
mode: rulePack === 'universal' ? 'Universal stylometry' : '8 language-specific lexical pack',
|
|
449
|
+
ngrams: types(['REPETITIVE_NGRAM']),
|
|
450
|
+
morphology: types(['SLOP_PHRASE', 'EXCESSIVE_NOMINALIZATION', 'PASSIVE_BLOATING', 'DEFINITIONAL_OPENING']),
|
|
451
|
+
},
|
|
452
|
+
domainHygiene: {
|
|
453
|
+
copywritingAndCta: types(['WEAK_CTA', 'HYPERBOLIC_CLAIM']),
|
|
454
|
+
codeComments: types(['TAUTOLOGICAL_CODE_COMMENT', 'DECORATIVE_COMMENT_BANNER', 'LAZY_AI_PLACEHOLDER']),
|
|
455
|
+
uiAccessibilityResponsive: types(['MISSING_ACCESSIBLE_NAME', 'MISSING_IMG_ALT', 'HEADING_ORDER_SKIP', 'AI_GRADIENT_CLICHE', 'NON_RESPONSIVE_FIXED_WIDTH']),
|
|
456
|
+
},
|
|
457
|
+
finalVerdict: { score, status, prioritizedFixes },
|
|
458
|
+
};
|
|
459
|
+
}
|
|
460
|
+
|
|
294
461
|
export function createAnalyzer({ rules, defaultProfile, calculateMetrics, calculateMetricsAsync }) {
|
|
295
462
|
if (!rules || !defaultProfile || typeof calculateMetrics !== 'function') {
|
|
296
463
|
throw new TypeError('createAnalyzer requires rules, defaultProfile, and calculateMetrics');
|
|
297
464
|
}
|
|
298
465
|
|
|
299
466
|
const prepare = (input, options) => {
|
|
300
|
-
const
|
|
467
|
+
const original = String(input ?? '');
|
|
468
|
+
const sanitized = sanitizeMarkdown(original);
|
|
301
469
|
const requestedLanguage = String(options.language ?? 'auto').toLowerCase();
|
|
302
470
|
const initial = tokenizeText(sanitized, requestedLanguage);
|
|
303
471
|
const language = requestedLanguage === 'auto' ? detectLanguage(sanitized, initial.words, rules) : requestedLanguage;
|
|
304
472
|
return {
|
|
473
|
+
original,
|
|
305
474
|
sanitized,
|
|
306
475
|
language,
|
|
307
476
|
rulePack: rules[language] ? language : 'universal',
|
|
@@ -309,15 +478,19 @@ export function createAnalyzer({ rules, defaultProfile, calculateMetrics, calcul
|
|
|
309
478
|
};
|
|
310
479
|
};
|
|
311
480
|
|
|
312
|
-
const analyzePrepared = ({ sanitized, language, rulePack, tokenized }, options, metrics) => {
|
|
481
|
+
const analyzePrepared = ({ original, sanitized, language, rulePack, tokenized }, options, metrics) => {
|
|
313
482
|
const profile = options.profile ?? defaultProfile;
|
|
314
483
|
const thresholds = { ...defaultProfile.thresholds, ...(profile.thresholds ?? {}) };
|
|
315
484
|
const mattrTarget = mattrThreshold(thresholds, language, tokenized);
|
|
316
485
|
const patternMatches = patternViolations(tokenized, rules[language], metrics.wordCount);
|
|
486
|
+
const domainFindings = lintDomainHygiene(original, options.sourcePath);
|
|
317
487
|
|
|
318
488
|
if (metrics.wordCount < thresholds.minimumWords || metrics.sentenceCount < thresholds.minimumSentences) {
|
|
319
|
-
const violations =
|
|
320
|
-
|
|
489
|
+
const violations = attachSurgicalHints([
|
|
490
|
+
...patternMatches.filter((violation) => violation.type === 'SLOP_PHRASE'),
|
|
491
|
+
...domainFindings,
|
|
492
|
+
], tokenized);
|
|
493
|
+
const result = {
|
|
321
494
|
status: 'SKIP',
|
|
322
495
|
score: 100,
|
|
323
496
|
language,
|
|
@@ -326,6 +499,8 @@ export function createAnalyzer({ rules, defaultProfile, calculateMetrics, calcul
|
|
|
326
499
|
metrics: Object.fromEntries(Object.entries({ ...metrics, nominalizationRatio: 0, passiveRatio: 0, voiceProfileDeviation: 0 }).map(([key, value]) => [key, typeof value === 'number' ? round(value) : value])),
|
|
327
500
|
violations,
|
|
328
501
|
};
|
|
502
|
+
result.deliveryGate = deliveryGate(result);
|
|
503
|
+
return result;
|
|
329
504
|
}
|
|
330
505
|
|
|
331
506
|
const morphology = morphologyViolations(tokenized, rules[language]?.morphology);
|
|
@@ -341,22 +516,27 @@ export function createAnalyzer({ rules, defaultProfile, calculateMetrics, calcul
|
|
|
341
516
|
match: 'Mean sentence length deviates >30% from target voice profile',
|
|
342
517
|
fixInstruction: 'Adjust overall sentence complexity so mean sentence length approaches the target profile.',
|
|
343
518
|
}] : [];
|
|
344
|
-
const violations = [
|
|
519
|
+
const violations = attachSurgicalHints([
|
|
345
520
|
...patternMatches,
|
|
346
521
|
...monotonyViolations(tokenized),
|
|
522
|
+
...paragraphRhythmViolations(tokenized),
|
|
347
523
|
...structuralViolations(tokenized, metrics),
|
|
524
|
+
...repetitiveNgramViolations(tokenized, rules[language]?.stopwords),
|
|
525
|
+
...rhetoricalViolations(tokenized),
|
|
348
526
|
...morphology.violations,
|
|
349
527
|
...profileViolations,
|
|
350
528
|
...statisticalViolations(metrics, thresholds, mattrTarget),
|
|
351
|
-
|
|
529
|
+
...domainFindings,
|
|
530
|
+
], tokenized);
|
|
352
531
|
const score = scoreResult(metrics, violations, thresholds, mattrTarget);
|
|
353
532
|
const cvTarget = metrics.sentenceCount >= 8 ? thresholds.burstinessCVLong : thresholds.burstinessCVShort;
|
|
354
|
-
const
|
|
533
|
+
const forbiddenTypes = new Set(['SLOP_PHRASE', 'DEFINITIONAL_OPENING', 'RHETORICAL_SLOP', 'WEAK_CTA', 'HYPERBOLIC_CLAIM', 'TAUTOLOGICAL_CODE_COMMENT', 'DECORATIVE_COMMENT_BANNER', 'LAZY_AI_PLACEHOLDER', 'MISSING_ACCESSIBLE_NAME', 'MISSING_IMG_ALT', 'HEADING_ORDER_SKIP', 'AI_GRADIENT_CLICHE', 'NON_RESPONSIVE_FIXED_WIDTH']);
|
|
534
|
+
const forbidden = violations.some((violation) => forbiddenTypes.has(violation.type));
|
|
355
535
|
const rhythmPass = metrics.adjacentDelta >= thresholds.adjacentDelta;
|
|
356
536
|
const distributionPass = metrics.sentenceCount < 8 || (metrics.shortSentenceRatio >= thresholds.shortSentenceRatio && metrics.longSentenceRatio >= thresholds.longSentenceRatio);
|
|
357
537
|
const status = score >= thresholds.passScore && !forbidden && metrics.burstinessCV >= cvTarget && rhythmPass && distributionPass ? 'PASS' : 'FAIL';
|
|
358
538
|
|
|
359
|
-
|
|
539
|
+
const result = {
|
|
360
540
|
status,
|
|
361
541
|
score,
|
|
362
542
|
language,
|
|
@@ -364,6 +544,8 @@ export function createAnalyzer({ rules, defaultProfile, calculateMetrics, calcul
|
|
|
364
544
|
metrics: Object.fromEntries(Object.entries(metrics).map(([key, value]) => [key, typeof value === 'number' ? round(value) : value])),
|
|
365
545
|
violations,
|
|
366
546
|
};
|
|
547
|
+
result.deliveryGate = deliveryGate(result);
|
|
548
|
+
return result;
|
|
367
549
|
};
|
|
368
550
|
|
|
369
551
|
return {
|
package/scripts/fingerprint.mjs
CHANGED
|
@@ -82,7 +82,9 @@ export async function runFingerprintCli(argv, io = {}) {
|
|
|
82
82
|
args: argv,
|
|
83
83
|
options: {
|
|
84
84
|
input: { type: 'string' },
|
|
85
|
+
dir: { type: 'string' },
|
|
85
86
|
output: { type: 'string' },
|
|
87
|
+
out: { type: 'string' },
|
|
86
88
|
name: { type: 'string' },
|
|
87
89
|
lang: { type: 'string', default: 'auto' },
|
|
88
90
|
help: { type: 'boolean' },
|
|
@@ -91,14 +93,16 @@ export async function runFingerprintCli(argv, io = {}) {
|
|
|
91
93
|
allowPositionals: false,
|
|
92
94
|
});
|
|
93
95
|
if (values.help) {
|
|
94
|
-
stdout('Usage: humanizer-fingerprint --input <file-or-dir> --output <profile.json> [--name <name>] [--lang id|en|auto]\n');
|
|
96
|
+
stdout('Usage: humanizer-fingerprint (--input <file-or-dir> | --dir <samples-dir>) (--output <profile.json> | --out <profile.json>) [--name <name>] [--lang id|en|auto]\n');
|
|
95
97
|
return 0;
|
|
96
98
|
}
|
|
97
|
-
|
|
99
|
+
const input = values.input ?? values.dir;
|
|
100
|
+
const output = values.output ?? values.out;
|
|
101
|
+
if (!input || !output) throw new Error('--input/--dir and --output/--out are required');
|
|
98
102
|
if (!['id', 'en', 'auto'].includes(values.lang)) throw new Error('--lang must be id, en, or auto');
|
|
99
|
-
const files = await collectInput(
|
|
103
|
+
const files = await collectInput(input);
|
|
100
104
|
if (!files.length) throw new Error('No .txt or .md samples found');
|
|
101
|
-
const profile = await fingerprintSamples(files, { output
|
|
105
|
+
const profile = await fingerprintSamples(files, { output, name: values.name, language: values.lang });
|
|
102
106
|
stdout(`${JSON.stringify(profile, null, 2)}\n`);
|
|
103
107
|
return 0;
|
|
104
108
|
} catch (error) {
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
const finding = (type, match, fixInstruction, location = 'Whole input') => ({
|
|
2
|
+
type,
|
|
3
|
+
location,
|
|
4
|
+
match,
|
|
5
|
+
fixInstruction,
|
|
6
|
+
});
|
|
7
|
+
|
|
8
|
+
function regexFindings(text, definitions) {
|
|
9
|
+
const source = String(text ?? '');
|
|
10
|
+
return definitions.flatMap(({ type, pattern, fixInstruction }) => [...source.matchAll(pattern)].map((match) => (
|
|
11
|
+
finding(type, match[0], fixInstruction)
|
|
12
|
+
)));
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
export function lintCopywriting(text) {
|
|
16
|
+
return regexFindings(text, [
|
|
17
|
+
{
|
|
18
|
+
type: 'WEAK_CTA',
|
|
19
|
+
pattern: /\b(?:click here|learn more|klik di sini|pelajari lebih lanjut)\b/giu,
|
|
20
|
+
fixInstruction: 'Replace the generic CTA with a specific action and outcome.',
|
|
21
|
+
},
|
|
22
|
+
{
|
|
23
|
+
type: 'HYPERBOLIC_CLAIM',
|
|
24
|
+
pattern: /\b(?:supercharge your workflow|solusi terlengkap dan terbaik)\b/giu,
|
|
25
|
+
fixInstruction: 'Replace the superlative claim with a concrete capability or supported result.',
|
|
26
|
+
},
|
|
27
|
+
]);
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
export function lintCodeComments(text) {
|
|
31
|
+
return regexFindings(text, [
|
|
32
|
+
{
|
|
33
|
+
type: 'TAUTOLOGICAL_CODE_COMMENT',
|
|
34
|
+
pattern: /^\s*\/\/\s*(?:import dependencies|function to\b.*)$/gimu,
|
|
35
|
+
fixInstruction: 'Delete the comment or explain a non-obvious constraint instead.',
|
|
36
|
+
},
|
|
37
|
+
{
|
|
38
|
+
type: 'DECORATIVE_COMMENT_BANNER',
|
|
39
|
+
pattern: /^\s*\/\/\s*=+\s*[^=\n]+\s*=+\s*$/gmu,
|
|
40
|
+
fixInstruction: 'Use a normal section comment only when the grouping adds navigation value.',
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
type: 'LAZY_AI_PLACEHOLDER',
|
|
44
|
+
pattern: /^\s*\/\/\s*(?:\.\.\.\s*)?rest of code\b.*$/gimu,
|
|
45
|
+
fixInstruction: 'Replace the placeholder with complete code or remove it.',
|
|
46
|
+
},
|
|
47
|
+
]);
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
function markupExtension(sourcePath) {
|
|
51
|
+
return /\.(?:html?|jsx|tsx|vue|astro)$/iu.test(String(sourcePath ?? ''));
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
export function lintUiAndAccessibility(text, sourcePath = '') {
|
|
55
|
+
if (sourcePath && !markupExtension(sourcePath)) return [];
|
|
56
|
+
const source = String(text ?? '');
|
|
57
|
+
const findings = [];
|
|
58
|
+
|
|
59
|
+
for (const match of source.matchAll(/<(button|a)\b([^>]*)>([\s\S]*?)<\/\1>/giu)) {
|
|
60
|
+
const [, tag, attributes, body] = match;
|
|
61
|
+
const visibleText = body.replace(/<[^>]+>/gu, '').trim();
|
|
62
|
+
if (!visibleText && !/\b(?:aria-label|aria-labelledby|title)\s*=/iu.test(attributes)) {
|
|
63
|
+
findings.push(finding('MISSING_ACCESSIBLE_NAME', match[0], `Add visible text or an accessible name to this ${tag}.`));
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
for (const match of source.matchAll(/<img\b([^>]*)>/giu)) {
|
|
68
|
+
if (!/\balt\s*=/iu.test(match[1])) findings.push(finding('MISSING_IMG_ALT', match[0], 'Add meaningful alt text, or alt="" for a decorative image.'));
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
const headings = [...source.matchAll(/<h([1-6])\b/giu)].map((match) => Number(match[1]));
|
|
72
|
+
for (let index = 1; index < headings.length; index += 1) {
|
|
73
|
+
if (headings[index] > headings[index - 1] + 1) {
|
|
74
|
+
findings.push(finding('HEADING_ORDER_SKIP', `h${headings[index - 1]} → h${headings[index]}`, 'Use the next heading level unless the document outline requires a separate region.'));
|
|
75
|
+
break;
|
|
76
|
+
}
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
if (/(?:bg-)?gradient-(?:to|from|via)|linear-gradient\s*\(/iu.test(source)) {
|
|
80
|
+
findings.push(finding('AI_GRADIENT_CLICHE', 'gradient styling', 'Use a restrained solid color or a gradient tied to the product visual system.'));
|
|
81
|
+
}
|
|
82
|
+
if (/(?:width\s*:\s*|\bw-\[)(?:[6-9]\d{2}|\d{4,})px/iu.test(source)) {
|
|
83
|
+
findings.push(finding('NON_RESPONSIVE_FIXED_WIDTH', 'large fixed width', 'Use max-width and fluid sizing so the layout fits narrow viewports.'));
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
return findings;
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
export function lintDomainHygiene(text, sourcePath = '') {
|
|
90
|
+
return [
|
|
91
|
+
...lintCopywriting(text),
|
|
92
|
+
...lintCodeComments(text),
|
|
93
|
+
...(markupExtension(sourcePath) ? lintUiAndAccessibility(text, sourcePath) : []),
|
|
94
|
+
];
|
|
95
|
+
}
|
|
@@ -49,6 +49,9 @@ export function punctuationRates(text, wordCount) {
|
|
|
49
49
|
|
|
50
50
|
export function calculateMetricsFromCompressionRatio(tokenized, compressionRatio) {
|
|
51
51
|
const lengths = tokenized.sentences.map((sentence) => sentence.wordCount);
|
|
52
|
+
const paragraphCVs = tokenized.paragraphs
|
|
53
|
+
.filter((paragraph) => paragraph.sentences.length >= 4)
|
|
54
|
+
.map((paragraph) => coefficientOfVariation(paragraph.sentences.map((sentence) => sentence.wordCount)));
|
|
52
55
|
const wordCount = tokenized.words.length;
|
|
53
56
|
const sentenceCount = lengths.length;
|
|
54
57
|
return {
|
|
@@ -57,6 +60,7 @@ export function calculateMetricsFromCompressionRatio(tokenized, compressionRatio
|
|
|
57
60
|
meanSentenceLength: mean(lengths),
|
|
58
61
|
stdDev: standardDeviation(lengths),
|
|
59
62
|
burstinessCV: coefficientOfVariation(lengths),
|
|
63
|
+
paragraphBurstinessCV: paragraphCVs.length ? mean(paragraphCVs) : 0,
|
|
60
64
|
adjacentDelta: adjacentDelta(lengths),
|
|
61
65
|
shortSentenceRatio: sentenceCount ? lengths.filter((length) => length <= 6).length / sentenceCount : 0,
|
|
62
66
|
longSentenceRatio: sentenceCount ? lengths.filter((length) => length >= 26).length / sentenceCount : 0,
|