patina-cli 3.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (180) hide show
  1. package/.patina.default.yaml +211 -0
  2. package/CHANGELOG.md +265 -0
  3. package/LICENSE +21 -0
  4. package/README.md +319 -0
  5. package/README_JA.md +254 -0
  6. package/README_KR.md +253 -0
  7. package/README_ZH.md +254 -0
  8. package/SKILL-MAX.md +455 -0
  9. package/SKILL.md +730 -0
  10. package/assets/brand/patina-icon.svg +9 -0
  11. package/assets/brand/patina-logo.svg +17 -0
  12. package/assets/social/patina-before-after.svg +46 -0
  13. package/assets/social/patina-og.svg +31 -0
  14. package/bin/patina.js +9 -0
  15. package/core/scoring.md +657 -0
  16. package/core/standalone-prompt.md +364 -0
  17. package/core/stylometry.md +754 -0
  18. package/core/voice.md +163 -0
  19. package/docs/AUTHENTICATION.md +105 -0
  20. package/docs/AUTHENTICATION_KR.md +105 -0
  21. package/docs/BRANDING.md +37 -0
  22. package/docs/CLI.md +80 -0
  23. package/docs/COMPARISON.md +38 -0
  24. package/docs/COOKBOOK.md +173 -0
  25. package/docs/DEMO.md +40 -0
  26. package/docs/ETHICS.md +27 -0
  27. package/docs/EXAMPLES.md +130 -0
  28. package/docs/EXAMPLES_KR.md +130 -0
  29. package/docs/EXIT-CODES.md +25 -0
  30. package/docs/FAQ.md +67 -0
  31. package/docs/FAQ_KR.md +65 -0
  32. package/docs/FLAG-PARITY.md +53 -0
  33. package/docs/GLOSSARY.md +123 -0
  34. package/docs/PATTERNS-EN.md +718 -0
  35. package/docs/PATTERNS-JA.md +706 -0
  36. package/docs/PATTERNS-KO.md +707 -0
  37. package/docs/PATTERNS-ZH.md +706 -0
  38. package/docs/PATTERNS.md +22 -0
  39. package/docs/ROADMAP.md +315 -0
  40. package/docs/audits/2026-05-deep-research.md +290 -0
  41. package/docs/benchmarks/detector-comparison.json +442 -0
  42. package/docs/benchmarks/detector-comparison.md +65 -0
  43. package/docs/benchmarks/latest.json +988 -0
  44. package/docs/benchmarks/latest.md +112 -0
  45. package/docs/integrations/docker.md +19 -0
  46. package/docs/integrations/github-action.md +59 -0
  47. package/docs/integrations/pre-commit.md +77 -0
  48. package/docs/integrations/release.md +43 -0
  49. package/docs/internal/HARNESS.md +14 -0
  50. package/docs/internal/README.md +14 -0
  51. package/docs/internal/WARP.md +23 -0
  52. package/docs/research/2025-rebaseline-plan.md +89 -0
  53. package/docs/research/ai-human-metrics.md +380 -0
  54. package/docs/social/gstack-cardnews.html +236 -0
  55. package/docs/social/gstack-cardnews.md +88 -0
  56. package/docs/social/gstack-thread.md +106 -0
  57. package/docs/social/patina-launch-copy.md +227 -0
  58. package/docs/superpowers/specs/2026-04-03-meaning-preservation-design.md +299 -0
  59. package/lexicon/ai-en.md +162 -0
  60. package/lexicon/ai-ko.md +159 -0
  61. package/package.json +100 -0
  62. package/patina-max/SKILL.md +523 -0
  63. package/patina-max/composite.py +457 -0
  64. package/patterns/en-communication.md +89 -0
  65. package/patterns/en-content.md +133 -0
  66. package/patterns/en-filler.md +113 -0
  67. package/patterns/en-language.md +163 -0
  68. package/patterns/en-structure.md +173 -0
  69. package/patterns/en-style.md +139 -0
  70. package/patterns/en-viral-hook.md +211 -0
  71. package/patterns/ja-communication.md +101 -0
  72. package/patterns/ja-content.md +153 -0
  73. package/patterns/ja-filler.md +123 -0
  74. package/patterns/ja-language.md +190 -0
  75. package/patterns/ja-structure.md +142 -0
  76. package/patterns/ja-style.md +147 -0
  77. package/patterns/ja-viral-hook.md +216 -0
  78. package/patterns/ko-communication.md +98 -0
  79. package/patterns/ko-content.md +154 -0
  80. package/patterns/ko-filler.md +105 -0
  81. package/patterns/ko-language.md +182 -0
  82. package/patterns/ko-structure.md +147 -0
  83. package/patterns/ko-style.md +146 -0
  84. package/patterns/ko-viral-hook.md +211 -0
  85. package/patterns/zh-communication.md +101 -0
  86. package/patterns/zh-content.md +153 -0
  87. package/patterns/zh-filler.md +118 -0
  88. package/patterns/zh-language.md +173 -0
  89. package/patterns/zh-structure.md +145 -0
  90. package/patterns/zh-style.md +159 -0
  91. package/patterns/zh-viral-hook.md +216 -0
  92. package/profiles/academic.md +53 -0
  93. package/profiles/blog.md +81 -0
  94. package/profiles/casual-conversation.md +105 -0
  95. package/profiles/code-comment.md +104 -0
  96. package/profiles/commit-message.md +99 -0
  97. package/profiles/default.md +62 -0
  98. package/profiles/email.md +52 -0
  99. package/profiles/formal.md +98 -0
  100. package/profiles/instructional.md +80 -0
  101. package/profiles/legal.md +57 -0
  102. package/profiles/marketing.md +56 -0
  103. package/profiles/medical.md +53 -0
  104. package/profiles/narrative.md +79 -0
  105. package/profiles/release-notes.md +98 -0
  106. package/profiles/social.md +56 -0
  107. package/profiles/technical.md +53 -0
  108. package/scripts/benchmark-report.mjs +252 -0
  109. package/scripts/check-release-metadata.mjs +48 -0
  110. package/scripts/detector-comparison.mjs +267 -0
  111. package/scripts/lint.mjs +40 -0
  112. package/scripts/precommit-score.mjs +31 -0
  113. package/scripts/prose-score.mjs +186 -0
  114. package/scripts/update-benchmark-ranges.mjs +108 -0
  115. package/src/api.js +330 -0
  116. package/src/auth.js +105 -0
  117. package/src/backends/claude-cli.js +112 -0
  118. package/src/backends/codex-cli.js +121 -0
  119. package/src/backends/contract.js +21 -0
  120. package/src/backends/gemini-cli.js +135 -0
  121. package/src/backends/index.js +159 -0
  122. package/src/cache.js +106 -0
  123. package/src/cli.js +1280 -0
  124. package/src/commands/doctor.js +229 -0
  125. package/src/commands/init.js +208 -0
  126. package/src/config.js +126 -0
  127. package/src/errors.js +53 -0
  128. package/src/features/index.js +96 -0
  129. package/src/features/lexicon.js +90 -0
  130. package/src/features/segment.js +49 -0
  131. package/src/features/stylometry.js +50 -0
  132. package/src/loader.js +103 -0
  133. package/src/logger.js +70 -0
  134. package/src/manifest.js +162 -0
  135. package/src/max-mode.js +207 -0
  136. package/src/ouroboros.js +233 -0
  137. package/src/output.js +480 -0
  138. package/src/prompt-builder.js +409 -0
  139. package/src/providers.js +100 -0
  140. package/src/scoring.js +531 -0
  141. package/src/security.js +133 -0
  142. package/tests/fixtures/suspect-zones/en/ai/en-ai-01.md +16 -0
  143. package/tests/fixtures/suspect-zones/en/ai/en-ai-02.md +16 -0
  144. package/tests/fixtures/suspect-zones/en/ai/en-ai-03.md +17 -0
  145. package/tests/fixtures/suspect-zones/en/ai/en-ai-04.md +15 -0
  146. package/tests/fixtures/suspect-zones/en/ai/en-ai-05.md +16 -0
  147. package/tests/fixtures/suspect-zones/en/ai/en-ai-06-chat-register.md +16 -0
  148. package/tests/fixtures/suspect-zones/en/natural/en-nat-01.md +15 -0
  149. package/tests/fixtures/suspect-zones/en/natural/en-nat-02.md +15 -0
  150. package/tests/fixtures/suspect-zones/en/natural/en-nat-03.md +15 -0
  151. package/tests/fixtures/suspect-zones/en/natural/en-nat-04.md +15 -0
  152. package/tests/fixtures/suspect-zones/en/natural/en-nat-05.md +15 -0
  153. package/tests/fixtures/suspect-zones/expected-ranges.json +939 -0
  154. package/tests/fixtures/suspect-zones/ja/ai/ja-ai-01.md +11 -0
  155. package/tests/fixtures/suspect-zones/ja/ai/ja-ai-02.md +11 -0
  156. package/tests/fixtures/suspect-zones/ja/ai/ja-ai-03.md +11 -0
  157. package/tests/fixtures/suspect-zones/ja/natural/ja-nat-01.md +11 -0
  158. package/tests/fixtures/suspect-zones/ja/natural/ja-nat-02.md +11 -0
  159. package/tests/fixtures/suspect-zones/ja/natural/ja-nat-03.md +11 -0
  160. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-01.md +14 -0
  161. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-02.md +16 -0
  162. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-03.md +15 -0
  163. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-04.md +15 -0
  164. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-05.md +16 -0
  165. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-06-chat-register.md +16 -0
  166. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-01.md +15 -0
  167. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-02.md +15 -0
  168. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-03.md +15 -0
  169. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-04.md +14 -0
  170. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-05.md +15 -0
  171. package/tests/fixtures/suspect-zones/zh/ai/zh-ai-01.md +11 -0
  172. package/tests/fixtures/suspect-zones/zh/ai/zh-ai-02.md +11 -0
  173. package/tests/fixtures/suspect-zones/zh/ai/zh-ai-03.md +11 -0
  174. package/tests/fixtures/suspect-zones/zh/natural/zh-nat-01.md +11 -0
  175. package/tests/fixtures/suspect-zones/zh/natural/zh-nat-02.md +11 -0
  176. package/tests/fixtures/suspect-zones/zh/natural/zh-nat-03.md +11 -0
  177. package/tests/quality/README.md +121 -0
  178. package/tests/quality/benchmark.mjs +306 -0
  179. package/tests/quality/detectors.manual.example.json +31 -0
  180. package/tests/quality/dogfood.mjs +44 -0
@@ -0,0 +1,657 @@
1
+ ---
2
+ name: AI-Likeness Scoring Algorithm
3
+ version: 1.0.0
4
+ description: Pattern-based AI-likeness scoring reference for patina score and ouroboros modes
5
+ ---
6
+
7
+ # AI-Likeness Scoring Algorithm
8
+
9
+ Pattern-based scoring that converts AI pattern detection results into a numeric 0-100 score.
10
+ Used by `--score` mode and `--ouroboros` loop for termination gating.
11
+
12
+ ---
13
+
14
+ ## 1. Severity Scale (Per-Detection)
15
+
16
+ Severity is assigned **per detection** by the LLM during the pattern scan phase.
17
+ It is NOT intrinsic to the pattern — the same pattern may receive different severity
18
+ depending on how egregiously it appears in context.
19
+
20
+ | Level | Points | Criteria |
21
+ |-------|--------|----------|
22
+ | High | 3 | Pattern is pervasive — appears multiple times or is especially blatant |
23
+ | Medium | 2 | Pattern present at moderate frequency or impact |
24
+ | Low | 1 | Pattern barely present — isolated occurrence |
25
+ | Not detected | 0 | Pattern not found in text |
26
+
27
+ ---
28
+
29
+ ## 2. Severity Assignment Rubric
30
+
31
+ To reduce variance between runs, follow these guidelines when assigning severity:
32
+
33
+ ### General Rubric (4+ paragraph text)
34
+
35
+ | Instances | Severity |
36
+ |-----------|----------|
37
+ | 1-2 isolated occurrences | Low (1) |
38
+ | 3-5 occurrences, or concentrated in one section | Medium (2) |
39
+ | 6+ occurrences, or pervasive throughout | High (3) |
40
+
41
+ ### Special Cases
42
+
43
+ - **Structure patterns (#25-28):** Assess at document level, not instance count.
44
+ A single structural issue (e.g., every paragraph follows identical template) is High.
45
+ - **Communication patterns (#19-21):** A single clear chatbot expression
46
+ (e.g., "좋은 질문입니다!") may be High — these are strong AI signals regardless of count.
47
+ - **Short text (1-2 paragraphs):** Adjust thresholds proportionally.
48
+ 2 instances in 2 paragraphs = Medium, not Low.
49
+
50
+ ---
51
+
52
+ ## 3. Category Derivation
53
+
54
+ Categories are derived **dynamically** from pack frontmatter, not hardcoded.
55
+
56
+ ```
57
+ Category = pack frontmatter `pack` field minus language prefix
58
+ Example: pack: ko-content → category: content
59
+ pack: en-style → category: style
60
+ pack: ko-custom → category: custom
61
+
62
+ Pattern count = pack frontmatter `patterns` field
63
+ ```
64
+
65
+ If multiple packs map to the same category (e.g., `ko-content` and `custom/ko-content`),
66
+ their patterns are merged and counts summed for that category.
67
+
68
+ Unknown categories (from custom packs not in the weight config) get default weight: **0.10**.
69
+
70
+ ---
71
+
72
+ ## 4. Category Weights
73
+
74
+ ### Korean (ko)
75
+
76
+ | Category | Weight | Patterns | Notes |
77
+ |----------|--------|----------|-------|
78
+ | content | 0.18 | 6 | |
79
+ | language | 0.18 | 7 | |
80
+ | style | 0.18 | 6 | |
81
+ | communication | 0.13 | 4 | |
82
+ | filler | 0.08 | 4 | |
83
+ | structure | 0.15 | 5 | |
84
+ | viral-hook | 0.10 | 8 | score-only (no rewrite) |
85
+ | **Total** | **1.00** | **40** | |
86
+
87
+ ### English (en)
88
+
89
+ | Category | Weight | Patterns | Notes |
90
+ |----------|--------|----------|-------|
91
+ | content | 0.20 | 6 | |
92
+ | language | 0.20 | 7 | |
93
+ | style | 0.20 | 6 | |
94
+ | communication | 0.12 | 4 | |
95
+ | filler | 0.08 | 4 | |
96
+ | structure | 0.10 | 5 | |
97
+ | viral-hook | 0.10 | 8 | score-only (no rewrite) |
98
+ | **Total** | **1.00** | **40** | |
99
+
100
+ ### Chinese (zh)
101
+
102
+ | Category | Weight | Patterns | Notes |
103
+ |----------|--------|----------|-------|
104
+ | content | 0.18 | 6 | |
105
+ | language | 0.18 | 7 | |
106
+ | style | 0.18 | 6 | |
107
+ | communication | 0.13 | 4 | |
108
+ | filler | 0.08 | 4 | |
109
+ | structure | 0.15 | 5 | |
110
+ | viral-hook | 0.10 | 8 | score-only (no rewrite) |
111
+ | **Total** | **1.00** | **40** | |
112
+
113
+ ### Japanese (ja)
114
+
115
+ | Category | Weight | Patterns | Notes |
116
+ |----------|--------|----------|-------|
117
+ | content | 0.18 | 6 | |
118
+ | language | 0.18 | 7 | |
119
+ | style | 0.18 | 6 | |
120
+ | communication | 0.13 | 4 | |
121
+ | filler | 0.08 | 4 | |
122
+ | structure | 0.15 | 5 | |
123
+ | viral-hook | 0.10 | 8 | score-only (no rewrite) |
124
+ | **Total** | **1.00** | **40** | |
125
+
126
+ Weights are configurable via `ouroboros.category-weights.{lang}` in `.patina.yaml`.
127
+
128
+ **viral-hook weight review (issue #154):** kept at `0.10`. The pack remains score-only and the three added patterns improve the category denominator/granularity rather than broadening it enough to justify a larger score contribution. Rewrite, diff, and Ouroboros modes still skip the pack.
129
+
130
+ ---
131
+
132
+ ## 5. Profile Override Adjustments
133
+
134
+ Before summing severities, apply profile `pattern-overrides` modifiers:
135
+
136
+ | Override | Factor | Effect |
137
+ |----------|--------|--------|
138
+ | amplify | × 1.5 (cap at 3) | Increases severity contribution |
139
+ | reduce | × 0.5 | Decreases severity contribution |
140
+ | suppress | × 0.0 | Excludes pattern entirely |
141
+ | normal (default) | × 1.0 | No change |
142
+
143
+ Example: blog profile suppresses #14 (bold) → pattern #14 severity becomes 0,
144
+ excluded from ko-style category calculation.
145
+
146
+ ### Language-Scoped Overrides
147
+
148
+ `pattern-overrides` may be nested under a language code (`ko:`, `en:`) to avoid
149
+ cross-language number collisions (e.g., ko #8 is "~적 접미사" while en #8 is
150
+ "Copula Avoidance" — the same number refers to unrelated patterns in each language).
151
+
152
+ ```yaml
153
+ # Language-scoped format (recommended for multi-language profiles)
154
+ pattern-overrides:
155
+ ko:
156
+ 8: amplify # ko-language #8 (~적 접미사)
157
+ 14: suppress # ko-style #14 (볼드체)
158
+ en:
159
+ 8: amplify # en-language #8 (Copula Avoidance)
160
+ 14: suppress # en-style #14 (Boldface)
161
+ ```
162
+
163
+ **Resolution rule:** When the active language has a sub-section under `pattern-overrides`,
164
+ apply **only** that sub-section's overrides. Top-level (unscoped) overrides apply to all
165
+ languages and are merged before language-scoped ones (language-scoped wins on conflict).
166
+
167
+ ---
168
+
169
+ ## 6. Scoring Formula
170
+
171
+ ### Per-Category Score
172
+
173
+ ```
174
+ category_score = (sum of adjusted severities / (pattern_count × 3)) × 100
175
+ ```
176
+
177
+ - `sum of adjusted severities`: sum severity points for all detected patterns in category,
178
+ after applying profile override factors
179
+ - `pattern_count × 3`: maximum possible score (all patterns detected at High severity)
180
+ - Result: 0-100 per category
181
+
182
+ ### Overall Score
183
+
184
+ ```
185
+ overall_score = Σ(category_score × category_weight) for all categories
186
+ ```
187
+
188
+ ### Worked Example (Korean, default profile)
189
+
190
+ Input text detected patterns:
191
+
192
+ | Pattern | Category | Raw Severity | Override | Adjusted |
193
+ |---------|----------|-------------|----------|----------|
194
+ | #1 과도한 중요성 부여 | content | High (3) | normal | 3 |
195
+ | #3 피상적 분석 | content | Medium (2) | normal | 2 |
196
+ | #5 모호한 출처 | content | Low (1) | normal | 1 |
197
+ | #8 ~적 접미사 | language | Medium (2) | normal | 2 |
198
+ | #14 볼드체 | style | High (3) | normal | 3 |
199
+ | #17 이모지 | style | Medium (2) | normal | 2 |
200
+ | #23 채움 표현 | filler | Low (1) | normal | 1 |
201
+ | #25 구조적 반복 | structure | High (3) | normal | 3 |
202
+
203
+ Category scores:
204
+
205
+ | Category | Detected | Sum | Max (count×3) | Score |
206
+ |----------|----------|-----|---------------|-------|
207
+ | content | 3/6 | 3+2+1=6 | 6×3=18 | 33.3 |
208
+ | language | 1/6 | 2 | 18 | 11.1 |
209
+ | style | 2/6 | 3+2=5 | 18 | 27.8 |
210
+ | communication | 0/3 | 0 | 9 | 0.0 |
211
+ | filler | 1/3 | 1 | 9 | 11.1 |
212
+ | structure | 1/4 | 3 | 12 | 25.0 |
213
+
214
+ Overall = 33.3×0.20 + 11.1×0.20 + 27.8×0.20 + 0.0×0.15 + 11.1×0.10 + 25.0×0.15
215
+ = 6.66 + 2.22 + 5.56 + 0.00 + 1.11 + 3.75
216
+ = **19.3**
217
+
218
+ Interpretation: 16-30 range = "거의 사람다움" (Mostly human, minor traces)
219
+
220
+ ---
221
+
222
+ ## 7. Score Interpretation
223
+
224
+ | Range | Label | Meaning |
225
+ |-------|-------|---------|
226
+ | 0-15 | 사람다움 | Strongly human-like |
227
+ | 16-30 | 거의 사람다움 | Mostly human, minor AI traces |
228
+ | 31-50 | 혼재 | Mixed signals, noticeable AI patterns |
229
+ | 51-70 | AI 느낌 | Clearly AI-generated |
230
+ | 71-100 | AI 생성 | Heavily AI-generated |
231
+
232
+ > **Variance Note:** Scores have expected variance of **±8-10 points** between runs
233
+ > due to LLM severity assignment. Use score ranges, not exact numbers, for comparison.
234
+ > A score of 42 should be interpreted as "roughly 32-52 range" for decision-making.
235
+
236
+ ---
237
+
238
+ ## 8. Known Limitations
239
+
240
+ - **Custom pattern packs** are auto-discovered and scored. A new category
241
+ (e.g., `custom/patterns/ko-domain.md` with `pack: ko-domain`) gets default weight 0.10.
242
+ - **LLM non-determinism** means the same text may score differently across runs.
243
+ The formula is deterministic; the severity assignment is not.
244
+ - **Fidelity scoring** (meaning preservation vs original) is defined in §§ 9–13 below
245
+ and integrated into `--score`, `--ouroboros`, and MAX mode pipelines.
246
+
247
+ ### Short-text boost (v3.11 Phase 3.2)
248
+
249
+ For inputs ≤200 non-whitespace chars OR ≤3 non-empty paragraphs, register-
250
+ sensitive categories get a 1.5x severity multiplier (capped at 3 per detection):
251
+
252
+ - `language` — 종결어미, register cues, sentence form patterns
253
+ - `style` — connectors, transition fillers, formulaic openers
254
+ - `viral-hook` — shock numbers, clickbait closes, hyperbolic lexicon
255
+
256
+ Rationale: case-04 found that voice/register shifts (e.g., `~다` ↔ `~습니다`)
257
+ are clearly perceived by humans but barely move the standard formula because
258
+ short texts only accumulate 1–2 pattern detections. The boost surfaces those
259
+ shifts so single-paragraph score deltas align with reader intuition.
260
+
261
+ The boost is applied at severity-assignment time by the scoring LLM (per the
262
+ prompt instruction in `buildScoreInstructions`), not as a post-multiplier on
263
+ category scores. This keeps the formula in §6 unchanged.
264
+
265
+ ---
266
+
267
+ ## 9. Fidelity Scoring — Overview
268
+
269
+ AI-likeness (§§ 1–7) measures *how AI-like the output sounds*.
270
+ Fidelity measures *how faithfully the output preserves the original meaning*.
271
+
272
+ Both dimensions are necessary: aggressive humanization can achieve a low AI score
273
+ by deleting content or changing meaning entirely. Fidelity scoring guards against this.
274
+
275
+ Fidelity scoring is integrated into `--score`, `--ouroboros`, and MAX mode pipelines.
276
+ See SKILL.md § 6 (score mode) and SKILL-MAX.md § 6 for integration details.
277
+
278
+ ---
279
+
280
+ ## 10. Fidelity Criteria
281
+
282
+ Four criteria, each scored independently by the LLM comparing original → output:
283
+
284
+ ### 10.1 Claims Preserved
285
+
286
+ Every factual claim in the original appears (perhaps rephrased) in the output.
287
+
288
+ | Level | Points | Criteria |
289
+ |-------|--------|----------|
290
+ | High | 3 | All key claims preserved — no factual content lost |
291
+ | Medium | 2 | Minor claims omitted — supporting details or examples dropped, but core argument intact |
292
+ | Low | 1 | Significant claims missing — one or more central facts or arguments absent |
293
+ | Fail | 0 | Core meaning lost — the output says something fundamentally different |
294
+
295
+ ### 10.2 No Fabrication
296
+
297
+ The output does not add claims, facts, or specifics not present or implied by the original.
298
+
299
+ | Level | Points | Criteria |
300
+ |-------|--------|----------|
301
+ | High | 3 | No fabrication — every claim in the output traces to the original |
302
+ | Medium | 2 | Minor additions — a reasonable inference stated as fact, or an illustrative example added |
303
+ | Low | 1 | Noticeable fabrication — specific numbers, names, or claims not in the original |
304
+ | Fail | 0 | Significant fabrication — output contains substantial invented content |
305
+
306
+ ### 10.3 Tone Match
307
+
308
+ The output's register matches the original (or the profile's target register, if explicitly overridden).
309
+
310
+ | Level | Points | Criteria |
311
+ |-------|--------|----------|
312
+ | High | 3 | Tone matches — formality level, domain register, and audience consistent |
313
+ | Medium | 2 | Slight drift — somewhat more/less formal, but still appropriate for the context |
314
+ | Low | 1 | Noticeable mismatch — formal original made casual (or vice versa) without profile justification |
315
+ | Fail | 0 | Register violation — academic text made into slang, or casual text made into legalese |
316
+
317
+ **Profile exception:** When a profile explicitly shifts register (e.g., blog profile amplifies
318
+ informality), tone match is assessed against the *profile target*, not the original register.
319
+
320
+ ### 10.4 Length Ratio
321
+
322
+ Compares output length to original. Extreme changes suggest content loss or padding.
323
+
324
+ | Ratio | Points | Criteria |
325
+ |-------|--------|----------|
326
+ | 70–130% | 3 | Length preserved — natural variation within ±30% |
327
+ | 50–69% or 131–150% | 2 | Moderate change — some compression or expansion, likely acceptable |
328
+ | 30–49% or 151–200% | 1 | Significant change — substantial content probably lost or padded |
329
+ | < 30% or > 200% | 0 | Extreme change — content almost certainly lost or heavily padded |
330
+
331
+ **Calculation:** `length_ratio = len(output) / len(original) × 100`
332
+
333
+ Length is measured in characters (not words or tokens) for language-agnostic consistency.
334
+
335
+ ---
336
+
337
+ ## 11. Fidelity Severity Assignment Rubric
338
+
339
+ To reduce variance, apply these guidelines when scoring fidelity criteria:
340
+
341
+ ### Claims Preserved
342
+ - Count discrete factual claims in the original. If all appear (rephrased or not) → High.
343
+ - If only supporting details are dropped but the argument structure survives → Medium.
344
+ - If a numbered list loses items, a causal chain loses a step, or a key qualifier is dropped → Low.
345
+
346
+ ### No Fabrication
347
+ - Paraphrasing that changes word choice but not meaning → High.
348
+ - Adding a commonly-known context note ("Seoul, the capital of South Korea") → Medium.
349
+ - Inventing a statistic, date, or name not in the original → Low or Fail.
350
+
351
+ ### Tone Match
352
+ - Compare the first and last paragraphs of original vs. output for register cues.
353
+ - Profile-targeted register shifts are expected, not penalized.
354
+ - Mixed register (formal opening, casual middle) counts as Low.
355
+
356
+ ### Length Ratio
357
+ - This criterion is deterministic — compute the ratio and look up the table.
358
+ - No LLM judgment needed. Include the raw ratio in the score output.
359
+
360
+ ---
361
+
362
+ ## 12. Fidelity Scoring Formula
363
+
364
+ ### Per-Criterion Score
365
+
366
+ Each criterion is scored 0–3 (same as AI-likeness severity). The fidelity score normalizes
367
+ across all four criteria:
368
+
369
+ ```
370
+ fidelity_score = ((claims + fabrication + tone + length) / 12) × 100
371
+ ```
372
+
373
+ - Maximum: (3+3+3+3) / 12 × 100 = **100** (perfect fidelity)
374
+ - Minimum: (0+0+0+0) / 12 × 100 = **0** (total meaning loss)
375
+
376
+ ### Criterion Weighting (Optional)
377
+
378
+ For profiles that need non-uniform criterion importance, weights can be configured:
379
+
380
+ ```yaml
381
+ fidelity:
382
+ weights:
383
+ claims-preserved: 0.35
384
+ no-fabrication: 0.30
385
+ tone-match: 0.20
386
+ length-ratio: 0.15
387
+ ```
388
+
389
+ When weights are configured:
390
+
391
+ ```
392
+ fidelity_score = Σ((criterion_points / 3) × criterion_weight) × 100
393
+ ```
394
+
395
+ Default weights (when not configured): equal at 0.25 each (equivalent to the simple formula).
396
+
397
+ ### Worked Example
398
+
399
+ Original: 4-paragraph academic article about climate change policy.
400
+ Output: Humanized version.
401
+
402
+ | Criterion | Score | Reasoning |
403
+ |-----------|-------|-----------|
404
+ | Claims preserved | 3 (High) | All policy recommendations and cited figures present |
405
+ | No fabrication | 2 (Medium) | Added "as widely reported" — minor inference stated as fact |
406
+ | Tone match | 3 (High) | Academic register maintained throughout |
407
+ | Length ratio | 2 (Moderate) | Output is 68% of original length (within 50-69% band) |
408
+
409
+ Fidelity = (3+2+3+2) / 12 × 100 = **83.3**
410
+
411
+ Interpretation: 76-90 range = "높은 충실도" (High fidelity, minor issues)
412
+
413
+ ### Fidelity Score Interpretation
414
+
415
+ | Range | Label | Meaning |
416
+ |-------|-------|---------|
417
+ | 91-100 | 완벽한 충실도 | Perfect fidelity — all meaning preserved |
418
+ | 76-90 | 높은 충실도 | High fidelity — minor omissions or additions |
419
+ | 51-75 | 보통 충실도 | Moderate fidelity — noticeable meaning changes |
420
+ | 26-50 | 낮은 충실도 | Low fidelity — significant meaning loss or fabrication |
421
+ | 0-25 | 의미 왜곡 | Meaning severely distorted or lost |
422
+
423
+ > **Note:** Unlike AI-likeness (lower = better), fidelity uses **higher = better**.
424
+ > A score of 83 means "roughly 73-93 range" given ±10 LLM variance —
425
+ > solidly in the "높은 충실도" band.
426
+
427
+ ---
428
+
429
+ ## 13. Combined Score
430
+
431
+ ### Formula
432
+
433
+ AI-likeness and fidelity compose into a combined score with configurable weighting:
434
+
435
+ ```
436
+ combined = (ai_likeness × ai_weight) + (fidelity_inverted × fidelity_weight)
437
+ ```
438
+
439
+ Where:
440
+ - `ai_likeness`: AI-likeness score from § 6 (0-100, lower = more human)
441
+ - `fidelity_inverted`: `100 - fidelity_score` (invert so both dimensions use "lower is better")
442
+ - `ai_weight + fidelity_weight = 1.0`
443
+
444
+ ### Default Weights
445
+
446
+ | Context | AI Weight | Fidelity Weight | Rationale |
447
+ |---------|-----------|-----------------|-----------|
448
+ | Default | 0.60 | 0.40 | Balanced — humanization is primary goal |
449
+ | Academic profile | 0.40 | 0.60 | Meaning preservation is critical in scholarly work |
450
+ | Blog profile | 0.70 | 0.30 | Creative rewriting tolerated |
451
+ | Technical profile | 0.35 | 0.65 | Accuracy is paramount in docs |
452
+ | Social profile | 0.75 | 0.25 | Tone transformation expected |
453
+ | Email profile | 0.50 | 0.50 | Equal importance |
454
+ | Legal profile | 0.35 | 0.65 | Legal precision must be preserved |
455
+ | Medical profile | 0.35 | 0.65 | Clinical accuracy is critical |
456
+ | Marketing profile | 0.65 | 0.35 | Tone transformation tolerated, creative rewriting expected |
457
+
458
+ Configurable via `ouroboros.combined-weights.{profile}` in `.patina.yaml`:
459
+
460
+ ```yaml
461
+ ouroboros:
462
+ combined-weights:
463
+ default:
464
+ ai-likeness: 0.60
465
+ fidelity: 0.40
466
+ academic:
467
+ ai-likeness: 0.40
468
+ fidelity: 0.60
469
+ ```
470
+
471
+ ### Combined Score Interpretation
472
+
473
+ | Range | Label | Meaning |
474
+ |-------|-------|---------|
475
+ | 0-15 | 최적 | Excellent — human-like and faithful |
476
+ | 16-30 | 양호 | Good — minor issues in one or both dimensions |
477
+ | 31-50 | 보통 | Acceptable — noticeable trade-offs |
478
+ | 51-70 | 주의 | Caution — significant AI traces or meaning loss |
479
+ | 71-100 | 부적합 | Poor — heavy AI patterns and/or substantial meaning loss |
480
+
481
+ ### Ouroboros Termination
482
+
483
+ When used with `--ouroboros`, the loop terminates when:
484
+ - Combined score ≤ threshold (default: 30), OR
485
+ - Fidelity score drops below floor (default: 70) — **hard stop**, even if AI score improves
486
+
487
+ This prevents the ouroboros loop from "improving" AI score by destroying content.
488
+
489
+ ---
490
+
491
+ ## 14. MPS (Meaning Preservation Score) — Overview
492
+
493
+ AI-likeness (§§ 1–7) measures *how AI-like the output sounds*.
494
+ Fidelity (§§ 9–13) measures *how faithfully the output preserves overall meaning*.
495
+ MPS measures *whether specific semantic anchors survive the humanization pipeline*.
496
+
497
+ MPS is anchor-based: it tracks discrete meaning units (claims, polarity, causation, quantifiers, negations) extracted from the original text in SKILL.md Step 4.5, and checks whether each anchor is preserved after each pipeline phase.
498
+
499
+ MPS complements fidelity scoring — fidelity is a holistic LLM judgment, MPS is a structured anchor-by-anchor verification.
500
+
501
+ ---
502
+
503
+ ## 15. MPS Anchor Verification Criteria
504
+
505
+ Each anchor extracted in Step 4.5 is verified against the pipeline output. Verification produces one of three verdicts:
506
+
507
+ ### PASS
508
+ Anchor content is present in the output and its polarity is preserved. The anchor may be rephrased but its core assertion is unambiguously recoverable.
509
+
510
+ ### SOFT FAIL
511
+ Anchor content is present but weakened or made ambiguous. Examples:
512
+ - Specific claim became vague: "매출이 30% 증가" → "매출이 크게 증가"
513
+ - Quantifier lost precision: "p<0.05" → "통계적으로 유의미하다"
514
+ - Causal link became correlational: "A 때문에 B가 발생" → "A와 B는 관련이 있다"
515
+ - Definitive statement became hedged: "시스템이 실패했다" → "시스템에 문제가 있었을 수 있다"
516
+
517
+ ### HARD FAIL
518
+ Anchor content is deleted or its polarity is inverted. Examples:
519
+ - Claim removed entirely from output
520
+ - Negation dropped: "검증되지 않았다" → "검증되었다"
521
+ - Causation reversed: "A가 B를 야기했다" → "B가 A를 야기했다"
522
+
523
+ ### PASS vs SOFT FAIL Boundary
524
+
525
+ If a reader can **unambiguously recover** the original anchor's meaning from the rewritten version, it is PASS. If a reader could **reasonably interpret** the rewritten version differently from the original, it is SOFT FAIL.
526
+
527
+ ---
528
+
529
+ ## 16. MPS Scoring Formula
530
+
531
+ ### Base Formula
532
+
533
+ ```
534
+ anchor_pass_rate = PASS_count / total_anchor_count
535
+ polarity_preserved = polarity_PASS_count / total_polarity_anchor_count
536
+
537
+ MPS = (anchor_pass_rate × 0.6 + polarity_preserved × 0.4) × 100
538
+ ```
539
+
540
+ Where:
541
+ - `PASS_count`: anchors with PASS verdict after all remediation (including successful retries)
542
+ - `total_anchor_count`: all extracted anchors
543
+ - `polarity_PASS_count`: polarity-type anchors (Polarity + Negation) with PASS verdict
544
+ - `total_polarity_anchor_count`: all polarity-type anchors (Polarity + Negation)
545
+
546
+ ### Retry Counting Rule
547
+
548
+ If a SOFT FAIL anchor passes after alternative correction (retry), it counts as **PASS** in the formula. Only anchors that remain SOFT FAIL or HARD FAIL after all remediation are counted as failures.
549
+
550
+ ### Fallback (No Polarity Anchors)
551
+
552
+ If the text contains no Polarity or Negation anchors:
553
+
554
+ ```
555
+ MPS = anchor_pass_rate × 100
556
+ ```
557
+
558
+ ### Fallback (No Anchors Extracted)
559
+
560
+ When anchor extraction is skipped (text ≤1 paragraph and ≤2 sentences) or yields zero anchors:
561
+
562
+ ```
563
+ MPS = N/A (not applicable)
564
+ ```
565
+
566
+ When MPS = N/A:
567
+ - `--score` mode displays: `의미 보존 (MPS): N/A (앵커 없음)`
568
+ - Ouroboros loop: MPS floor check is bypassed (only fidelity floor applies)
569
+ - MAX mode: MPS gate is bypassed (selection uses AI score only)
570
+
571
+ ### MPS Interpretation
572
+
573
+ | Range | Label | Meaning |
574
+ |-------|-------|---------|
575
+ | 90–100 | 우수 (Excellent) | Full meaning preservation |
576
+ | 70–89 | 양호 (Good) | Minor weakening, acceptable |
577
+ | 50–69 | 주의 (Warning) | Significant anchor loss, review needed |
578
+ | < 50 | 위험 (Critical) | Severe meaning corruption |
579
+
580
+ ### Worked Example
581
+
582
+ Original text (3 paragraphs) with extracted anchors:
583
+
584
+ | # | Type | Content | Verdict | After Retry |
585
+ |---|------|---------|---------|-------------|
586
+ | 1 | Claim | "시스템이 실패했다" | SOFT FAIL | PASS (retry succeeded) |
587
+ | 2 | Polarity | "아직 검증되지 않았다" (negative) | PASS | — |
588
+ | 3 | Quantifier | "매출 30% 증가" | PASS | — |
589
+ | 4 | Causation | "A 때문에 B 발생" | HARD FAIL | — (original restored) |
590
+ | 5 | Negation | "불가능하다" | PASS | — |
591
+
592
+ After remediation:
593
+ - PASS: #1 (retry), #2, #3, #5 = 4
594
+ - HARD FAIL: #4 = 1 (original restored, so meaning is preserved in output but pattern not humanized)
595
+ - Total anchors: 5
596
+ - Polarity anchors (#2, #5): both PASS = 2/2
597
+
598
+ ```
599
+ anchor_pass_rate = 4/5 = 0.80
600
+ polarity_preserved = 2/2 = 1.00
601
+ MPS = (0.80 × 0.6 + 1.00 × 0.4) × 100 = (0.48 + 0.40) × 100 = 88
602
+ ```
603
+
604
+ Interpretation: 70–89 range = "양호" (Good, minor weakening)
605
+
606
+ > **Note:** Anchor #4 was a HARD FAIL, so its original sentence was restored in the output.
607
+ > The anchor counts as a failure in MPS (reducing the score), but the meaning IS preserved
608
+ > in the output because the original was kept. MPS reflects humanization success rate,
609
+ > not output meaning accuracy (which is always preserved via fallback).
610
+
611
+ ### MPS vs Fidelity: Complementary Metrics
612
+
613
+ MPS measures **humanization coverage** — what fraction of meaning anchors were successfully humanized while being preserved. A HARD FAIL anchor that was restored to its original wording counts as a humanization failure (the pattern wasn't removable without meaning loss), even though the final output's meaning is intact. Fidelity (§§ 9-13) measures **overall output meaning accuracy** against the original — restored sentences score perfectly on fidelity. Use both metrics together: high fidelity + low MPS means "meaning is safe but some AI patterns couldn't be removed."
614
+
615
+ ---
616
+
617
+ ## 17. MPS Integration Points
618
+
619
+ ### `--score` Mode Output
620
+
621
+ When `--score` is used with rewrite or ouroboros mode (original text available),
622
+ MPS is displayed alongside AI-likeness and Fidelity:
623
+
624
+ | 지표 | 점수 |
625
+ |------|------|
626
+ | AI 유사도 | 23/100 (낮을수록 좋음) |
627
+ | 충실도 | 87/100 (높을수록 좋음) |
628
+ | 의미 보존 (MPS) | 92/100 (높을수록 좋음) |
629
+ | 종합 | 25/100 (낮을수록 좋음) |
630
+
631
+ > **Note:** MPS is NOT included in the combined score formula (§13).
632
+ > Combined score uses fidelity (holistic) while MPS is a structural verification metric.
633
+ > Both are displayed for transparency but serve different purposes.
634
+
635
+ ### Ouroboros Loop Gating
636
+
637
+ MPS floor = 70 (default). Independent of fidelity floor.
638
+
639
+ Termination condition:
640
+ - MPS < mps-floor → terminate with reason: **의미 보존 하한 위반** → rollback to previous iteration
641
+
642
+ Both fidelity floor AND MPS floor must pass for an iteration to be accepted.
643
+
644
+ Configurable via `.patina.yaml`:
645
+
646
+ ```yaml
647
+ ouroboros:
648
+ mps-floor: 70 # default
649
+ ```
650
+
651
+ ### MAX Mode Candidate Selection
652
+
653
+ Current: Select candidate with lowest AI score.
654
+ New: Select candidate with lowest AI score **WHERE MPS ≥ 70**.
655
+
656
+ Candidates with MPS < 70 are disqualified regardless of AI score.
657
+ If ALL candidates have MPS < 70, select the one with the highest MPS (least meaning loss).