patina-cli 3.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (180) hide show
  1. package/.patina.default.yaml +211 -0
  2. package/CHANGELOG.md +265 -0
  3. package/LICENSE +21 -0
  4. package/README.md +319 -0
  5. package/README_JA.md +254 -0
  6. package/README_KR.md +253 -0
  7. package/README_ZH.md +254 -0
  8. package/SKILL-MAX.md +455 -0
  9. package/SKILL.md +730 -0
  10. package/assets/brand/patina-icon.svg +9 -0
  11. package/assets/brand/patina-logo.svg +17 -0
  12. package/assets/social/patina-before-after.svg +46 -0
  13. package/assets/social/patina-og.svg +31 -0
  14. package/bin/patina.js +9 -0
  15. package/core/scoring.md +657 -0
  16. package/core/standalone-prompt.md +364 -0
  17. package/core/stylometry.md +754 -0
  18. package/core/voice.md +163 -0
  19. package/docs/AUTHENTICATION.md +105 -0
  20. package/docs/AUTHENTICATION_KR.md +105 -0
  21. package/docs/BRANDING.md +37 -0
  22. package/docs/CLI.md +80 -0
  23. package/docs/COMPARISON.md +38 -0
  24. package/docs/COOKBOOK.md +173 -0
  25. package/docs/DEMO.md +40 -0
  26. package/docs/ETHICS.md +27 -0
  27. package/docs/EXAMPLES.md +130 -0
  28. package/docs/EXAMPLES_KR.md +130 -0
  29. package/docs/EXIT-CODES.md +25 -0
  30. package/docs/FAQ.md +67 -0
  31. package/docs/FAQ_KR.md +65 -0
  32. package/docs/FLAG-PARITY.md +53 -0
  33. package/docs/GLOSSARY.md +123 -0
  34. package/docs/PATTERNS-EN.md +718 -0
  35. package/docs/PATTERNS-JA.md +706 -0
  36. package/docs/PATTERNS-KO.md +707 -0
  37. package/docs/PATTERNS-ZH.md +706 -0
  38. package/docs/PATTERNS.md +22 -0
  39. package/docs/ROADMAP.md +315 -0
  40. package/docs/audits/2026-05-deep-research.md +290 -0
  41. package/docs/benchmarks/detector-comparison.json +442 -0
  42. package/docs/benchmarks/detector-comparison.md +65 -0
  43. package/docs/benchmarks/latest.json +988 -0
  44. package/docs/benchmarks/latest.md +112 -0
  45. package/docs/integrations/docker.md +19 -0
  46. package/docs/integrations/github-action.md +59 -0
  47. package/docs/integrations/pre-commit.md +77 -0
  48. package/docs/integrations/release.md +43 -0
  49. package/docs/internal/HARNESS.md +14 -0
  50. package/docs/internal/README.md +14 -0
  51. package/docs/internal/WARP.md +23 -0
  52. package/docs/research/2025-rebaseline-plan.md +89 -0
  53. package/docs/research/ai-human-metrics.md +380 -0
  54. package/docs/social/gstack-cardnews.html +236 -0
  55. package/docs/social/gstack-cardnews.md +88 -0
  56. package/docs/social/gstack-thread.md +106 -0
  57. package/docs/social/patina-launch-copy.md +227 -0
  58. package/docs/superpowers/specs/2026-04-03-meaning-preservation-design.md +299 -0
  59. package/lexicon/ai-en.md +162 -0
  60. package/lexicon/ai-ko.md +159 -0
  61. package/package.json +100 -0
  62. package/patina-max/SKILL.md +523 -0
  63. package/patina-max/composite.py +457 -0
  64. package/patterns/en-communication.md +89 -0
  65. package/patterns/en-content.md +133 -0
  66. package/patterns/en-filler.md +113 -0
  67. package/patterns/en-language.md +163 -0
  68. package/patterns/en-structure.md +173 -0
  69. package/patterns/en-style.md +139 -0
  70. package/patterns/en-viral-hook.md +211 -0
  71. package/patterns/ja-communication.md +101 -0
  72. package/patterns/ja-content.md +153 -0
  73. package/patterns/ja-filler.md +123 -0
  74. package/patterns/ja-language.md +190 -0
  75. package/patterns/ja-structure.md +142 -0
  76. package/patterns/ja-style.md +147 -0
  77. package/patterns/ja-viral-hook.md +216 -0
  78. package/patterns/ko-communication.md +98 -0
  79. package/patterns/ko-content.md +154 -0
  80. package/patterns/ko-filler.md +105 -0
  81. package/patterns/ko-language.md +182 -0
  82. package/patterns/ko-structure.md +147 -0
  83. package/patterns/ko-style.md +146 -0
  84. package/patterns/ko-viral-hook.md +211 -0
  85. package/patterns/zh-communication.md +101 -0
  86. package/patterns/zh-content.md +153 -0
  87. package/patterns/zh-filler.md +118 -0
  88. package/patterns/zh-language.md +173 -0
  89. package/patterns/zh-structure.md +145 -0
  90. package/patterns/zh-style.md +159 -0
  91. package/patterns/zh-viral-hook.md +216 -0
  92. package/profiles/academic.md +53 -0
  93. package/profiles/blog.md +81 -0
  94. package/profiles/casual-conversation.md +105 -0
  95. package/profiles/code-comment.md +104 -0
  96. package/profiles/commit-message.md +99 -0
  97. package/profiles/default.md +62 -0
  98. package/profiles/email.md +52 -0
  99. package/profiles/formal.md +98 -0
  100. package/profiles/instructional.md +80 -0
  101. package/profiles/legal.md +57 -0
  102. package/profiles/marketing.md +56 -0
  103. package/profiles/medical.md +53 -0
  104. package/profiles/narrative.md +79 -0
  105. package/profiles/release-notes.md +98 -0
  106. package/profiles/social.md +56 -0
  107. package/profiles/technical.md +53 -0
  108. package/scripts/benchmark-report.mjs +252 -0
  109. package/scripts/check-release-metadata.mjs +48 -0
  110. package/scripts/detector-comparison.mjs +267 -0
  111. package/scripts/lint.mjs +40 -0
  112. package/scripts/precommit-score.mjs +31 -0
  113. package/scripts/prose-score.mjs +186 -0
  114. package/scripts/update-benchmark-ranges.mjs +108 -0
  115. package/src/api.js +330 -0
  116. package/src/auth.js +105 -0
  117. package/src/backends/claude-cli.js +112 -0
  118. package/src/backends/codex-cli.js +121 -0
  119. package/src/backends/contract.js +21 -0
  120. package/src/backends/gemini-cli.js +135 -0
  121. package/src/backends/index.js +159 -0
  122. package/src/cache.js +106 -0
  123. package/src/cli.js +1280 -0
  124. package/src/commands/doctor.js +229 -0
  125. package/src/commands/init.js +208 -0
  126. package/src/config.js +126 -0
  127. package/src/errors.js +53 -0
  128. package/src/features/index.js +96 -0
  129. package/src/features/lexicon.js +90 -0
  130. package/src/features/segment.js +49 -0
  131. package/src/features/stylometry.js +50 -0
  132. package/src/loader.js +103 -0
  133. package/src/logger.js +70 -0
  134. package/src/manifest.js +162 -0
  135. package/src/max-mode.js +207 -0
  136. package/src/ouroboros.js +233 -0
  137. package/src/output.js +480 -0
  138. package/src/prompt-builder.js +409 -0
  139. package/src/providers.js +100 -0
  140. package/src/scoring.js +531 -0
  141. package/src/security.js +133 -0
  142. package/tests/fixtures/suspect-zones/en/ai/en-ai-01.md +16 -0
  143. package/tests/fixtures/suspect-zones/en/ai/en-ai-02.md +16 -0
  144. package/tests/fixtures/suspect-zones/en/ai/en-ai-03.md +17 -0
  145. package/tests/fixtures/suspect-zones/en/ai/en-ai-04.md +15 -0
  146. package/tests/fixtures/suspect-zones/en/ai/en-ai-05.md +16 -0
  147. package/tests/fixtures/suspect-zones/en/ai/en-ai-06-chat-register.md +16 -0
  148. package/tests/fixtures/suspect-zones/en/natural/en-nat-01.md +15 -0
  149. package/tests/fixtures/suspect-zones/en/natural/en-nat-02.md +15 -0
  150. package/tests/fixtures/suspect-zones/en/natural/en-nat-03.md +15 -0
  151. package/tests/fixtures/suspect-zones/en/natural/en-nat-04.md +15 -0
  152. package/tests/fixtures/suspect-zones/en/natural/en-nat-05.md +15 -0
  153. package/tests/fixtures/suspect-zones/expected-ranges.json +939 -0
  154. package/tests/fixtures/suspect-zones/ja/ai/ja-ai-01.md +11 -0
  155. package/tests/fixtures/suspect-zones/ja/ai/ja-ai-02.md +11 -0
  156. package/tests/fixtures/suspect-zones/ja/ai/ja-ai-03.md +11 -0
  157. package/tests/fixtures/suspect-zones/ja/natural/ja-nat-01.md +11 -0
  158. package/tests/fixtures/suspect-zones/ja/natural/ja-nat-02.md +11 -0
  159. package/tests/fixtures/suspect-zones/ja/natural/ja-nat-03.md +11 -0
  160. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-01.md +14 -0
  161. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-02.md +16 -0
  162. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-03.md +15 -0
  163. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-04.md +15 -0
  164. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-05.md +16 -0
  165. package/tests/fixtures/suspect-zones/ko/ai/ko-ai-06-chat-register.md +16 -0
  166. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-01.md +15 -0
  167. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-02.md +15 -0
  168. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-03.md +15 -0
  169. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-04.md +14 -0
  170. package/tests/fixtures/suspect-zones/ko/natural/ko-nat-05.md +15 -0
  171. package/tests/fixtures/suspect-zones/zh/ai/zh-ai-01.md +11 -0
  172. package/tests/fixtures/suspect-zones/zh/ai/zh-ai-02.md +11 -0
  173. package/tests/fixtures/suspect-zones/zh/ai/zh-ai-03.md +11 -0
  174. package/tests/fixtures/suspect-zones/zh/natural/zh-nat-01.md +11 -0
  175. package/tests/fixtures/suspect-zones/zh/natural/zh-nat-02.md +11 -0
  176. package/tests/fixtures/suspect-zones/zh/natural/zh-nat-03.md +11 -0
  177. package/tests/quality/README.md +121 -0
  178. package/tests/quality/benchmark.mjs +306 -0
  179. package/tests/quality/detectors.manual.example.json +31 -0
  180. package/tests/quality/dogfood.mjs +44 -0
package/src/scoring.js ADDED
@@ -0,0 +1,531 @@
1
+ import { callLLM as defaultCallLLM } from './api.js';
2
+ import { getRepoRoot } from './config.js';
3
+ import { analyzeText } from './features/index.js';
4
+ import { createLogger } from './logger.js';
5
+
6
+ export const DEFAULT_DETERMINISTIC_DIVERGENCE_THRESHOLD = 20;
7
+
8
+ class SchemaError extends Error {
9
+ constructor(message, raw) {
10
+ super(message);
11
+ this.name = 'SchemaError';
12
+ this.raw = raw;
13
+ }
14
+ }
15
+
16
+ function parseStrictJson(text) {
17
+ if (!text) throw new SchemaError('Empty response', text);
18
+
19
+ let body = text;
20
+ const codeBlockMatch = text.match(/```(?:json)?\s*([\s\S]*?)\s*```/);
21
+ if (codeBlockMatch) body = codeBlockMatch[1];
22
+ body = body.trim();
23
+
24
+ const firstBrace = body.indexOf('{');
25
+ const lastBrace = body.lastIndexOf('}');
26
+ if (firstBrace === -1 || lastBrace === -1 || lastBrace <= firstBrace) {
27
+ throw new SchemaError('No JSON object found', text);
28
+ }
29
+
30
+ try {
31
+ return JSON.parse(body.slice(firstBrace, lastBrace + 1));
32
+ } catch (e) {
33
+ throw new SchemaError(`JSON parse failed: ${e.message}`, text);
34
+ }
35
+ }
36
+
37
+ // Call LLM and parse strict JSON. On schema failure, retry once at temperature 0.
38
+ async function callAndParseJson({
39
+ prompt,
40
+ apiKey,
41
+ baseURL,
42
+ model,
43
+ temperature = 0.1,
44
+ deadline,
45
+ signal,
46
+ callLLM = defaultCallLLM,
47
+ logger = createLogger(),
48
+ now,
49
+ sleep,
50
+ }) {
51
+ let lastError;
52
+ for (let attempt = 0; attempt < 2; attempt++) {
53
+ const t = attempt === 0 ? temperature : 0;
54
+ const result = await callLLM({
55
+ prompt,
56
+ apiKey,
57
+ baseURL,
58
+ model,
59
+ temperature: t,
60
+ deadline,
61
+ signal,
62
+ now,
63
+ sleep,
64
+ });
65
+ try {
66
+ return { parsed: parseStrictJson(result), raw: result };
67
+ } catch (e) {
68
+ lastError = e;
69
+ if (attempt === 0) {
70
+ logger.warn('score.json_parse_retry', {
71
+ message: `[patina] score JSON parse failed (${e.message}); retrying at temperature 0`,
72
+ });
73
+ }
74
+ }
75
+ }
76
+ throw lastError;
77
+ }
78
+
79
+ export async function scoreText({
80
+ text,
81
+ config,
82
+ patterns,
83
+ apiKey,
84
+ baseURL,
85
+ model,
86
+ deadline,
87
+ signal,
88
+ callLLM = defaultCallLLM,
89
+ logger = createLogger(),
90
+ now,
91
+ sleep,
92
+ }) {
93
+ const lang = config.language || 'ko';
94
+ const weights = config.ouroboros?.['category-weights']?.[lang] || {};
95
+ const deterministicScore = scoreDeterministicSignals({ text, config });
96
+
97
+ const prompt = `You are an AI-likeness scoring engine. Score the following text for AI-writing patterns.
98
+
99
+ ## Scoring Rules
100
+
101
+ Severity per detection: Low=1, Medium=2, High=3 points.
102
+
103
+ Category weights for ${lang}:
104
+ ${Object.entries(weights).map(([cat, w]) => `- ${cat}: ${w}`).join('\n')}
105
+
106
+ Per-category score = (sum of adjusted severities / (pattern_count × 3)) × 100
107
+ Overall = weighted average of category scores.
108
+
109
+ ## Output Format (strict)
110
+
111
+ Return ONLY a JSON object in this exact format (no markdown, no explanation):
112
+
113
+ {
114
+ "categories": {
115
+ "content": {"detected": 0, "sum": 0, "max": 18, "score": 0.0, "weighted": 0.0},
116
+ ...
117
+ },
118
+ "overall": 0.0,
119
+ "interpretation": "human | mostly human | mixed | AI-like | heavily AI"
120
+ }
121
+
122
+ ## Text to Score
123
+
124
+ ${text}
125
+ `;
126
+
127
+ try {
128
+ const { parsed } = await callAndParseJson({
129
+ prompt,
130
+ apiKey,
131
+ baseURL,
132
+ model,
133
+ deadline,
134
+ signal,
135
+ callLLM,
136
+ logger,
137
+ now,
138
+ sleep,
139
+ });
140
+ return withShadowScore(parsed, { deterministicScore, config, logger });
141
+ } catch (e) {
142
+ rethrowIfAborted(e, signal);
143
+ logger.warn('score.text_schema_failure', {
144
+ message: `[patina] scoreText schema failure after retry: ${e.message}`,
145
+ });
146
+ return {
147
+ overall: null,
148
+ llmScore: { overall: null, interpretation: null, error: 'schema-failure' },
149
+ deterministicScore,
150
+ error: 'schema-failure',
151
+ raw: e.raw,
152
+ };
153
+ }
154
+ }
155
+
156
+ export function scoreDeterministicSignals({
157
+ text,
158
+ config = {},
159
+ repoRoot = getRepoRoot(),
160
+ analyzer = analyzeText,
161
+ } = {}) {
162
+ const options = deterministicScoringOptions(config);
163
+ if (!options.enabled) return null;
164
+
165
+ const lang = config.language || 'ko';
166
+ const enabledLanguages = config.stylometry?.languages;
167
+ if (Array.isArray(enabledLanguages) && !enabledLanguages.includes(lang)) {
168
+ return {
169
+ overall: null,
170
+ interpretation: null,
171
+ skipped: true,
172
+ skipReason: 'language-disabled',
173
+ paragraphCount: 0,
174
+ hotParagraphs: 0,
175
+ bands: emptyDeterministicBands(),
176
+ };
177
+ }
178
+
179
+ try {
180
+ const result = analyzer(String(text || ''), {
181
+ lang,
182
+ repoRoot,
183
+ burstinessBands: config.stylometry?.burstiness?.bands,
184
+ mattrBands: config.stylometry?.ttr?.bands,
185
+ mattrWindow: config.stylometry?.ttr?.window,
186
+ lexiconDensityThreshold: config.lexicon?.density_threshold,
187
+ });
188
+ const paragraphs = Array.isArray(result?.paragraphs) ? result.paragraphs : [];
189
+ const paragraphCount = paragraphs.length;
190
+ const hotParagraphs = paragraphs.filter((p) => p.hot).length;
191
+ const overall = paragraphCount > 0 ? roundScore((hotParagraphs / paragraphCount) * 100) : 0;
192
+
193
+ return {
194
+ overall,
195
+ interpretation: interpretScore(overall),
196
+ skipped: Boolean(result?.skipped),
197
+ skipReason: result?.skipReason ?? null,
198
+ paragraphCount,
199
+ hotParagraphs,
200
+ bands: {
201
+ burstiness: countBands(paragraphs.map((p) => p.burstiness?.band)),
202
+ mattr: countBands(paragraphs.map((p) => p.mattr?.band)),
203
+ lexicon: {
204
+ hot: paragraphs.filter((p) => p.lexicon?.hot).length,
205
+ threshold: config.lexicon?.density_threshold ?? null,
206
+ },
207
+ },
208
+ };
209
+ } catch (err) {
210
+ return {
211
+ overall: null,
212
+ interpretation: null,
213
+ skipped: true,
214
+ skipReason: 'deterministic-failure',
215
+ paragraphCount: 0,
216
+ hotParagraphs: 0,
217
+ bands: emptyDeterministicBands(),
218
+ error: err?.message || 'deterministic scoring failed',
219
+ };
220
+ }
221
+ }
222
+
223
+ export function withShadowScore(parsed, { deterministicScore, config = {}, logger } = {}) {
224
+ const llmOverall = toFiniteScore(parsed?.overall);
225
+ const llmScore = {
226
+ overall: llmOverall,
227
+ interpretation: parsed?.interpretation ?? (llmOverall === null ? null : interpretScore(llmOverall)),
228
+ categories: parsed?.categories ?? null,
229
+ };
230
+ const reconciliation = reconcileScoreOverall({
231
+ llmOverall,
232
+ deterministicScore,
233
+ config,
234
+ logger,
235
+ });
236
+ const overall = reconciliation.overall ?? llmOverall;
237
+ return {
238
+ ...parsed,
239
+ overall,
240
+ interpretation: overall === null
241
+ ? parsed?.interpretation ?? null
242
+ : interpretScore(overall),
243
+ llmScore,
244
+ deterministicScore,
245
+ ...(reconciliation.scorePreference ? { scorePreference: reconciliation.scorePreference } : {}),
246
+ };
247
+ }
248
+
249
+ export function reconcileScoreOverall({
250
+ llmOverall,
251
+ deterministicScore,
252
+ config = {},
253
+ logger,
254
+ } = {}) {
255
+ const llm = toFiniteScore(llmOverall);
256
+ const deterministic = toFiniteScore(deterministicScore?.overall);
257
+ if (llm === null) return { overall: null, scorePreference: null };
258
+ if (deterministic === null) return { overall: llm, scorePreference: null };
259
+ if (deterministicScore?.skipped) return { overall: llm, scorePreference: null };
260
+
261
+ const threshold = deterministicScoringOptions(config).divergenceThreshold;
262
+ const delta = Math.abs(llm - deterministic);
263
+ if (delta <= threshold) return { overall: llm, scorePreference: null };
264
+
265
+ const overall = Math.max(llm, deterministic);
266
+ const selected = overall === deterministic ? 'deterministic' : 'llm';
267
+ const scorePreference = {
268
+ reason: 'deterministic-divergence',
269
+ selected,
270
+ threshold,
271
+ llmOverall: llm,
272
+ deterministicOverall: deterministic,
273
+ overall,
274
+ };
275
+ logger?.warn?.('score.deterministic_divergence', {
276
+ message: `[patina] deterministic score diverged from LLM score (${llm} vs ${deterministic}); using pessimistic ${overall}`,
277
+ llm_overall: llm,
278
+ deterministic_overall: deterministic,
279
+ selected,
280
+ });
281
+ return { overall, scorePreference };
282
+ }
283
+
284
+ export async function scoreMPS({
285
+ original,
286
+ rewritten,
287
+ apiKey,
288
+ baseURL,
289
+ model,
290
+ deadline,
291
+ signal,
292
+ callLLM = defaultCallLLM,
293
+ logger = createLogger(),
294
+ now,
295
+ sleep,
296
+ }) {
297
+ const prompt = `You are a Meaning Preservation evaluator. Compare the ORIGINAL text with the REWRITTEN text.
298
+
299
+ Extract semantic anchors from the original (claims, polarity, causation, quantifiers, negations) and check if each is preserved in the rewritten text.
300
+
301
+ Verdict per anchor: PASS | SOFT_FAIL | HARD_FAIL
302
+
303
+ Return ONLY a JSON object:
304
+
305
+ {
306
+ "anchors": [
307
+ {"type": "claim", "content": "...", "verdict": "PASS"}
308
+ ],
309
+ "pass_count": 0,
310
+ "total_count": 0,
311
+ "polarity_pass_count": 0,
312
+ "polarity_total_count": 0,
313
+ "mps": 0.0
314
+ }
315
+
316
+ MPS formula: (pass_rate × 0.6 + polarity_preserved × 0.4) × 100
317
+ If no polarity anchors: MPS = pass_rate × 100
318
+
319
+ ## Original
320
+
321
+ ${original}
322
+
323
+ ## Rewritten
324
+
325
+ ${rewritten}
326
+ `;
327
+
328
+ try {
329
+ const { parsed } = await callAndParseJson({
330
+ prompt,
331
+ apiKey,
332
+ baseURL,
333
+ model,
334
+ deadline,
335
+ signal,
336
+ callLLM,
337
+ logger,
338
+ now,
339
+ sleep,
340
+ });
341
+ return parsed;
342
+ } catch (e) {
343
+ rethrowIfAborted(e, signal);
344
+ logger.warn('score.mps_schema_failure', {
345
+ message: `[patina] scoreMPS schema failure after retry: ${e.message}`,
346
+ });
347
+ return { mps: null, error: 'schema-failure', raw: e.raw };
348
+ }
349
+ }
350
+
351
+ export function interpretScore(score) {
352
+ if (score <= 15) return 'human';
353
+ if (score <= 30) return 'mostly human';
354
+ if (score <= 50) return 'mixed';
355
+ if (score <= 70) return 'AI-like';
356
+ return 'heavily AI';
357
+ }
358
+
359
+ // Length ratio is deterministic — bucket per core/scoring.md §10.4.
360
+ export function lengthRatioPoints(original, rewritten) {
361
+ if (!original || original.length === 0) return 3;
362
+ const ratio = (rewritten.length / original.length) * 100;
363
+ if (ratio >= 70 && ratio <= 130) return 3;
364
+ if ((ratio >= 50 && ratio < 70) || (ratio > 130 && ratio <= 150)) return 2;
365
+ if ((ratio >= 30 && ratio < 50) || (ratio > 150 && ratio <= 200)) return 1;
366
+ return 0;
367
+ }
368
+
369
+ export async function scoreFidelity({
370
+ original,
371
+ rewritten,
372
+ apiKey,
373
+ baseURL,
374
+ model,
375
+ deadline,
376
+ signal,
377
+ callLLM = defaultCallLLM,
378
+ logger = createLogger(),
379
+ now,
380
+ sleep,
381
+ }) {
382
+ // Length is deterministic; only ask LLM for the three judgment criteria.
383
+ const lengthPoints = lengthRatioPoints(original, rewritten);
384
+ const lengthRatio = original ? Math.round((rewritten.length / original.length) * 100) : 100;
385
+
386
+ const prompt = `You are a Fidelity evaluator. Compare ORIGINAL vs REWRITTEN text and score three criteria.
387
+
388
+ Each criterion: 0-3 points. High=3 (preserved), Medium=2 (minor drift), Low=1 (noticeable drift), Fail=0 (broken).
389
+
390
+ Criteria:
391
+ 1. claims_preserved — every factual claim in ORIGINAL appears (perhaps rephrased) in REWRITTEN.
392
+ 2. no_fabrication — REWRITTEN does not add claims/facts not present in ORIGINAL.
393
+ 3. tone_match — register/formality of REWRITTEN matches ORIGINAL.
394
+
395
+ Return ONLY this JSON, no markdown:
396
+
397
+ {
398
+ "claims_preserved": 0,
399
+ "no_fabrication": 0,
400
+ "tone_match": 0,
401
+ "rationale": "one sentence per criterion"
402
+ }
403
+
404
+ ## ORIGINAL
405
+
406
+ ${original}
407
+
408
+ ## REWRITTEN
409
+
410
+ ${rewritten}
411
+ `;
412
+
413
+ let parsed = null;
414
+ let schemaError = null;
415
+ try {
416
+ const result = await callAndParseJson({
417
+ prompt,
418
+ apiKey,
419
+ baseURL,
420
+ model,
421
+ deadline,
422
+ signal,
423
+ callLLM,
424
+ logger,
425
+ now,
426
+ sleep,
427
+ });
428
+ parsed = result.parsed;
429
+ } catch (e) {
430
+ rethrowIfAborted(e, signal);
431
+ logger.warn('score.fidelity_schema_failure', {
432
+ message: `[patina] scoreFidelity schema failure after retry: ${e.message}`,
433
+ });
434
+ schemaError = e;
435
+ }
436
+
437
+ const claims = clamp03(parsed?.claims_preserved);
438
+ const noFab = clamp03(parsed?.no_fabrication);
439
+ const tone = clamp03(parsed?.tone_match);
440
+ const fidelity = ((claims + noFab + tone + lengthPoints) / 12) * 100;
441
+
442
+ return {
443
+ criteria: {
444
+ claims_preserved: claims,
445
+ no_fabrication: noFab,
446
+ tone_match: tone,
447
+ length_ratio: lengthPoints,
448
+ },
449
+ length_ratio_pct: lengthRatio,
450
+ rationale: parsed?.rationale ?? null,
451
+ fidelity: Math.round(fidelity * 10) / 10,
452
+ ...(schemaError ? { error: 'schema-failure', raw: schemaError.raw } : {}),
453
+ };
454
+ }
455
+
456
+ export function clamp03(v) {
457
+ const n = Number(v);
458
+ if (!Number.isFinite(n)) return 0;
459
+ if (n < 0) return 0;
460
+ if (n > 3) return 3;
461
+ return Math.round(n);
462
+ }
463
+
464
+ function rethrowIfAborted(err, signal) {
465
+ if (signal?.aborted || err?.name === 'AbortError') throw err;
466
+ }
467
+
468
+ // Combined score per core/scoring.md §13: AI-likeness × ai_weight + (100 - fidelity) × fidelity_weight.
469
+ // Lower is better. Falls back to default weights if profile not configured.
470
+ export function combinedScore({ aiLikeness, fidelity, profile, config, deterministicScore }) {
471
+ const profileWeights = config?.ouroboros?.['combined-weights']?.[profile];
472
+ const ai = profileWeights?.['ai-likeness'] ?? 0.6;
473
+ const fid = profileWeights?.fidelity ?? 0.4;
474
+ const deterministicWeight = deterministicScoringOptions(config).combinedWeight;
475
+ const deterministic = toFiniteScore(deterministicScore?.overall ?? deterministicScore);
476
+ const fidelityInverted = 100 - fidelity;
477
+ if (deterministicWeight > 0 && deterministic !== null) {
478
+ const totalWeight = ai + fid + deterministicWeight;
479
+ return roundScore(
480
+ (aiLikeness * ai + fidelityInverted * fid + deterministic * deterministicWeight) /
481
+ totalWeight
482
+ );
483
+ }
484
+ return roundScore(aiLikeness * ai + fidelityInverted * fid);
485
+ }
486
+
487
+ function deterministicScoringOptions(config = {}) {
488
+ const cfg = config.scoring?.deterministic || {};
489
+ const enabled = cfg.enabled !== false;
490
+ const divergenceThreshold = Math.max(0, positiveNumber(
491
+ cfg['divergence-threshold'] ?? cfg.divergenceThreshold,
492
+ DEFAULT_DETERMINISTIC_DIVERGENCE_THRESHOLD
493
+ ));
494
+ const combinedWeight = Math.max(0, positiveNumber(
495
+ cfg['combined-weight'] ?? cfg.combinedWeight,
496
+ 0
497
+ ));
498
+ return { enabled, divergenceThreshold, combinedWeight };
499
+ }
500
+
501
+ function positiveNumber(value, fallback) {
502
+ const n = Number(value);
503
+ return Number.isFinite(n) ? n : fallback;
504
+ }
505
+
506
+ function countBands(values) {
507
+ const counts = { low: 0, mid: 0, high: 0, null: 0 };
508
+ for (const value of values) {
509
+ if (value === 'low' || value === 'mid' || value === 'high') counts[value]++;
510
+ else counts.null++;
511
+ }
512
+ return counts;
513
+ }
514
+
515
+ function emptyDeterministicBands() {
516
+ return {
517
+ burstiness: { low: 0, mid: 0, high: 0, null: 0 },
518
+ mattr: { low: 0, mid: 0, high: 0, null: 0 },
519
+ lexicon: { hot: 0, threshold: null },
520
+ };
521
+ }
522
+
523
+ function toFiniteScore(value) {
524
+ if (value === null || value === undefined || value === '') return null;
525
+ const n = Number(value);
526
+ return Number.isFinite(n) ? n : null;
527
+ }
528
+
529
+ function roundScore(value) {
530
+ return Math.round(value * 10) / 10;
531
+ }
@@ -0,0 +1,133 @@
1
+ // Boundary validation for untrusted CLI/env input.
2
+ //
3
+ // Profile names come from --profile and from .patina.yaml. Base URLs come from
4
+ // --base-url, PATINA_API_BASE, and provider presets. Both are sent into either
5
+ // fs.readFileSync or fetch() with the API key attached, so they need to be
6
+ // validated before use.
7
+ import { inputError } from './errors.js';
8
+
9
+ const PROFILE_NAME_RE = /^[A-Za-z0-9_][A-Za-z0-9_-]*$/;
10
+
11
+ export function validateProfileName(name) {
12
+ if (typeof name !== 'string' || !PROFILE_NAME_RE.test(name)) {
13
+ throw inputError(
14
+ `Invalid profile name: ${JSON.stringify(name)}`,
15
+ 'Profile names may only contain letters, numbers, underscore, and hyphen, and cannot contain slashes or "..".',
16
+ 'Run `patina --help` to see profile examples.'
17
+ );
18
+ }
19
+ }
20
+
21
+ export function isLoopbackHost(hostname) {
22
+ if (!hostname) return false;
23
+ if (hostname === 'localhost') return true;
24
+ if (hostname === '127.0.0.1' || hostname.startsWith('127.')) return true;
25
+ if (hostname === '::1' || hostname === '[::1]') return true;
26
+ return false;
27
+ }
28
+
29
+ // Detects literal IPs in special-use ranges (RFC 1918, link-local/IMDS,
30
+ // CGNAT, multicast, IPv6 ULA/link-local). Sync only — no DNS resolution,
31
+ // so DNS rebinding is NOT covered by this check. The goal is to catch the
32
+ // common case: --base-url pointed at 169.254.169.254 (cloud metadata) or
33
+ // internal RFC 1918 hosts that should not receive Bearer tokens.
34
+ export function isPrivateOrSpecialIP(hostname) {
35
+ if (!hostname) return false;
36
+ const h = hostname.startsWith('[') && hostname.endsWith(']')
37
+ ? hostname.slice(1, -1)
38
+ : hostname;
39
+ if (/^\d{1,3}(?:\.\d{1,3}){3}$/.test(h)) {
40
+ const o = h.split('.').map((n) => Number(n));
41
+ if (o.some((n) => n < 0 || n > 255)) return false;
42
+ if (o[0] === 0) return true; // 0.0.0.0/8
43
+ if (o[0] === 10) return true; // 10/8
44
+ if (o[0] === 100 && o[1] >= 64 && o[1] <= 127) return true; // CGNAT
45
+ if (o[0] === 127) return true; // loopback
46
+ if (o[0] === 169 && o[1] === 254) return true; // link-local / IMDS
47
+ if (o[0] === 172 && o[1] >= 16 && o[1] <= 31) return true; // 172.16/12
48
+ if (o[0] === 192 && o[1] === 168) return true; // 192.168/16
49
+ if (o[0] === 198 && (o[1] === 18 || o[1] === 19)) return true;
50
+ if (o[0] >= 224) return true; // multicast / reserved
51
+ return false;
52
+ }
53
+ if (h.includes(':')) {
54
+ const lower = h.toLowerCase();
55
+ if (lower === '::1' || lower === '::') return true;
56
+ if (/^f[cd][0-9a-f]{2}:/.test(lower)) return true; // fc00::/7
57
+ if (/^fe[89ab][0-9a-f]:/.test(lower)) return true; // fe80::/10
58
+ if (lower.startsWith('ff')) return true; // multicast
59
+ const v4mapped = lower.match(/^::ffff:(\d{1,3}(?:\.\d{1,3}){3})$/);
60
+ if (v4mapped) return isPrivateOrSpecialIP(v4mapped[1]);
61
+ return false;
62
+ }
63
+ return false;
64
+ }
65
+
66
+ export function validateBaseURL(baseURL, { allowInsecure = false, allowPrivate = false } = {}) {
67
+ let url;
68
+ try {
69
+ url = new URL(baseURL);
70
+ } catch {
71
+ throw inputError(
72
+ `Invalid base URL: ${JSON.stringify(baseURL)}`,
73
+ 'The value is not a parseable URL.',
74
+ 'Use an https:// URL, or http://127.0.0.1 for local test servers.'
75
+ );
76
+ }
77
+ if (url.protocol !== 'https:' && url.protocol !== 'http:') {
78
+ throw inputError(
79
+ 'base URL must use http or https',
80
+ `Received ${url.protocol}: ${baseURL}`,
81
+ 'Use an https:// URL, or http://127.0.0.1 for local test servers.'
82
+ );
83
+ }
84
+ // Either an explicit caller opt-in or the env var override is enough to
85
+ // permit plaintext HTTP. cli.js sets the env when --allow-insecure-base-url
86
+ // is passed so downstream callLLM calls don't have to plumb the flag.
87
+ const allowInsec = allowInsecure || shouldAllowInsecureBaseURL();
88
+ if (url.protocol === 'http:' && !isLoopbackHost(url.hostname) && !allowInsec) {
89
+ throw inputError(
90
+ `refusing plaintext HTTP to ${url.hostname}`,
91
+ 'patina will not send prompts and API keys over non-loopback HTTP by default.',
92
+ 'Use an https:// URL, or pass --allow-insecure-base-url for a trusted endpoint.'
93
+ );
94
+ }
95
+ // SSRF guard: refuse non-loopback private/IMDS literal IPs unless explicitly
96
+ // opted in. Loopback is allowed (covered above) so local proxies still work.
97
+ const allowPriv = allowPrivate || shouldAllowPrivateBaseURL();
98
+ if (
99
+ !isLoopbackHost(url.hostname) &&
100
+ isPrivateOrSpecialIP(url.hostname) &&
101
+ !allowPriv
102
+ ) {
103
+ throw inputError(
104
+ `refusing private/reserved base URL ${url.hostname}`,
105
+ 'This blocks sending API keys to cloud metadata or RFC 1918/private endpoints by accident.',
106
+ 'Pass --allow-private-base-url only if you intentionally target an internal endpoint.'
107
+ );
108
+ }
109
+ }
110
+
111
+ export function shouldAllowInsecureBaseURL(parsed) {
112
+ if (parsed && parsed.allowInsecureBaseURL) return true;
113
+ const env = process.env.PATINA_ALLOW_INSECURE_BASE_URL;
114
+ return env === '1' || env === 'true' || env === 'yes';
115
+ }
116
+
117
+ export function applyInsecureBaseURLOptIn(parsed) {
118
+ if (parsed && parsed.allowInsecureBaseURL) {
119
+ process.env.PATINA_ALLOW_INSECURE_BASE_URL = '1';
120
+ }
121
+ }
122
+
123
+ export function shouldAllowPrivateBaseURL(parsed) {
124
+ if (parsed && parsed.allowPrivateBaseURL) return true;
125
+ const env = process.env.PATINA_ALLOW_PRIVATE_BASE_URL;
126
+ return env === '1' || env === 'true' || env === 'yes';
127
+ }
128
+
129
+ export function applyPrivateBaseURLOptIn(parsed) {
130
+ if (parsed && parsed.allowPrivateBaseURL) {
131
+ process.env.PATINA_ALLOW_PRIVATE_BASE_URL = '1';
132
+ }
133
+ }
@@ -0,0 +1,16 @@
1
+ ---
2
+ fixture_id: en-ai-01
3
+ language: en
4
+ class: ai
5
+ expected_hot: true
6
+ why_designed_this_way: |
7
+ Burstiness only. Sentence word counts: 14, 13, 15, 13, 14 → mean = 13.8, stddev ≈ 0.75,
8
+ CV ≈ 0.054 (very low, well under 0.25). Content words span distinct concepts (flexibility,
9
+ productivity, commute, boundaries, adoption) so MATTR is mid-range (~0.65, not flagged).
10
+ No catalogued en patterns triggered: no "it is worth noting", no "in today's world",
11
+ no "delve into", no excessive connector stacking, no bold/lists, no hype adjectives.
12
+ Pure uniform sentence rhythm is the sole signal.
13
+ topic: remote work
14
+ ---
15
+
16
+ Remote work has reshaped how employees think about daily schedules and flexibility. Many workers report higher satisfaction when they can choose their own hours. The daily commute, once unavoidable, is no longer a fixed part of professional life. Maintaining clear boundaries between work and personal time has become a shared challenge. Companies continue to adapt their policies as remote work adoption grows each year.
@@ -0,0 +1,16 @@
1
+ ---
2
+ fixture_id: en-ai-02
3
+ language: en
4
+ class: ai
5
+ expected_hot: true
6
+ why_designed_this_way: |
7
+ MATTR only. Heavy lexical cycling on a tight cluster: cycling/bike/bicycle,
8
+ city/urban/cities, lane/infrastructure/path, rider/commuter — four semantic pairs
9
+ reused across all five sentences with minimal lexical variation. Estimated MATTR ~0.47
10
+ (low band, under 0.55). Sentence lengths: 15, 12, 16, 11, 14 → CV ≈ 0.14 (mid-low,
11
+ not flagged by burstiness). Only MATTR fires. No catalogued patterns: no "robust",
12
+ no "leverage", no connector overload, no chatbot closings, no 3-of-3 list structures.
13
+ topic: urban cycling
14
+ ---
15
+
16
+ Cities around the world are investing in cycling infrastructure to make urban bike travel safer. Dedicated bike lanes help riders navigate city streets without competing with car traffic. Expanding cycling infrastructure gives urban commuters a reliable alternative to driving or transit. When a city builds more bike paths, rider numbers tend to rise across all age groups. Cycling infrastructure improvements make cities more accessible and reduce urban congestion over time.