kotonoha 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/README-ja.md +6 -1
- package/README.md +17 -1
- package/package.json +1 -1
- package/skills/japanese-prose/NOTICE.md +7 -1
- package/skills/japanese-prose/SKILL.md +44 -15
- package/skills/japanese-prose/references/review-workflow.md +57 -0
- package/skills/japanese-prose/references/scoring.md +6 -0
- package/skills/japanese-prose/scripts/core.py +6 -2
- package/skills/japanese-prose/scripts/invariants.py +182 -0
- package/skills/japanese-prose/scripts/lint.py +68 -3
- package/skills/japanese-prose/scripts/test_engine.py +168 -0
- package/skills/tech-writer/references/japanese-prose-optimization.md +14 -6
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.4.0] - 2026-10-19
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- Add duplicate-particle/word, Japanese bracket, and paragraph style-mixing
|
|
15
|
+
diagnostics, plus a minimal-edit `proofread` skill mode.
|
|
16
|
+
- Add a Markdown-aware invariant gate with protected-content checks and
|
|
17
|
+
required hash-bound semantic review of edits.
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
|
|
21
|
+
- Avoid treating obligation expressions as double-negative errors and
|
|
22
|
+
conclusion questions as formulaic assertions.
|
|
23
|
+
- Keep shorter fence markers and fence-like info strings inside code blocks
|
|
24
|
+
out of prose diagnostics.
|
|
25
|
+
- Require the invariant gate and current semantic review to pass before
|
|
26
|
+
claiming Japanese prose optimization completed.
|
|
27
|
+
|
|
10
28
|
## [0.3.0] - 2026-09-30
|
|
11
29
|
|
|
12
30
|
### Added
|
package/README-ja.md
CHANGED
|
@@ -97,7 +97,7 @@ White Paper向けの分析を作成して
|
|
|
97
97
|
|
|
98
98
|
- GitHub Copilot CLIがインストールされていること
|
|
99
99
|
- tech-writerの構造lintを実行する場合はPython 3.9以上
|
|
100
|
-
- 同梱`japanese-prose`の文章lint
|
|
100
|
+
- 同梱`japanese-prose`の文章lint、読解負荷分析、アウトライン抽出、用語検査、編集前後の不変条件検査を実行する場合はPython 3.10以上と`uv`。GiNZA、`ja_ginza`、spaCy、Markdown解析用の依存関係は、各スクリプトの初回実行時に`uv`が解決
|
|
101
101
|
|
|
102
102
|
### npmからインストール
|
|
103
103
|
|
|
@@ -136,6 +136,10 @@ npx kotonoha install --skill all --force
|
|
|
136
136
|
|
|
137
137
|
kotonohaには独自開発の`japanese-prose`が同梱されるため、文章最適化スキルを別途インストールする必要はありません。`npx kotonoha install`により`tech-writer`と同じスキルルートへ配置され、GiNZAを使った文章lint、読解負荷検査、用語抽出、アウトライン検査、反復的な文章レビューをローカルで実行できます。この実装はkotonoha向けにゼロから開発しており、`natural-japanese`のソースコードを含みません。GiNZAの依存関係とライセンス情報は`skills/japanese-prose/NOTICE.md`に記録しています。
|
|
138
138
|
|
|
139
|
+
文章lintでは、隣接する助詞・単語の重複、和文括弧の不整合、同一段落内の敬体・常体の混在も確認します。指摘は自動置換の指示ではなく、文脈を確認する対象です。`proofread`モードでは、任意の言い換えを避け、誤字と文法上の誤りだけを最小限に修正します。100点満点のスコアは確認箇所を探す補助であり、訂正精度や品質の保証ではありません。
|
|
140
|
+
|
|
141
|
+
編集前の原文は変更せずに保存します。同梱の`scripts/invariants.py`が、数値、識別子、コード、表、リンク、指定された固有名詞・専門用語などの変更を検査します。編集した文章には、原文と編集後のファイルのハッシュに対応した意味保存レビューが必要です。機械的な一致だけでは、意味が保たれたとは判定しません。レビュー形式と終了コードは、[`review-workflow.md`](skills/japanese-prose/references/review-workflow.md)を参照してください。
|
|
142
|
+
|
|
139
143
|
自然言語で指示するときは、GiNZAを明示する必要はありません。「日本語文書を自然にする」「読みやすくする」「簡潔にする」「AIらしい定型表現を減らす」といった依頼をすると、同梱の文章最適化ワークフローが呼び出されます。
|
|
140
144
|
|
|
141
145
|
```text
|
|
@@ -150,6 +154,7 @@ README-ja.mdの日本語を自然で読みやすい表現に修正して
|
|
|
150
154
|
目的に応じて、次のようにモードを指定できます。
|
|
151
155
|
|
|
152
156
|
- ファイルを修正する: `README-ja.mdの日本語表現を最適化して`
|
|
157
|
+
- 誤りだけを修正する: `README-ja.mdをproofreadモードで校正して。任意の言い換えはせず、誤字と文法上の誤りだけを最小限に修正して`
|
|
153
158
|
- 修正せずにレビューする: `README-ja.mdの日本語をレビューして。ファイルは修正しないで`
|
|
154
159
|
- 採点だけ行う: `README-ja.mdの日本語表現を100点満点で採点して`
|
|
155
160
|
- すべての診断を実行する: `README-ja.mdをfullモードで日本語表現最適化して`
|
package/README.md
CHANGED
|
@@ -155,7 +155,8 @@ handoff without inventing missing evidence.
|
|
|
155
155
|
|
|
156
156
|
- GitHub Copilot CLI installed
|
|
157
157
|
- Python 3.9+ for tech-writer's structural lint
|
|
158
|
-
- Python 3.10+ and `uv` for the
|
|
158
|
+
- Python 3.10+ and `uv` for the edit-invariant gate and bundled GiNZA
|
|
159
|
+
diagnostics such as prose lint,
|
|
159
160
|
reading-load analysis, outline extraction, and terminology checks; `uv`
|
|
160
161
|
resolves GiNZA, `ja_ginza`, and spaCy on first use
|
|
161
162
|
|
|
@@ -209,6 +210,20 @@ scratch for kotonoha and does not include `natural-japanese` source code.
|
|
|
209
210
|
GiNZA dependency and license information is recorded in
|
|
210
211
|
`skills/japanese-prose/NOTICE.md`.
|
|
211
212
|
|
|
213
|
+
The lint also flags adjacent duplicate particles/words, Japanese bracket
|
|
214
|
+
mismatches, and polite/plain style mixing within a paragraph. Findings are
|
|
215
|
+
review targets, not automatic replacements. Use `proofread` mode for minimal
|
|
216
|
+
typo/grammar corrections without optional stylistic rewriting. The 0–100
|
|
217
|
+
score is a triage aid, not correction accuracy or a quality certificate.
|
|
218
|
+
|
|
219
|
+
Before editing, the workflow saves an immutable original. The bundled
|
|
220
|
+
`scripts/invariants.py` checks protected content and requires a hash-bound
|
|
221
|
+
semantic review of changed prose. Numbers, identifiers, code, tables, links,
|
|
222
|
+
and caller-supplied proper names/domain terms cannot be silently changed.
|
|
223
|
+
Mechanical equality alone does not establish preserved meaning. See
|
|
224
|
+
[`review-workflow.md`](skills/japanese-prose/references/review-workflow.md)
|
|
225
|
+
for the review format and gate exit codes.
|
|
226
|
+
|
|
212
227
|
You do not need to name GiNZA explicitly in a natural-language request.
|
|
213
228
|
Requests to make a Japanese document more natural, readable, concise, or less
|
|
214
229
|
formulaic invoke the bundled prose-optimization workflow. For example:
|
|
@@ -225,6 +240,7 @@ and repetitive AI-like phrasing, then fix the findings.
|
|
|
225
240
|
You can specify the intended mode more precisely:
|
|
226
241
|
|
|
227
242
|
- Edit the file: `Optimize the Japanese prose in README-ja.md.`
|
|
243
|
+
- Correct errors only: `Proofread README-ja.md; make minimal typo/grammar corrections without stylistic rewrites.`
|
|
228
244
|
- Review without editing: `Review the Japanese in README-ja.md, but do not modify the file.`
|
|
229
245
|
- Score only: `Score the Japanese prose in README-ja.md out of 100.`
|
|
230
246
|
- Run every diagnostic: `Run the full Japanese prose optimization workflow on README-ja.md.`
|
package/package.json
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Diagnostic dependency notices
|
|
2
2
|
|
|
3
3
|
This skill is an original kotonoha implementation and does not include source
|
|
4
4
|
code from `coji/natural-japanese`.
|
|
@@ -15,3 +15,9 @@ datasets.
|
|
|
15
15
|
The implementation targets GiNZA 5.2.x and uses token surface forms, lemmas,
|
|
16
16
|
universal part-of-speech tags, dependency relationships, sentence boundaries,
|
|
17
17
|
and named-entity labels.
|
|
18
|
+
|
|
19
|
+
The edit-invariant gate uses
|
|
20
|
+
[markdown-it-py](https://github.com/executablebooks/markdown-it-py), distributed
|
|
21
|
+
under the MIT License, to parse Markdown without treating short fences
|
|
22
|
+
inside longer fenced code blocks as prose. Its dependencies are resolved by
|
|
23
|
+
`uv` when `scripts/invariants.py` is run.
|
|
@@ -6,10 +6,10 @@ description: >-
|
|
|
6
6
|
reviewing, or scoring Japanese prose; when the user asks for more natural,
|
|
7
7
|
readable, concise, or less AI-like Japanese; or when tech-writer delegates
|
|
8
8
|
its sentence-level quality pass. Owns wording, sentence rhythm, reading
|
|
9
|
-
load,
|
|
10
|
-
technical-document structure or presentation storylines.
|
|
9
|
+
load, typo and grammar review, terminology, and edit-invariant checks.
|
|
10
|
+
Does not own technical-document structure or presentation storylines.
|
|
11
11
|
license: MIT
|
|
12
|
-
argument-hint: "[write|review|score] [quick|full] <target file or request>"
|
|
12
|
+
argument-hint: "[write|proofread|review|score] [quick|full] <target file or request>"
|
|
13
13
|
---
|
|
14
14
|
|
|
15
15
|
# japanese-prose
|
|
@@ -39,6 +39,8 @@ acceptance criteria, approval states, or normative force.
|
|
|
39
39
|
- `write`: write or rewrite Japanese prose. Use `quick` unless the document
|
|
40
40
|
is external-facing, high-risk, or longer than roughly 10,000 Japanese
|
|
41
41
|
characters.
|
|
42
|
+
- `proofread`: correct demonstrable typos and grammatical errors with minimal
|
|
43
|
+
edits. Do not perform optional stylistic rewrites or chase a higher score.
|
|
42
44
|
- `review`: report problems and proposed corrections without editing.
|
|
43
45
|
- `score`: run diagnostics and report the 0–100 score without rewriting.
|
|
44
46
|
- `quick`: run prose lint once, review findings in context, edit, and rerun
|
|
@@ -50,33 +52,57 @@ acceptance criteria, approval states, or normative force.
|
|
|
50
52
|
## Workflow
|
|
51
53
|
|
|
52
54
|
1. Read the target, audience, intended outcome, and calling skill's frozen
|
|
53
|
-
invariants.
|
|
55
|
+
invariants. Before editing, save an immutable copy of the entire original
|
|
56
|
+
file as `<workdir>/prose-original.md`. Keep this separate from diagnostic
|
|
57
|
+
baselines, which may change between rounds.
|
|
54
58
|
2. Read `references/writing-guidelines.md` before generating or rewriting
|
|
55
|
-
prose.
|
|
59
|
+
prose. Resolve `<japanese-prose-dir>` to the directory containing this
|
|
60
|
+
loaded skill; do not assume the shell is running in that directory.
|
|
56
61
|
3. Run the GiNZA prose diagnostic:
|
|
57
62
|
|
|
58
63
|
```bash
|
|
59
|
-
uv run scripts/lint.py <target-file> --genre tech --json > <workdir>/prose-baseline.json
|
|
64
|
+
uv run <japanese-prose-dir>/scripts/lint.py <target-file> --genre tech --json > <workdir>/prose-baseline.json
|
|
60
65
|
```
|
|
61
66
|
|
|
62
67
|
4. For `full` mode, also run:
|
|
63
68
|
|
|
64
69
|
```bash
|
|
65
|
-
uv run scripts/lint.py <target-file> --genre tech --reading-load --json
|
|
66
|
-
uv run scripts/outline.py <target-file>
|
|
67
|
-
uv run scripts/terms.py <target-file> --json
|
|
70
|
+
uv run <japanese-prose-dir>/scripts/lint.py <target-file> --genre tech --reading-load --json
|
|
71
|
+
uv run <japanese-prose-dir>/scripts/outline.py <target-file>
|
|
72
|
+
uv run <japanese-prose-dir>/scripts/terms.py <target-file> --json
|
|
68
73
|
```
|
|
69
74
|
|
|
70
|
-
5.
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
75
|
+
5. Read the entire prose for typos, particle errors, subject/predicate
|
|
76
|
+
mismatches, and incorrect inflections even when lint reports no findings.
|
|
77
|
+
Classify each finding as `fix` or `keep`. A detector identifies a review
|
|
78
|
+
target; it does not authorize blind replacement. Adjacent duplicate
|
|
79
|
+
particles/words, Japanese bracket mismatches, and within-paragraph style
|
|
80
|
+
mixing are proofreading targets, not proof of an error.
|
|
81
|
+
6. Correct demonstrable errors first. In `write` mode, then consider optional
|
|
82
|
+
wording changes that serve the reader's stated goal. Keep these separate
|
|
83
|
+
from necessary corrections. Preserve acceptable wording when a change
|
|
84
|
+
merely substitutes a synonym or removes a legitimate qualification.
|
|
85
|
+
Never infer a factual or proper-name correction from plausibility alone.
|
|
74
86
|
7. Rerun the prose diagnostic with
|
|
75
87
|
`--baseline <workdir>/prose-baseline.json` to classify new, persisting,
|
|
76
88
|
and resolved findings. Replace the baseline only after recording the
|
|
77
89
|
decisions for the current round.
|
|
78
|
-
8.
|
|
79
|
-
|
|
90
|
+
8. Run the edit-invariant gate against the immutable original:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
uv run <japanese-prose-dir>/scripts/invariants.py <workdir>/prose-original.md <target-file> --protect <proper-name-or-domain-term> --json
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Repeat `--protect` for each frozen proper name or domain term; omit it only
|
|
97
|
+
when there are none. Exit 2 blocks changed protected content. Any edit
|
|
98
|
+
requires a contextual semantic review, even if all mechanical checks
|
|
99
|
+
pass: exit 3 means that review is pending or rejected, not completion.
|
|
100
|
+
Follow `references/review-workflow.md` to record the hash-bound review,
|
|
101
|
+
then rerun with `--review <workdir>/semantic-review.json --json`.
|
|
102
|
+
Recheck after every subsequent edit; a stale review cannot approve it.
|
|
103
|
+
9. Stop only when every finding has a decision, no new actionable finding
|
|
104
|
+
appears, and the invariant gate returns `passed`. If the gate cannot run,
|
|
105
|
+
report the failure rather than declaring completion.
|
|
80
106
|
|
|
81
107
|
Allow at most three edit-and-diagnose rounds per invocation. If actionable
|
|
82
108
|
findings or invariant violations remain, report that the optimization did not
|
|
@@ -108,4 +134,7 @@ Report:
|
|
|
108
134
|
- findings fixed and findings deliberately kept
|
|
109
135
|
- any skipped diagnostic and its reason
|
|
110
136
|
- invariant verification result
|
|
137
|
+
- necessary corrections versus optional wording changes
|
|
138
|
+
- semantic review status and any remaining uncertainty; mechanical equality
|
|
139
|
+
is not a guarantee of preserved meaning
|
|
111
140
|
- `completed` or `did not converge`
|
|
@@ -40,6 +40,61 @@ Read only headings and paragraph openings. The argument must still be
|
|
|
40
40
|
predictable. Then read the finished prose continuously and verify that local
|
|
41
41
|
rewrites did not damage transitions or create repeated explanations.
|
|
42
42
|
|
|
43
|
+
## Require a semantic review of edits
|
|
44
|
+
|
|
45
|
+
Run `scripts/invariants.py` on the immutable pre-edit file and the edited
|
|
46
|
+
file. It protects numbers and known quantity units in occurrence order,
|
|
47
|
+
uppercase identifiers, URL/link destinations, frontmatter, heading levels,
|
|
48
|
+
inline code, fenced/indented code, Markdown tables, HTML, and caller-supplied
|
|
49
|
+
`--protect` terms. The Markdown parser recognizes nested and long fences.
|
|
50
|
+
The check is conservative: it may block a legitimate reorder or protected
|
|
51
|
+
content correction. Such changes need separate authorization, not a
|
|
52
|
+
reviewer override of a failed invariant.
|
|
53
|
+
|
|
54
|
+
Mechanical equality cannot establish equivalent meaning. For every changed
|
|
55
|
+
passage, compare the original and edited text, including nearby context:
|
|
56
|
+
|
|
57
|
+
- `actors`: who performs or receives each action is unchanged
|
|
58
|
+
- `negation`: polarity and the scope of negation are unchanged
|
|
59
|
+
- `conditions`: prerequisites, exceptions, acceptance criteria, and approval states are unchanged
|
|
60
|
+
- `modality`: obligations, recommendations, permissions, and uncertainty are unchanged
|
|
61
|
+
- `scope`: limitations such as 「だけ」「すべて」「少なくとも」 are unchanged
|
|
62
|
+
- `causality`: causal and temporal relationships are unchanged
|
|
63
|
+
- `quantity_bindings`: each quantity still refers to the same object or action
|
|
64
|
+
- `terminology`: proper names and domain concepts are unchanged
|
|
65
|
+
- `structure`: section identity, order, quoted meaning, and traceability are unchanged
|
|
66
|
+
|
|
67
|
+
Record the review as JSON. Copy `before_sha256` and `after_sha256` from the
|
|
68
|
+
gate output, identify the actual reviewer, and set a check to `true` only
|
|
69
|
+
after inspecting it. Do not infer approval from a score, an unchanged token
|
|
70
|
+
count, or the absence of warnings. A review can be performed by the agent or
|
|
71
|
+
a human, but an agent review is not a human approval or a semantic proof.
|
|
72
|
+
|
|
73
|
+
```json
|
|
74
|
+
{
|
|
75
|
+
"before_sha256": "<hash from the gate>",
|
|
76
|
+
"after_sha256": "<hash from the gate>",
|
|
77
|
+
"reviewer": "<actual reviewer identifier>",
|
|
78
|
+
"checks": {
|
|
79
|
+
"actors": false,
|
|
80
|
+
"negation": false,
|
|
81
|
+
"conditions": false,
|
|
82
|
+
"modality": false,
|
|
83
|
+
"scope": false,
|
|
84
|
+
"causality": false,
|
|
85
|
+
"quantity_bindings": false,
|
|
86
|
+
"terminology": false,
|
|
87
|
+
"structure": false
|
|
88
|
+
}
|
|
89
|
+
}
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Rerun the gate with `--review <review-file>`. Exit codes are 0 for `passed`,
|
|
93
|
+
1 for input/runtime errors, 2 for protected-content violations, and 3 for
|
|
94
|
+
missing, stale, incomplete, or rejected semantic reviews. Subsequent edits
|
|
95
|
+
invalidate the review. If equivalence is uncertain, keep the original
|
|
96
|
+
wording or report the unresolved concern.
|
|
97
|
+
|
|
43
98
|
## Convergence
|
|
44
99
|
|
|
45
100
|
The review is complete when:
|
|
@@ -47,4 +102,6 @@ The review is complete when:
|
|
|
47
102
|
- every finding has a `fix` or `keep` decision
|
|
48
103
|
- rerunning diagnostics produces no new actionable finding
|
|
49
104
|
- frozen invariants match the pre-edit document
|
|
105
|
+
- `scripts/invariants.py` returns `passed` with a current semantic review
|
|
106
|
+
for changed text
|
|
50
107
|
- the reader can reach the requested outcome without an unstated inference
|
|
@@ -22,3 +22,9 @@ A high score does not prove that facts are correct, the argument is complete,
|
|
|
22
22
|
or the prose is natural. A low score does not require every finding to be
|
|
23
23
|
changed. Always report the finding categories and contextual decisions beside
|
|
24
24
|
the score.
|
|
25
|
+
|
|
26
|
+
Proofreading findings and style/readability findings contribute to the same
|
|
27
|
+
triage score, but they are not interchangeable measures of quality. Report
|
|
28
|
+
the categories separately. Do not use this score as typo-detection recall,
|
|
29
|
+
correction accuracy, semantic fidelity, or evidence that an LLM-assisted
|
|
30
|
+
rewrite is better. A correctly repaired typo may leave the score unchanged.
|
|
@@ -76,10 +76,14 @@ def iter_content_lines(text: str) -> Iterable[tuple[int, str]]:
|
|
|
76
76
|
|
|
77
77
|
fence = FENCE_RE.match(raw)
|
|
78
78
|
if fence:
|
|
79
|
-
marker = fence.group(1)
|
|
79
|
+
marker = fence.group(1)
|
|
80
80
|
if not fence_marker:
|
|
81
81
|
fence_marker = marker
|
|
82
|
-
elif
|
|
82
|
+
elif (
|
|
83
|
+
marker[0] == fence_marker[0]
|
|
84
|
+
and len(marker) >= len(fence_marker)
|
|
85
|
+
and not raw[fence.end():].strip()
|
|
86
|
+
):
|
|
83
87
|
fence_marker = ""
|
|
84
88
|
yield index, ""
|
|
85
89
|
continue
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
# /// script
|
|
3
|
+
# requires-python = ">=3.10"
|
|
4
|
+
# dependencies = ["markdown-it-py>=3,<5"]
|
|
5
|
+
# ///
|
|
6
|
+
"""Check protected content and require a hash-bound semantic review of edits."""
|
|
7
|
+
|
|
8
|
+
from __future__ import annotations
|
|
9
|
+
|
|
10
|
+
import argparse
|
|
11
|
+
import hashlib
|
|
12
|
+
import json
|
|
13
|
+
from pathlib import Path
|
|
14
|
+
import re
|
|
15
|
+
import sys
|
|
16
|
+
|
|
17
|
+
from markdown_it import MarkdownIt
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
SEMANTIC_CHECKS = (
|
|
21
|
+
"actors", "negation", "conditions", "modality", "scope", "causality",
|
|
22
|
+
"quantity_bindings", "terminology", "structure",
|
|
23
|
+
)
|
|
24
|
+
NUMBER_RE = re.compile(r"\d+(?:[.,]\d+)*")
|
|
25
|
+
IDENTIFIER_RE = re.compile(r"(?<![A-Za-z0-9_])[A-Z][A-Z0-9]*(?:[-_][A-Z0-9]+)+(?![A-Za-z0-9_])")
|
|
26
|
+
QUANTITY_RE = re.compile(
|
|
27
|
+
r"(\d+(?:[.,]\d+)*)\s*"
|
|
28
|
+
r"(ミリ秒|時間|年間|か月|ヶ月|秒|分|日|週|月|年|回|件|台|人|個|円|倍|"
|
|
29
|
+
r"[A-Za-zµμ]+(?:/[A-Za-zµμ]+)?|%|%|℃)(?![A-Za-z])"
|
|
30
|
+
)
|
|
31
|
+
BARE_URL_RE = re.compile(r"https?://[^\s<>`\"()\[\]、。!?「」『』()]+")
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def fingerprint(value: object) -> str:
|
|
35
|
+
return hashlib.sha256(
|
|
36
|
+
json.dumps(value, ensure_ascii=False, sort_keys=True).encode("utf-8")
|
|
37
|
+
).hexdigest()
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def snapshot(text: str, protected_terms: list[str]) -> dict:
|
|
41
|
+
lines = text.splitlines(keepends=True)
|
|
42
|
+
frontmatter = ""
|
|
43
|
+
body = text
|
|
44
|
+
if lines and lines[0].strip() == "---":
|
|
45
|
+
end = next(
|
|
46
|
+
(index for index in range(1, len(lines)) if lines[index].strip() in {"---", "..."}),
|
|
47
|
+
None,
|
|
48
|
+
)
|
|
49
|
+
if end is not None:
|
|
50
|
+
frontmatter = "".join(lines[:end + 1])
|
|
51
|
+
body = "".join(lines[end + 1:])
|
|
52
|
+
body_lines = body.splitlines(keepends=True)
|
|
53
|
+
tokens = MarkdownIt("commonmark").enable("table").parse(body)
|
|
54
|
+
blocks = []
|
|
55
|
+
headings = []
|
|
56
|
+
code = []
|
|
57
|
+
links = []
|
|
58
|
+
html = []
|
|
59
|
+
for token in tokens:
|
|
60
|
+
if token.type in {"fence", "code_block", "table_open", "html_block"}:
|
|
61
|
+
if token.map is None:
|
|
62
|
+
raise ValueError(f"Missing source map for {token.type}")
|
|
63
|
+
start, end = token.map
|
|
64
|
+
blocks.append((token.type, "".join(body_lines[start:end])))
|
|
65
|
+
if token.type == "heading_open":
|
|
66
|
+
headings.append(token.tag)
|
|
67
|
+
for child in token.children or []:
|
|
68
|
+
if child.type == "code_inline":
|
|
69
|
+
code.append(child.content)
|
|
70
|
+
elif child.type == "link_open":
|
|
71
|
+
links.append(child.attrGet("href"))
|
|
72
|
+
elif child.type == "image":
|
|
73
|
+
links.append(child.attrGet("src"))
|
|
74
|
+
elif child.type == "html_inline":
|
|
75
|
+
html.append(child.content)
|
|
76
|
+
return {
|
|
77
|
+
"frontmatter": frontmatter,
|
|
78
|
+
"protected_blocks": blocks,
|
|
79
|
+
"heading_levels": headings,
|
|
80
|
+
"inline_code": code,
|
|
81
|
+
"link_destinations": links,
|
|
82
|
+
"html_tags": html,
|
|
83
|
+
"numbers_in_order": NUMBER_RE.findall(text),
|
|
84
|
+
"quantities_in_order": QUANTITY_RE.findall(text),
|
|
85
|
+
"identifiers_in_order": IDENTIFIER_RE.findall(text),
|
|
86
|
+
"bare_urls_in_order": BARE_URL_RE.findall(text),
|
|
87
|
+
"protected_terms": {term: text.count(term) for term in protected_terms},
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
def check(
|
|
92
|
+
before: Path,
|
|
93
|
+
after: Path,
|
|
94
|
+
protected_terms: list[str] | None = None,
|
|
95
|
+
review_path: Path | None = None,
|
|
96
|
+
) -> dict:
|
|
97
|
+
terms = protected_terms or []
|
|
98
|
+
if any(not term.strip() for term in terms):
|
|
99
|
+
raise ValueError("Protected terms must not be empty")
|
|
100
|
+
original = before.read_bytes()
|
|
101
|
+
edited = after.read_bytes()
|
|
102
|
+
before_hash = hashlib.sha256(original).hexdigest()
|
|
103
|
+
after_hash = hashlib.sha256(edited).hexdigest()
|
|
104
|
+
initial = snapshot(original.decode("utf-8"), terms)
|
|
105
|
+
final = snapshot(edited.decode("utf-8"), terms)
|
|
106
|
+
violations = [
|
|
107
|
+
{
|
|
108
|
+
"category": name,
|
|
109
|
+
"message": f"Protected content changed: {name}",
|
|
110
|
+
"before_fingerprint": fingerprint(initial[name]),
|
|
111
|
+
"after_fingerprint": fingerprint(final[name]),
|
|
112
|
+
}
|
|
113
|
+
for name in initial if initial[name] != final[name]
|
|
114
|
+
]
|
|
115
|
+
review_errors = []
|
|
116
|
+
review_required = original != edited
|
|
117
|
+
reviewer = None
|
|
118
|
+
if review_path is not None:
|
|
119
|
+
review = json.loads(review_path.read_text(encoding="utf-8"))
|
|
120
|
+
if not isinstance(review, dict):
|
|
121
|
+
raise ValueError("Semantic review must be a JSON object")
|
|
122
|
+
reviewer = review.get("reviewer")
|
|
123
|
+
if not isinstance(reviewer, str) or not reviewer.strip():
|
|
124
|
+
review_errors.append("reviewer must be a non-empty identifier")
|
|
125
|
+
if review.get("before_sha256") != before_hash or review.get("after_sha256") != after_hash:
|
|
126
|
+
review_errors.append("review hashes do not match this exact before/after pair")
|
|
127
|
+
checks = review.get("checks")
|
|
128
|
+
if not isinstance(checks, dict):
|
|
129
|
+
review_errors.append("checks must be an object")
|
|
130
|
+
else:
|
|
131
|
+
review_errors.extend(
|
|
132
|
+
f"{name} has not been affirmed" for name in SEMANTIC_CHECKS
|
|
133
|
+
if checks.get(name) is not True
|
|
134
|
+
)
|
|
135
|
+
elif review_required:
|
|
136
|
+
review_errors.append("changed text requires a hash-bound semantic review")
|
|
137
|
+
status = "blocked" if violations else "review_required" if review_errors else "passed"
|
|
138
|
+
return {
|
|
139
|
+
"before": str(before),
|
|
140
|
+
"after": str(after),
|
|
141
|
+
"before_sha256": before_hash,
|
|
142
|
+
"after_sha256": after_hash,
|
|
143
|
+
"status": status,
|
|
144
|
+
"mechanical_violations": violations,
|
|
145
|
+
"semantic_review_required": review_required,
|
|
146
|
+
"semantic_checks": list(SEMANTIC_CHECKS),
|
|
147
|
+
"semantic_review_errors": review_errors,
|
|
148
|
+
"reviewer": reviewer,
|
|
149
|
+
"limitations": [
|
|
150
|
+
"Mechanical equality does not prove semantic equivalence.",
|
|
151
|
+
"Proper names and domain terms require caller-supplied --protect values and contextual review.",
|
|
152
|
+
"Semantic checks are reviewer attestations, not automated entailment judgments.",
|
|
153
|
+
],
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
|
|
157
|
+
def main() -> int:
|
|
158
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
159
|
+
parser.add_argument("before")
|
|
160
|
+
parser.add_argument("after")
|
|
161
|
+
parser.add_argument("--protect", action="append", default=[])
|
|
162
|
+
parser.add_argument("--review", type=Path)
|
|
163
|
+
parser.add_argument("--json", action="store_true")
|
|
164
|
+
args = parser.parse_args()
|
|
165
|
+
try:
|
|
166
|
+
result = check(Path(args.before), Path(args.after), args.protect, args.review)
|
|
167
|
+
except (OSError, ValueError, UnicodeError) as error:
|
|
168
|
+
print(f"error: {error}", file=sys.stderr)
|
|
169
|
+
return 1
|
|
170
|
+
if args.json:
|
|
171
|
+
print(json.dumps(result, ensure_ascii=False, indent=2))
|
|
172
|
+
else:
|
|
173
|
+
print(f"{result['status']}: {args.before} -> {args.after}")
|
|
174
|
+
for violation in result["mechanical_violations"]:
|
|
175
|
+
print(f" {violation['message']}")
|
|
176
|
+
for error in result["semantic_review_errors"]:
|
|
177
|
+
print(f" Semantic review: {error}")
|
|
178
|
+
return {"passed": 0, "blocked": 2, "review_required": 3}[result["status"]]
|
|
179
|
+
|
|
180
|
+
|
|
181
|
+
if __name__ == "__main__":
|
|
182
|
+
sys.exit(main())
|
|
@@ -75,10 +75,12 @@ DOUBLE_NEGATIVE_RULES = (
|
|
|
75
75
|
re.compile(r"ないわけではない"),
|
|
76
76
|
re.compile(r"ないことはない"),
|
|
77
77
|
re.compile(r"なくはない"),
|
|
78
|
-
re.compile(r"なくてはならない"),
|
|
79
|
-
re.compile(r"ざるを得ない"),
|
|
80
78
|
)
|
|
81
79
|
|
|
80
|
+
BRACKET_PAIRS = dict(zip("([{「『【", ")]}」』】"))
|
|
81
|
+
POLITE_END_RE = re.compile(r"(?:です|ます|でした|ました|ません(?:でした)?|でしょう|ましょう|ください|下さい)$")
|
|
82
|
+
PLAIN_END_RE = re.compile(r"(?:である|だった|だ|る|た|ない)$")
|
|
83
|
+
|
|
82
84
|
GENRE_LIMITS = {
|
|
83
85
|
"general": {"length": 90, "critical_length": 150, "commas": 5, "depth": 8},
|
|
84
86
|
"tech": {"length": 110, "critical_length": 180, "commas": 6, "depth": 9},
|
|
@@ -100,6 +102,64 @@ def add(
|
|
|
100
102
|
)
|
|
101
103
|
|
|
102
104
|
|
|
105
|
+
def proofread(block, doc, findings: list[Finding]) -> None:
|
|
106
|
+
stack: list[str] = []
|
|
107
|
+
unbalanced = False
|
|
108
|
+
for character in block.text:
|
|
109
|
+
if character in BRACKET_PAIRS:
|
|
110
|
+
stack.append(character)
|
|
111
|
+
elif character in BRACKET_PAIRS.values():
|
|
112
|
+
if not stack or BRACKET_PAIRS[stack[-1]] != character:
|
|
113
|
+
unbalanced = True
|
|
114
|
+
else:
|
|
115
|
+
stack.pop()
|
|
116
|
+
if stack or unbalanced:
|
|
117
|
+
add(
|
|
118
|
+
findings, block.start_line, "unbalanced_brackets", "warning",
|
|
119
|
+
"括弧の対応が取れていません。引用や数式として意図した表記か確認します。",
|
|
120
|
+
block.text,
|
|
121
|
+
)
|
|
122
|
+
|
|
123
|
+
for previous, current in zip(doc, doc[1:]):
|
|
124
|
+
if previous.idx + len(previous.text) != current.idx or previous.text != current.text:
|
|
125
|
+
continue
|
|
126
|
+
particle = previous.pos_ == current.pos_ == "ADP"
|
|
127
|
+
word = (
|
|
128
|
+
len(current.text) >= 2
|
|
129
|
+
and previous.pos_ == current.pos_
|
|
130
|
+
and current.pos_ in {"NOUN", "VERB", "ADJ", "ADV"}
|
|
131
|
+
)
|
|
132
|
+
if particle or word:
|
|
133
|
+
add(
|
|
134
|
+
findings, sentence_line(block, current.idx),
|
|
135
|
+
"duplicate_particle" if particle else "duplicate_word", "warning",
|
|
136
|
+
"同じ助詞が隣接しています。誤入力による重複か確認します。"
|
|
137
|
+
if particle else "同じ語が隣接しています。意図した反復か確認します。",
|
|
138
|
+
current.sent.text, {"repeated": current.text},
|
|
139
|
+
)
|
|
140
|
+
|
|
141
|
+
first_style = None
|
|
142
|
+
for sent in doc.sents:
|
|
143
|
+
if re.search(r'[「」『』"“”]', sent.text):
|
|
144
|
+
continue
|
|
145
|
+
ending = sent.text.strip().rstrip("。!?!?")
|
|
146
|
+
style = (
|
|
147
|
+
"polite" if POLITE_END_RE.search(ending)
|
|
148
|
+
else "plain" if PLAIN_END_RE.search(ending)
|
|
149
|
+
else None
|
|
150
|
+
)
|
|
151
|
+
if style and first_style and style != first_style:
|
|
152
|
+
add(
|
|
153
|
+
findings, sentence_line(block, sent.start_char),
|
|
154
|
+
"mixed_sentence_style", "warning",
|
|
155
|
+
"同じ段落に敬体と常体が混在しています。引用や意図した使い分けか確認します。",
|
|
156
|
+
sent.text,
|
|
157
|
+
)
|
|
158
|
+
break
|
|
159
|
+
if style:
|
|
160
|
+
first_style = style
|
|
161
|
+
|
|
162
|
+
|
|
103
163
|
def analyze(path: Path, genre: str, reading_load: bool) -> dict:
|
|
104
164
|
text = read_text(path)
|
|
105
165
|
blocks = iter_prose_blocks(text)
|
|
@@ -110,12 +170,17 @@ def analyze(path: Path, genre: str, reading_load: bool) -> dict:
|
|
|
110
170
|
paragraph_leads: list[tuple[int, str]] = []
|
|
111
171
|
|
|
112
172
|
for block, doc in zip(blocks, nlp.pipe([item.text for item in blocks])):
|
|
173
|
+
if not reading_load:
|
|
174
|
+
proofread(block, doc, findings)
|
|
113
175
|
paragraph_tokens = content_tokens(doc)
|
|
114
176
|
if paragraph_tokens:
|
|
115
177
|
paragraph_leads.append((block.start_line, paragraph_tokens[0].lemma_))
|
|
116
178
|
|
|
117
179
|
for phrase, (category, message) in PHRASE_RULES.items():
|
|
118
|
-
|
|
180
|
+
pattern = re.escape(phrase)
|
|
181
|
+
if category == "formulaic_conclusion":
|
|
182
|
+
pattern += r"(?!か|[??])"
|
|
183
|
+
if not reading_load and re.search(pattern, block.text):
|
|
119
184
|
add(findings, block.start_line, category, "warning", message, block.text)
|
|
120
185
|
|
|
121
186
|
if not reading_load:
|
|
@@ -5,6 +5,7 @@
|
|
|
5
5
|
# "ginza>=5.2.0,<5.3",
|
|
6
6
|
# "ja-ginza>=5.2.0,<5.3",
|
|
7
7
|
# "spacy>=3.8.0,<4",
|
|
8
|
+
# "markdown-it-py>=3,<5",
|
|
8
9
|
# ]
|
|
9
10
|
# ///
|
|
10
11
|
"""Behavioral tests for kotonoha's original GiNZA diagnostics."""
|
|
@@ -12,6 +13,9 @@
|
|
|
12
13
|
from __future__ import annotations
|
|
13
14
|
|
|
14
15
|
import json
|
|
16
|
+
import hashlib
|
|
17
|
+
import subprocess
|
|
18
|
+
import sys
|
|
15
19
|
import tempfile
|
|
16
20
|
import unittest
|
|
17
21
|
from pathlib import Path
|
|
@@ -20,6 +24,7 @@ from core import strip_markdown_inline
|
|
|
20
24
|
from lint import analyze, compare_baseline
|
|
21
25
|
from outline import extract as extract_outline
|
|
22
26
|
from terms import extract as extract_terms
|
|
27
|
+
from invariants import SEMANTIC_CHECKS, check as check_invariants
|
|
23
28
|
|
|
24
29
|
|
|
25
30
|
SCRIPT_DIR = Path(__file__).resolve().parent
|
|
@@ -27,6 +32,70 @@ FIXTURES = SCRIPT_DIR / "fixtures"
|
|
|
27
32
|
|
|
28
33
|
|
|
29
34
|
class DiagnosticTests(unittest.TestCase):
|
|
35
|
+
def analyze_text(self, text: str, reading_load: bool = False) -> dict:
|
|
36
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
37
|
+
path = Path(directory) / "sample.md"
|
|
38
|
+
path.write_text(text, encoding="utf-8")
|
|
39
|
+
return analyze(path, "tech", reading_load)
|
|
40
|
+
|
|
41
|
+
def test_proofreading_detects_adjacent_duplicates(self) -> None:
|
|
42
|
+
for text, category in (
|
|
43
|
+
("管理者が設定をを確認します。", "duplicate_particle"),
|
|
44
|
+
("管理者がが設定を確認します。", "duplicate_particle"),
|
|
45
|
+
("管理者にに通知します。", "duplicate_particle"),
|
|
46
|
+
("設定設定を保存します。", "duplicate_word"),
|
|
47
|
+
):
|
|
48
|
+
with self.subTest(text=text):
|
|
49
|
+
result = self.analyze_text(text)
|
|
50
|
+
self.assertIn(category, {item["category"] for item in result["findings"]})
|
|
51
|
+
self.assertNotIn(
|
|
52
|
+
category,
|
|
53
|
+
{item["category"] for item in self.analyze_text(text, True)["findings"]},
|
|
54
|
+
)
|
|
55
|
+
|
|
56
|
+
def test_proofreading_checks_brackets_and_paragraph_style(self) -> None:
|
|
57
|
+
for text, category in (
|
|
58
|
+
("設定(管理者向け))を保存します。", "unbalanced_brackets"),
|
|
59
|
+
("設定を保存します。変更はすぐに反映される。", "mixed_sentence_style"),
|
|
60
|
+
):
|
|
61
|
+
with self.subTest(text=text):
|
|
62
|
+
self.assertIn(category, {
|
|
63
|
+
item["category"] for item in self.analyze_text(text)["findings"]
|
|
64
|
+
})
|
|
65
|
+
|
|
66
|
+
def test_proofreading_preserves_names_code_and_deliberate_styles(self) -> None:
|
|
67
|
+
text = (
|
|
68
|
+
"ののさんは、ももを食べます。操作を確認してください。\n\n"
|
|
69
|
+
"「設定を保存する」と説明します。結果を確認します。\n\n"
|
|
70
|
+
"設定を保存する。\n\n設定を確認します。\n\n"
|
|
71
|
+
"`設定設定(`という識別子を使用します。\n\n"
|
|
72
|
+
"```text\n設定をを確認します(\n```\n"
|
|
73
|
+
)
|
|
74
|
+
categories = {item["category"] for item in self.analyze_text(text)["findings"]}
|
|
75
|
+
self.assertFalse(categories & {
|
|
76
|
+
"duplicate_particle", "duplicate_word", "unbalanced_brackets", "mixed_sentence_style",
|
|
77
|
+
})
|
|
78
|
+
|
|
79
|
+
def test_short_fences_and_fence_info_do_not_expose_code_as_prose(self) -> None:
|
|
80
|
+
for text in (
|
|
81
|
+
"````text\n```python\n設定設定(をを確認する。\n```\n````\n",
|
|
82
|
+
"```text\n```python\n設定設定(をを確認する。\n```\n",
|
|
83
|
+
):
|
|
84
|
+
with self.subTest(text=text):
|
|
85
|
+
self.assertEqual(self.analyze_text(text)["findings"], [])
|
|
86
|
+
|
|
87
|
+
def test_questions_and_obligations_are_not_style_errors(self) -> None:
|
|
88
|
+
for text in (
|
|
89
|
+
"この結果だけでは、安全だと言えるでしょうか。",
|
|
90
|
+
"この結果だけでは、安全だと言えるだろうか。",
|
|
91
|
+
"管理者は確認しなくてはならない。",
|
|
92
|
+
"この場合は停止せざるを得ない。",
|
|
93
|
+
):
|
|
94
|
+
with self.subTest(text=text):
|
|
95
|
+
categories = {item["category"] for item in self.analyze_text(text)["findings"]}
|
|
96
|
+
self.assertNotIn("formulaic_conclusion", categories)
|
|
97
|
+
self.assertNotIn("double_negative", categories)
|
|
98
|
+
|
|
30
99
|
def test_unnatural_fixture_scores_below_natural_fixture(self) -> None:
|
|
31
100
|
unnatural = analyze(FIXTURES / "unnatural.md", "tech", False)
|
|
32
101
|
natural = analyze(FIXTURES / "natural.md", "tech", False)
|
|
@@ -113,5 +182,104 @@ class DiagnosticTests(unittest.TestCase):
|
|
|
113
182
|
self.assertTrue(values.isdisjoint({"npm", "run", "build", "workspace", "core"}))
|
|
114
183
|
|
|
115
184
|
|
|
185
|
+
class InvariantTests(unittest.TestCase):
|
|
186
|
+
def test_protected_changes_cannot_be_approved_by_semantic_review(self) -> None:
|
|
187
|
+
original = (
|
|
188
|
+
"---\nowner: admin\n---\n# 概要\n\n"
|
|
189
|
+
"REQ-001ではAcmeが3回、30秒で確認します。帯域は3MB/sです。[仕様](https://example.com/spec)\n\n"
|
|
190
|
+
"`MAX_RETRY_COUNT`を使用します。\n\n"
|
|
191
|
+
"| 項目 | 値 |\n| --- | --- |\n| 上限 | 3 |\n\n"
|
|
192
|
+
"````text\n```python\nprint('old')\n```\n````\n"
|
|
193
|
+
)
|
|
194
|
+
changes = (
|
|
195
|
+
("3回、30秒", "30回、3秒", "numbers_in_order"),
|
|
196
|
+
("30秒", "30分", "quantities_in_order"),
|
|
197
|
+
("3MB/s", "3MB/min", "quantities_in_order"),
|
|
198
|
+
("REQ-001", "REQ-002", "identifiers_in_order"),
|
|
199
|
+
("https://example.com/spec", "https://example.com/new", "link_destinations"),
|
|
200
|
+
("MAX_RETRY_COUNT", "RETRY_COUNT", "inline_code"),
|
|
201
|
+
("| 上限 | 3 |", "| 上限 | 4 |", "protected_blocks"),
|
|
202
|
+
("owner: admin", "owner: user", "frontmatter"),
|
|
203
|
+
("# 概要", "## 概要", "heading_levels"),
|
|
204
|
+
("Acme", "Other", "protected_terms"),
|
|
205
|
+
("print('old')", "print('new')", "protected_blocks"),
|
|
206
|
+
)
|
|
207
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
208
|
+
before, after, review = (Path(directory) / name for name in ("before.md", "after.md", "review.json"))
|
|
209
|
+
before.write_text(original, encoding="utf-8")
|
|
210
|
+
for old, new, category in changes:
|
|
211
|
+
with self.subTest(category=category):
|
|
212
|
+
after.write_text(original.replace(old, new), encoding="utf-8")
|
|
213
|
+
self.write_review(before, after, review)
|
|
214
|
+
result = check_invariants(before, after, ["Acme"], review)
|
|
215
|
+
self.assertEqual(result["status"], "blocked")
|
|
216
|
+
self.assertIn(category, {
|
|
217
|
+
item["category"] for item in result["mechanical_violations"]
|
|
218
|
+
})
|
|
219
|
+
|
|
220
|
+
def write_review(self, before: Path, after: Path, review: Path) -> None:
|
|
221
|
+
review.write_text(json.dumps({
|
|
222
|
+
"before_sha256": hashlib.sha256(before.read_bytes()).hexdigest(),
|
|
223
|
+
"after_sha256": hashlib.sha256(after.read_bytes()).hexdigest(),
|
|
224
|
+
"reviewer": "test-reviewer",
|
|
225
|
+
"checks": {name: True for name in SEMANTIC_CHECKS},
|
|
226
|
+
}), encoding="utf-8")
|
|
227
|
+
|
|
228
|
+
def test_review_must_cover_exact_edits_and_all_semantic_checks(self) -> None:
|
|
229
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
230
|
+
before, after, review = (Path(directory) / name for name in ("before.md", "after.md", "review.json"))
|
|
231
|
+
before.write_text("管理者だけが設定を確認します。", encoding="utf-8")
|
|
232
|
+
after.write_text("設定を確認できるのは管理者だけです。", encoding="utf-8")
|
|
233
|
+
self.assertEqual(check_invariants(before, before)["status"], "passed")
|
|
234
|
+
self.assertEqual(check_invariants(before, after)["status"], "review_required")
|
|
235
|
+
self.write_review(before, after, review)
|
|
236
|
+
self.assertEqual(check_invariants(before, after, review_path=review)["status"], "passed")
|
|
237
|
+
after.write_text("誰でも設定を確認できます。", encoding="utf-8")
|
|
238
|
+
self.assertEqual(check_invariants(before, after, review_path=review)["status"], "review_required")
|
|
239
|
+
self.write_review(before, after, review)
|
|
240
|
+
value = json.loads(review.read_text(encoding="utf-8"))
|
|
241
|
+
value["checks"]["scope"] = False
|
|
242
|
+
review.write_text(json.dumps(value), encoding="utf-8")
|
|
243
|
+
self.assertEqual(check_invariants(before, after, review_path=review)["status"], "review_required")
|
|
244
|
+
|
|
245
|
+
def test_invariant_cli_exit_codes_and_input_errors(self) -> None:
|
|
246
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
247
|
+
before, after = (Path(directory) / name for name in ("before.md", "after.md"))
|
|
248
|
+
before.write_text("上限は3回です。", encoding="utf-8")
|
|
249
|
+
for text, code in (
|
|
250
|
+
("上限は3回です。", 0),
|
|
251
|
+
("上限は30回です。", 2),
|
|
252
|
+
("3回を上限とします。", 3),
|
|
253
|
+
):
|
|
254
|
+
with self.subTest(code=code):
|
|
255
|
+
after.write_text(text, encoding="utf-8")
|
|
256
|
+
process = subprocess.run(
|
|
257
|
+
[sys.executable, str(SCRIPT_DIR / "invariants.py"), str(before), str(after), "--json"],
|
|
258
|
+
capture_output=True, text=True,
|
|
259
|
+
)
|
|
260
|
+
self.assertEqual(process.returncode, code, process.stderr)
|
|
261
|
+
self.assertIn("status", json.loads(process.stdout))
|
|
262
|
+
process = subprocess.run(
|
|
263
|
+
[sys.executable, str(SCRIPT_DIR / "invariants.py"), str(before), str(Path(directory) / "missing.md")],
|
|
264
|
+
capture_output=True, text=True,
|
|
265
|
+
)
|
|
266
|
+
self.assertEqual(process.returncode, 1)
|
|
267
|
+
self.assertIn("error:", process.stderr)
|
|
268
|
+
|
|
269
|
+
def test_units_across_line_breaks_and_boolean_review_types(self) -> None:
|
|
270
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
271
|
+
before, after, review = (Path(directory) / name for name in ("before.md", "after.md", "review.json"))
|
|
272
|
+
before.write_text("上限は30\n秒です。", encoding="utf-8")
|
|
273
|
+
after.write_text("上限は30\n分です。", encoding="utf-8")
|
|
274
|
+
self.assertEqual(check_invariants(before, after)["status"], "blocked")
|
|
275
|
+
before.write_text("設定をを保存します。", encoding="utf-8")
|
|
276
|
+
after.write_text("設定を保存します。", encoding="utf-8")
|
|
277
|
+
self.write_review(before, after, review)
|
|
278
|
+
value = json.loads(review.read_text(encoding="utf-8"))
|
|
279
|
+
value["checks"]["negation"] = "true"
|
|
280
|
+
review.write_text(json.dumps(value), encoding="utf-8")
|
|
281
|
+
self.assertEqual(check_invariants(before, after, review_path=review)["status"], "review_required")
|
|
282
|
+
|
|
283
|
+
|
|
116
284
|
if __name__ == "__main__":
|
|
117
285
|
unittest.main()
|
|
@@ -55,7 +55,9 @@ Never trade technical precision for conversational phrasing.
|
|
|
55
55
|
discovered skill,
|
|
56
56
|
and follow its workflow.
|
|
57
57
|
2. Ask the loaded skill to review and rewrite only the Japanese prose under
|
|
58
|
-
the frozen invariants.
|
|
58
|
+
the frozen invariants. Save an immutable pre-edit copy, pass the frozen
|
|
59
|
+
proper names and domain terms for `--protect`, and keep necessary
|
|
60
|
+
corrections separate from optional wording changes.
|
|
59
61
|
3. Let `<japanese-prose-dir>` be the directory containing the loaded
|
|
60
62
|
optimizer's `SKILL.md`. When its diagnostics are available, run the
|
|
61
63
|
optimizer's scripts with their qualified paths:
|
|
@@ -82,6 +84,12 @@ Never trade technical precision for conversational phrasing.
|
|
|
82
84
|
violating edit, give the violated invariant back to the optimizer, and
|
|
83
85
|
retry within the round limit. If the violation cannot be resolved, report
|
|
84
86
|
`Japanese prose optimization did not converge`.
|
|
87
|
+
Run `<japanese-prose-dir>/scripts/invariants.py` against the immutable
|
|
88
|
+
original and current target. Require a hash-bound contextual semantic
|
|
89
|
+
review for changed prose, following the optimizer's review workflow.
|
|
90
|
+
A `review_required` result or an unavailable/failed invariant gate cannot
|
|
91
|
+
be treated as completion. Rerun after rubber-duck edits; old review hashes
|
|
92
|
+
do not approve new text.
|
|
85
93
|
7. Re-run kotonoha's doctype checklist and structural lint before
|
|
86
94
|
rubber-duck review.
|
|
87
95
|
|
|
@@ -91,9 +99,9 @@ When this pass is in scope (a living, multi-section document whose requested
|
|
|
91
99
|
final language is Japanese, in `write` mode), report exactly one status:
|
|
92
100
|
|
|
93
101
|
- `Japanese prose optimization completed`: the registered optimizer ran and
|
|
94
|
-
no actionable prose findings remain
|
|
95
|
-
|
|
96
|
-
|
|
102
|
+
no actionable prose findings remain, and the invariant gate passed with
|
|
103
|
+
a current semantic review of any edits. A missing optional diagnostic may
|
|
104
|
+
be noted, but a missing or failed invariant gate prevents this status.
|
|
97
105
|
- `Japanese prose optimization not performed`: the bundled optimizer was
|
|
98
106
|
missing or unreadable, so no prose review or rewrite ran; include the
|
|
99
107
|
concrete reason. A standard `npx kotonoha install` should prevent this
|
|
@@ -102,8 +110,8 @@ final language is Japanese, in `write` mode), report exactly one status:
|
|
|
102
110
|
unresolved actionable findings or invariant violations, or the optimizer
|
|
103
111
|
started but failed before producing a valid result, a required
|
|
104
112
|
post-rubber-duck rerun failed or could not start because all 15 handoffs
|
|
105
|
-
were already used
|
|
106
|
-
fixes.
|
|
113
|
+
were already used, or the invariant gate/semantic review could not pass;
|
|
114
|
+
list the findings, violations, failure, and attempted fixes.
|
|
107
115
|
|
|
108
116
|
Do not claim natural-language optimization based only on kotonoha's
|
|
109
117
|
structural lint. Documents outside the scope of this pass require no Japanese
|