kotonoha 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.4.0] - 2026-10-19
11
+
12
+ ### Added
13
+
14
+ - Add duplicate-particle/word, Japanese bracket, and paragraph style-mixing
15
+ diagnostics, plus a minimal-edit `proofread` skill mode.
16
+ - Add a Markdown-aware invariant gate with protected-content checks and
17
+ required hash-bound semantic review of edits.
18
+
19
+ ### Fixed
20
+
21
+ - Avoid treating obligation expressions as double-negative errors and
22
+ conclusion questions as formulaic assertions.
23
+ - Keep shorter fence markers and fence-like info strings inside code blocks
24
+ out of prose diagnostics.
25
+ - Require the invariant gate and current semantic review to pass before
26
+ claiming Japanese prose optimization completed.
27
+
10
28
  ## [0.3.0] - 2026-09-30
11
29
 
12
30
  ### Added
package/README-ja.md CHANGED
@@ -97,7 +97,7 @@ White Paper向けの分析を作成して
97
97
 
98
98
  - GitHub Copilot CLIがインストールされていること
99
99
  - tech-writerの構造lintを実行する場合はPython 3.9以上
100
- - 同梱`japanese-prose`の文章lint、読解負荷分析、アウトライン抽出、用語検査を実行する場合はPython 3.10以上と`uv`。GiNZA、`ja_ginza`、spaCyは初回実行時に`uv`が解決
100
+ - 同梱`japanese-prose`の文章lint、読解負荷分析、アウトライン抽出、用語検査、編集前後の不変条件検査を実行する場合はPython 3.10以上と`uv`。GiNZA、`ja_ginza`、spaCy、Markdown解析用の依存関係は、各スクリプトの初回実行時に`uv`が解決
101
101
 
102
102
  ### npmからインストール
103
103
 
@@ -136,6 +136,10 @@ npx kotonoha install --skill all --force
136
136
 
137
137
  kotonohaには独自開発の`japanese-prose`が同梱されるため、文章最適化スキルを別途インストールする必要はありません。`npx kotonoha install`により`tech-writer`と同じスキルルートへ配置され、GiNZAを使った文章lint、読解負荷検査、用語抽出、アウトライン検査、反復的な文章レビューをローカルで実行できます。この実装はkotonoha向けにゼロから開発しており、`natural-japanese`のソースコードを含みません。GiNZAの依存関係とライセンス情報は`skills/japanese-prose/NOTICE.md`に記録しています。
138
138
 
139
+ 文章lintでは、隣接する助詞・単語の重複、和文括弧の不整合、同一段落内の敬体・常体の混在も確認します。指摘は自動置換の指示ではなく、文脈を確認する対象です。`proofread`モードでは、任意の言い換えを避け、誤字と文法上の誤りだけを最小限に修正します。100点満点のスコアは確認箇所を探す補助であり、訂正精度や品質の保証ではありません。
140
+
141
+ 編集前の原文は変更せずに保存します。同梱の`scripts/invariants.py`が、数値、識別子、コード、表、リンク、指定された固有名詞・専門用語などの変更を検査します。編集した文章には、原文と編集後のファイルのハッシュに対応した意味保存レビューが必要です。機械的な一致だけでは、意味が保たれたとは判定しません。レビュー形式と終了コードは、[`review-workflow.md`](skills/japanese-prose/references/review-workflow.md)を参照してください。
142
+
139
143
  自然言語で指示するときは、GiNZAを明示する必要はありません。「日本語文書を自然にする」「読みやすくする」「簡潔にする」「AIらしい定型表現を減らす」といった依頼をすると、同梱の文章最適化ワークフローが呼び出されます。
140
144
 
141
145
  ```text
@@ -150,6 +154,7 @@ README-ja.mdの日本語を自然で読みやすい表現に修正して
150
154
  目的に応じて、次のようにモードを指定できます。
151
155
 
152
156
  - ファイルを修正する: `README-ja.mdの日本語表現を最適化して`
157
+ - 誤りだけを修正する: `README-ja.mdをproofreadモードで校正して。任意の言い換えはせず、誤字と文法上の誤りだけを最小限に修正して`
153
158
  - 修正せずにレビューする: `README-ja.mdの日本語をレビューして。ファイルは修正しないで`
154
159
  - 採点だけ行う: `README-ja.mdの日本語表現を100点満点で採点して`
155
160
  - すべての診断を実行する: `README-ja.mdをfullモードで日本語表現最適化して`
package/README.md CHANGED
@@ -155,7 +155,8 @@ handoff without inventing missing evidence.
155
155
 
156
156
  - GitHub Copilot CLI installed
157
157
  - Python 3.9+ for tech-writer's structural lint
158
- - Python 3.10+ and `uv` for the bundled GiNZA diagnostics such as prose lint,
158
+ - Python 3.10+ and `uv` for the edit-invariant gate and bundled GiNZA
159
+ diagnostics such as prose lint,
159
160
  reading-load analysis, outline extraction, and terminology checks; `uv`
160
161
  resolves GiNZA, `ja_ginza`, and spaCy on first use
161
162
 
@@ -209,6 +210,20 @@ scratch for kotonoha and does not include `natural-japanese` source code.
209
210
  GiNZA dependency and license information is recorded in
210
211
  `skills/japanese-prose/NOTICE.md`.
211
212
 
213
+ The lint also flags adjacent duplicate particles/words, Japanese bracket
214
+ mismatches, and polite/plain style mixing within a paragraph. Findings are
215
+ review targets, not automatic replacements. Use `proofread` mode for minimal
216
+ typo/grammar corrections without optional stylistic rewriting. The 0–100
217
+ score is a triage aid, not correction accuracy or a quality certificate.
218
+
219
+ Before editing, the workflow saves an immutable original. The bundled
220
+ `scripts/invariants.py` checks protected content and requires a hash-bound
221
+ semantic review of changed prose. Numbers, identifiers, code, tables, links,
222
+ and caller-supplied proper names/domain terms cannot be silently changed.
223
+ Mechanical equality alone does not establish preserved meaning. See
224
+ [`review-workflow.md`](skills/japanese-prose/references/review-workflow.md)
225
+ for the review format and gate exit codes.
226
+
212
227
  You do not need to name GiNZA explicitly in a natural-language request.
213
228
  Requests to make a Japanese document more natural, readable, concise, or less
214
229
  formulaic invoke the bundled prose-optimization workflow. For example:
@@ -225,6 +240,7 @@ and repetitive AI-like phrasing, then fix the findings.
225
240
  You can specify the intended mode more precisely:
226
241
 
227
242
  - Edit the file: `Optimize the Japanese prose in README-ja.md.`
243
+ - Correct errors only: `Proofread README-ja.md; make minimal typo/grammar corrections without stylistic rewrites.`
228
244
  - Review without editing: `Review the Japanese in README-ja.md, but do not modify the file.`
229
245
  - Score only: `Score the Japanese prose in README-ja.md out of 100.`
230
246
  - Run every diagnostic: `Run the full Japanese prose optimization workflow on README-ja.md.`
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "kotonoha",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "GitHub Copilot CLI skills for consulting analysis, technical writing, Japanese prose, and PPTX planning.",
5
5
  "license": "MIT",
6
6
  "author": "nahisaho",
@@ -1,4 +1,4 @@
1
- # GiNZA dependency notice
1
+ # Diagnostic dependency notices
2
2
 
3
3
  This skill is an original kotonoha implementation and does not include source
4
4
  code from `coji/natural-japanese`.
@@ -15,3 +15,9 @@ datasets.
15
15
  The implementation targets GiNZA 5.2.x and uses token surface forms, lemmas,
16
16
  universal part-of-speech tags, dependency relationships, sentence boundaries,
17
17
  and named-entity labels.
18
+
19
+ The edit-invariant gate uses
20
+ [markdown-it-py](https://github.com/executablebooks/markdown-it-py), distributed
21
+ under the MIT License, to parse Markdown without treating short fences
22
+ inside longer fenced code blocks as prose. Its dependencies are resolved by
23
+ `uv` when `scripts/invariants.py` is run.
@@ -6,10 +6,10 @@ description: >-
6
6
  reviewing, or scoring Japanese prose; when the user asks for more natural,
7
7
  readable, concise, or less AI-like Japanese; or when tech-writer delegates
8
8
  its sentence-level quality pass. Owns wording, sentence rhythm, reading
9
- load, terminology review, and formulaic-expression detection. Does not own
10
- technical-document structure or presentation storylines.
9
+ load, typo and grammar review, terminology, and edit-invariant checks.
10
+ Does not own technical-document structure or presentation storylines.
11
11
  license: MIT
12
- argument-hint: "[write|review|score] [quick|full] <target file or request>"
12
+ argument-hint: "[write|proofread|review|score] [quick|full] <target file or request>"
13
13
  ---
14
14
 
15
15
  # japanese-prose
@@ -39,6 +39,8 @@ acceptance criteria, approval states, or normative force.
39
39
  - `write`: write or rewrite Japanese prose. Use `quick` unless the document
40
40
  is external-facing, high-risk, or longer than roughly 10,000 Japanese
41
41
  characters.
42
+ - `proofread`: correct demonstrable typos and grammatical errors with minimal
43
+ edits. Do not perform optional stylistic rewrites or chase a higher score.
42
44
  - `review`: report problems and proposed corrections without editing.
43
45
  - `score`: run diagnostics and report the 0–100 score without rewriting.
44
46
  - `quick`: run prose lint once, review findings in context, edit, and rerun
@@ -50,33 +52,57 @@ acceptance criteria, approval states, or normative force.
50
52
  ## Workflow
51
53
 
52
54
  1. Read the target, audience, intended outcome, and calling skill's frozen
53
- invariants.
55
+ invariants. Before editing, save an immutable copy of the entire original
56
+ file as `<workdir>/prose-original.md`. Keep this separate from diagnostic
57
+ baselines, which may change between rounds.
54
58
  2. Read `references/writing-guidelines.md` before generating or rewriting
55
- prose.
59
+ prose. Resolve `<japanese-prose-dir>` to the directory containing this
60
+ loaded skill; do not assume the shell is running in that directory.
56
61
  3. Run the GiNZA prose diagnostic:
57
62
 
58
63
  ```bash
59
- uv run scripts/lint.py <target-file> --genre tech --json > <workdir>/prose-baseline.json
64
+ uv run <japanese-prose-dir>/scripts/lint.py <target-file> --genre tech --json > <workdir>/prose-baseline.json
60
65
  ```
61
66
 
62
67
  4. For `full` mode, also run:
63
68
 
64
69
  ```bash
65
- uv run scripts/lint.py <target-file> --genre tech --reading-load --json
66
- uv run scripts/outline.py <target-file>
67
- uv run scripts/terms.py <target-file> --json
70
+ uv run <japanese-prose-dir>/scripts/lint.py <target-file> --genre tech --reading-load --json
71
+ uv run <japanese-prose-dir>/scripts/outline.py <target-file>
72
+ uv run <japanese-prose-dir>/scripts/terms.py <target-file> --json
68
73
  ```
69
74
 
70
- 5. Classify each finding as `fix` or `keep`. A detector identifies a review
71
- target; it does not authorize blind replacement.
72
- 6. Apply only fixes that improve the intended reader's understanding while
73
- preserving the frozen invariants.
75
+ 5. Read the entire prose for typos, particle errors, subject/predicate
76
+ mismatches, and incorrect inflections even when lint reports no findings.
77
+ Classify each finding as `fix` or `keep`. A detector identifies a review
78
+ target; it does not authorize blind replacement. Adjacent duplicate
79
+ particles/words, Japanese bracket mismatches, and within-paragraph style
80
+ mixing are proofreading targets, not proof of an error.
81
+ 6. Correct demonstrable errors first. In `write` mode, then consider optional
82
+ wording changes that serve the reader's stated goal. Keep these separate
83
+ from necessary corrections. Preserve acceptable wording when a change
84
+ merely substitutes a synonym or removes a legitimate qualification.
85
+ Never infer a factual or proper-name correction from plausibility alone.
74
86
  7. Rerun the prose diagnostic with
75
87
  `--baseline <workdir>/prose-baseline.json` to classify new, persisting,
76
88
  and resolved findings. Replace the baseline only after recording the
77
89
  decisions for the current round.
78
- 8. Stop when every finding has a decision, no new actionable finding appears,
79
- and the document passes the checks in `references/review-workflow.md`.
90
+ 8. Run the edit-invariant gate against the immutable original:
91
+
92
+ ```bash
93
+ uv run <japanese-prose-dir>/scripts/invariants.py <workdir>/prose-original.md <target-file> --protect <proper-name-or-domain-term> --json
94
+ ```
95
+
96
+ Repeat `--protect` for each frozen proper name or domain term; omit it only
97
+ when there are none. Exit 2 blocks changed protected content. Any edit
98
+ requires a contextual semantic review, even if all mechanical checks
99
+ pass: exit 3 means that review is pending or rejected, not completion.
100
+ Follow `references/review-workflow.md` to record the hash-bound review,
101
+ then rerun with `--review <workdir>/semantic-review.json --json`.
102
+ Recheck after every subsequent edit; a stale review cannot approve it.
103
+ 9. Stop only when every finding has a decision, no new actionable finding
104
+ appears, and the invariant gate returns `passed`. If the gate cannot run,
105
+ report the failure rather than declaring completion.
80
106
 
81
107
  Allow at most three edit-and-diagnose rounds per invocation. If actionable
82
108
  findings or invariant violations remain, report that the optimization did not
@@ -108,4 +134,7 @@ Report:
108
134
  - findings fixed and findings deliberately kept
109
135
  - any skipped diagnostic and its reason
110
136
  - invariant verification result
137
+ - necessary corrections versus optional wording changes
138
+ - semantic review status and any remaining uncertainty; mechanical equality
139
+ is not a guarantee of preserved meaning
111
140
  - `completed` or `did not converge`
@@ -40,6 +40,61 @@ Read only headings and paragraph openings. The argument must still be
40
40
  predictable. Then read the finished prose continuously and verify that local
41
41
  rewrites did not damage transitions or create repeated explanations.
42
42
 
43
+ ## Require a semantic review of edits
44
+
45
+ Run `scripts/invariants.py` on the immutable pre-edit file and the edited
46
+ file. It protects numbers and known quantity units in occurrence order,
47
+ uppercase identifiers, URL/link destinations, frontmatter, heading levels,
48
+ inline code, fenced/indented code, Markdown tables, HTML, and caller-supplied
49
+ `--protect` terms. The Markdown parser recognizes nested and long fences.
50
+ The check is conservative: it may block a legitimate reorder or protected
51
+ content correction. Such changes need separate authorization, not a
52
+ reviewer override of a failed invariant.
53
+
54
+ Mechanical equality cannot establish equivalent meaning. For every changed
55
+ passage, compare the original and edited text, including nearby context:
56
+
57
+ - `actors`: who performs or receives each action is unchanged
58
+ - `negation`: polarity and the scope of negation are unchanged
59
+ - `conditions`: prerequisites, exceptions, acceptance criteria, and approval states are unchanged
60
+ - `modality`: obligations, recommendations, permissions, and uncertainty are unchanged
61
+ - `scope`: limitations such as 「だけ」「すべて」「少なくとも」 are unchanged
62
+ - `causality`: causal and temporal relationships are unchanged
63
+ - `quantity_bindings`: each quantity still refers to the same object or action
64
+ - `terminology`: proper names and domain concepts are unchanged
65
+ - `structure`: section identity, order, quoted meaning, and traceability are unchanged
66
+
67
+ Record the review as JSON. Copy `before_sha256` and `after_sha256` from the
68
+ gate output, identify the actual reviewer, and set a check to `true` only
69
+ after inspecting it. Do not infer approval from a score, an unchanged token
70
+ count, or the absence of warnings. A review can be performed by the agent or
71
+ a human, but an agent review is not a human approval or a semantic proof.
72
+
73
+ ```json
74
+ {
75
+ "before_sha256": "<hash from the gate>",
76
+ "after_sha256": "<hash from the gate>",
77
+ "reviewer": "<actual reviewer identifier>",
78
+ "checks": {
79
+ "actors": false,
80
+ "negation": false,
81
+ "conditions": false,
82
+ "modality": false,
83
+ "scope": false,
84
+ "causality": false,
85
+ "quantity_bindings": false,
86
+ "terminology": false,
87
+ "structure": false
88
+ }
89
+ }
90
+ ```
91
+
92
+ Rerun the gate with `--review <review-file>`. Exit codes are 0 for `passed`,
93
+ 1 for input/runtime errors, 2 for protected-content violations, and 3 for
94
+ missing, stale, incomplete, or rejected semantic reviews. Subsequent edits
95
+ invalidate the review. If equivalence is uncertain, keep the original
96
+ wording or report the unresolved concern.
97
+
43
98
  ## Convergence
44
99
 
45
100
  The review is complete when:
@@ -47,4 +102,6 @@ The review is complete when:
47
102
  - every finding has a `fix` or `keep` decision
48
103
  - rerunning diagnostics produces no new actionable finding
49
104
  - frozen invariants match the pre-edit document
105
+ - `scripts/invariants.py` returns `passed` with a current semantic review
106
+ for changed text
50
107
  - the reader can reach the requested outcome without an unstated inference
@@ -22,3 +22,9 @@ A high score does not prove that facts are correct, the argument is complete,
22
22
  or the prose is natural. A low score does not require every finding to be
23
23
  changed. Always report the finding categories and contextual decisions beside
24
24
  the score.
25
+
26
+ Proofreading findings and style/readability findings contribute to the same
27
+ triage score, but they are not interchangeable measures of quality. Report
28
+ the categories separately. Do not use this score as typo-detection recall,
29
+ correction accuracy, semantic fidelity, or evidence that an LLM-assisted
30
+ rewrite is better. A correctly repaired typo may leave the score unchanged.
@@ -76,10 +76,14 @@ def iter_content_lines(text: str) -> Iterable[tuple[int, str]]:
76
76
 
77
77
  fence = FENCE_RE.match(raw)
78
78
  if fence:
79
- marker = fence.group(1)[0]
79
+ marker = fence.group(1)
80
80
  if not fence_marker:
81
81
  fence_marker = marker
82
- elif marker == fence_marker:
82
+ elif (
83
+ marker[0] == fence_marker[0]
84
+ and len(marker) >= len(fence_marker)
85
+ and not raw[fence.end():].strip()
86
+ ):
83
87
  fence_marker = ""
84
88
  yield index, ""
85
89
  continue
@@ -0,0 +1,182 @@
1
+ #!/usr/bin/env python3
2
+ # /// script
3
+ # requires-python = ">=3.10"
4
+ # dependencies = ["markdown-it-py>=3,<5"]
5
+ # ///
6
+ """Check protected content and require a hash-bound semantic review of edits."""
7
+
8
+ from __future__ import annotations
9
+
10
+ import argparse
11
+ import hashlib
12
+ import json
13
+ from pathlib import Path
14
+ import re
15
+ import sys
16
+
17
+ from markdown_it import MarkdownIt
18
+
19
+
20
+ SEMANTIC_CHECKS = (
21
+ "actors", "negation", "conditions", "modality", "scope", "causality",
22
+ "quantity_bindings", "terminology", "structure",
23
+ )
24
+ NUMBER_RE = re.compile(r"\d+(?:[.,]\d+)*")
25
+ IDENTIFIER_RE = re.compile(r"(?<![A-Za-z0-9_])[A-Z][A-Z0-9]*(?:[-_][A-Z0-9]+)+(?![A-Za-z0-9_])")
26
+ QUANTITY_RE = re.compile(
27
+ r"(\d+(?:[.,]\d+)*)\s*"
28
+ r"(ミリ秒|時間|年間|か月|ヶ月|秒|分|日|週|月|年|回|件|台|人|個|円|倍|"
29
+ r"[A-Za-zµμ]+(?:/[A-Za-zµμ]+)?|%|%|℃)(?![A-Za-z])"
30
+ )
31
+ BARE_URL_RE = re.compile(r"https?://[^\s<>`\"()\[\]、。!?「」『』()]+")
32
+
33
+
34
+ def fingerprint(value: object) -> str:
35
+ return hashlib.sha256(
36
+ json.dumps(value, ensure_ascii=False, sort_keys=True).encode("utf-8")
37
+ ).hexdigest()
38
+
39
+
40
+ def snapshot(text: str, protected_terms: list[str]) -> dict:
41
+ lines = text.splitlines(keepends=True)
42
+ frontmatter = ""
43
+ body = text
44
+ if lines and lines[0].strip() == "---":
45
+ end = next(
46
+ (index for index in range(1, len(lines)) if lines[index].strip() in {"---", "..."}),
47
+ None,
48
+ )
49
+ if end is not None:
50
+ frontmatter = "".join(lines[:end + 1])
51
+ body = "".join(lines[end + 1:])
52
+ body_lines = body.splitlines(keepends=True)
53
+ tokens = MarkdownIt("commonmark").enable("table").parse(body)
54
+ blocks = []
55
+ headings = []
56
+ code = []
57
+ links = []
58
+ html = []
59
+ for token in tokens:
60
+ if token.type in {"fence", "code_block", "table_open", "html_block"}:
61
+ if token.map is None:
62
+ raise ValueError(f"Missing source map for {token.type}")
63
+ start, end = token.map
64
+ blocks.append((token.type, "".join(body_lines[start:end])))
65
+ if token.type == "heading_open":
66
+ headings.append(token.tag)
67
+ for child in token.children or []:
68
+ if child.type == "code_inline":
69
+ code.append(child.content)
70
+ elif child.type == "link_open":
71
+ links.append(child.attrGet("href"))
72
+ elif child.type == "image":
73
+ links.append(child.attrGet("src"))
74
+ elif child.type == "html_inline":
75
+ html.append(child.content)
76
+ return {
77
+ "frontmatter": frontmatter,
78
+ "protected_blocks": blocks,
79
+ "heading_levels": headings,
80
+ "inline_code": code,
81
+ "link_destinations": links,
82
+ "html_tags": html,
83
+ "numbers_in_order": NUMBER_RE.findall(text),
84
+ "quantities_in_order": QUANTITY_RE.findall(text),
85
+ "identifiers_in_order": IDENTIFIER_RE.findall(text),
86
+ "bare_urls_in_order": BARE_URL_RE.findall(text),
87
+ "protected_terms": {term: text.count(term) for term in protected_terms},
88
+ }
89
+
90
+
91
+ def check(
92
+ before: Path,
93
+ after: Path,
94
+ protected_terms: list[str] | None = None,
95
+ review_path: Path | None = None,
96
+ ) -> dict:
97
+ terms = protected_terms or []
98
+ if any(not term.strip() for term in terms):
99
+ raise ValueError("Protected terms must not be empty")
100
+ original = before.read_bytes()
101
+ edited = after.read_bytes()
102
+ before_hash = hashlib.sha256(original).hexdigest()
103
+ after_hash = hashlib.sha256(edited).hexdigest()
104
+ initial = snapshot(original.decode("utf-8"), terms)
105
+ final = snapshot(edited.decode("utf-8"), terms)
106
+ violations = [
107
+ {
108
+ "category": name,
109
+ "message": f"Protected content changed: {name}",
110
+ "before_fingerprint": fingerprint(initial[name]),
111
+ "after_fingerprint": fingerprint(final[name]),
112
+ }
113
+ for name in initial if initial[name] != final[name]
114
+ ]
115
+ review_errors = []
116
+ review_required = original != edited
117
+ reviewer = None
118
+ if review_path is not None:
119
+ review = json.loads(review_path.read_text(encoding="utf-8"))
120
+ if not isinstance(review, dict):
121
+ raise ValueError("Semantic review must be a JSON object")
122
+ reviewer = review.get("reviewer")
123
+ if not isinstance(reviewer, str) or not reviewer.strip():
124
+ review_errors.append("reviewer must be a non-empty identifier")
125
+ if review.get("before_sha256") != before_hash or review.get("after_sha256") != after_hash:
126
+ review_errors.append("review hashes do not match this exact before/after pair")
127
+ checks = review.get("checks")
128
+ if not isinstance(checks, dict):
129
+ review_errors.append("checks must be an object")
130
+ else:
131
+ review_errors.extend(
132
+ f"{name} has not been affirmed" for name in SEMANTIC_CHECKS
133
+ if checks.get(name) is not True
134
+ )
135
+ elif review_required:
136
+ review_errors.append("changed text requires a hash-bound semantic review")
137
+ status = "blocked" if violations else "review_required" if review_errors else "passed"
138
+ return {
139
+ "before": str(before),
140
+ "after": str(after),
141
+ "before_sha256": before_hash,
142
+ "after_sha256": after_hash,
143
+ "status": status,
144
+ "mechanical_violations": violations,
145
+ "semantic_review_required": review_required,
146
+ "semantic_checks": list(SEMANTIC_CHECKS),
147
+ "semantic_review_errors": review_errors,
148
+ "reviewer": reviewer,
149
+ "limitations": [
150
+ "Mechanical equality does not prove semantic equivalence.",
151
+ "Proper names and domain terms require caller-supplied --protect values and contextual review.",
152
+ "Semantic checks are reviewer attestations, not automated entailment judgments.",
153
+ ],
154
+ }
155
+
156
+
157
+ def main() -> int:
158
+ parser = argparse.ArgumentParser(description=__doc__)
159
+ parser.add_argument("before")
160
+ parser.add_argument("after")
161
+ parser.add_argument("--protect", action="append", default=[])
162
+ parser.add_argument("--review", type=Path)
163
+ parser.add_argument("--json", action="store_true")
164
+ args = parser.parse_args()
165
+ try:
166
+ result = check(Path(args.before), Path(args.after), args.protect, args.review)
167
+ except (OSError, ValueError, UnicodeError) as error:
168
+ print(f"error: {error}", file=sys.stderr)
169
+ return 1
170
+ if args.json:
171
+ print(json.dumps(result, ensure_ascii=False, indent=2))
172
+ else:
173
+ print(f"{result['status']}: {args.before} -> {args.after}")
174
+ for violation in result["mechanical_violations"]:
175
+ print(f" {violation['message']}")
176
+ for error in result["semantic_review_errors"]:
177
+ print(f" Semantic review: {error}")
178
+ return {"passed": 0, "blocked": 2, "review_required": 3}[result["status"]]
179
+
180
+
181
+ if __name__ == "__main__":
182
+ sys.exit(main())
@@ -75,10 +75,12 @@ DOUBLE_NEGATIVE_RULES = (
75
75
  re.compile(r"ないわけではない"),
76
76
  re.compile(r"ないことはない"),
77
77
  re.compile(r"なくはない"),
78
- re.compile(r"なくてはならない"),
79
- re.compile(r"ざるを得ない"),
80
78
  )
81
79
 
80
+ BRACKET_PAIRS = dict(zip("([{「『【", ")]}」』】"))
81
+ POLITE_END_RE = re.compile(r"(?:です|ます|でした|ました|ません(?:でした)?|でしょう|ましょう|ください|下さい)$")
82
+ PLAIN_END_RE = re.compile(r"(?:である|だった|だ|る|た|ない)$")
83
+
82
84
  GENRE_LIMITS = {
83
85
  "general": {"length": 90, "critical_length": 150, "commas": 5, "depth": 8},
84
86
  "tech": {"length": 110, "critical_length": 180, "commas": 6, "depth": 9},
@@ -100,6 +102,64 @@ def add(
100
102
  )
101
103
 
102
104
 
105
+ def proofread(block, doc, findings: list[Finding]) -> None:
106
+ stack: list[str] = []
107
+ unbalanced = False
108
+ for character in block.text:
109
+ if character in BRACKET_PAIRS:
110
+ stack.append(character)
111
+ elif character in BRACKET_PAIRS.values():
112
+ if not stack or BRACKET_PAIRS[stack[-1]] != character:
113
+ unbalanced = True
114
+ else:
115
+ stack.pop()
116
+ if stack or unbalanced:
117
+ add(
118
+ findings, block.start_line, "unbalanced_brackets", "warning",
119
+ "括弧の対応が取れていません。引用や数式として意図した表記か確認します。",
120
+ block.text,
121
+ )
122
+
123
+ for previous, current in zip(doc, doc[1:]):
124
+ if previous.idx + len(previous.text) != current.idx or previous.text != current.text:
125
+ continue
126
+ particle = previous.pos_ == current.pos_ == "ADP"
127
+ word = (
128
+ len(current.text) >= 2
129
+ and previous.pos_ == current.pos_
130
+ and current.pos_ in {"NOUN", "VERB", "ADJ", "ADV"}
131
+ )
132
+ if particle or word:
133
+ add(
134
+ findings, sentence_line(block, current.idx),
135
+ "duplicate_particle" if particle else "duplicate_word", "warning",
136
+ "同じ助詞が隣接しています。誤入力による重複か確認します。"
137
+ if particle else "同じ語が隣接しています。意図した反復か確認します。",
138
+ current.sent.text, {"repeated": current.text},
139
+ )
140
+
141
+ first_style = None
142
+ for sent in doc.sents:
143
+ if re.search(r'[「」『』"“”]', sent.text):
144
+ continue
145
+ ending = sent.text.strip().rstrip("。!?!?")
146
+ style = (
147
+ "polite" if POLITE_END_RE.search(ending)
148
+ else "plain" if PLAIN_END_RE.search(ending)
149
+ else None
150
+ )
151
+ if style and first_style and style != first_style:
152
+ add(
153
+ findings, sentence_line(block, sent.start_char),
154
+ "mixed_sentence_style", "warning",
155
+ "同じ段落に敬体と常体が混在しています。引用や意図した使い分けか確認します。",
156
+ sent.text,
157
+ )
158
+ break
159
+ if style:
160
+ first_style = style
161
+
162
+
103
163
  def analyze(path: Path, genre: str, reading_load: bool) -> dict:
104
164
  text = read_text(path)
105
165
  blocks = iter_prose_blocks(text)
@@ -110,12 +170,17 @@ def analyze(path: Path, genre: str, reading_load: bool) -> dict:
110
170
  paragraph_leads: list[tuple[int, str]] = []
111
171
 
112
172
  for block, doc in zip(blocks, nlp.pipe([item.text for item in blocks])):
173
+ if not reading_load:
174
+ proofread(block, doc, findings)
113
175
  paragraph_tokens = content_tokens(doc)
114
176
  if paragraph_tokens:
115
177
  paragraph_leads.append((block.start_line, paragraph_tokens[0].lemma_))
116
178
 
117
179
  for phrase, (category, message) in PHRASE_RULES.items():
118
- if not reading_load and phrase in block.text:
180
+ pattern = re.escape(phrase)
181
+ if category == "formulaic_conclusion":
182
+ pattern += r"(?!か|[??])"
183
+ if not reading_load and re.search(pattern, block.text):
119
184
  add(findings, block.start_line, category, "warning", message, block.text)
120
185
 
121
186
  if not reading_load:
@@ -5,6 +5,7 @@
5
5
  # "ginza>=5.2.0,<5.3",
6
6
  # "ja-ginza>=5.2.0,<5.3",
7
7
  # "spacy>=3.8.0,<4",
8
+ # "markdown-it-py>=3,<5",
8
9
  # ]
9
10
  # ///
10
11
  """Behavioral tests for kotonoha's original GiNZA diagnostics."""
@@ -12,6 +13,9 @@
12
13
  from __future__ import annotations
13
14
 
14
15
  import json
16
+ import hashlib
17
+ import subprocess
18
+ import sys
15
19
  import tempfile
16
20
  import unittest
17
21
  from pathlib import Path
@@ -20,6 +24,7 @@ from core import strip_markdown_inline
20
24
  from lint import analyze, compare_baseline
21
25
  from outline import extract as extract_outline
22
26
  from terms import extract as extract_terms
27
+ from invariants import SEMANTIC_CHECKS, check as check_invariants
23
28
 
24
29
 
25
30
  SCRIPT_DIR = Path(__file__).resolve().parent
@@ -27,6 +32,70 @@ FIXTURES = SCRIPT_DIR / "fixtures"
27
32
 
28
33
 
29
34
  class DiagnosticTests(unittest.TestCase):
35
+ def analyze_text(self, text: str, reading_load: bool = False) -> dict:
36
+ with tempfile.TemporaryDirectory() as directory:
37
+ path = Path(directory) / "sample.md"
38
+ path.write_text(text, encoding="utf-8")
39
+ return analyze(path, "tech", reading_load)
40
+
41
+ def test_proofreading_detects_adjacent_duplicates(self) -> None:
42
+ for text, category in (
43
+ ("管理者が設定をを確認します。", "duplicate_particle"),
44
+ ("管理者がが設定を確認します。", "duplicate_particle"),
45
+ ("管理者にに通知します。", "duplicate_particle"),
46
+ ("設定設定を保存します。", "duplicate_word"),
47
+ ):
48
+ with self.subTest(text=text):
49
+ result = self.analyze_text(text)
50
+ self.assertIn(category, {item["category"] for item in result["findings"]})
51
+ self.assertNotIn(
52
+ category,
53
+ {item["category"] for item in self.analyze_text(text, True)["findings"]},
54
+ )
55
+
56
+ def test_proofreading_checks_brackets_and_paragraph_style(self) -> None:
57
+ for text, category in (
58
+ ("設定(管理者向け))を保存します。", "unbalanced_brackets"),
59
+ ("設定を保存します。変更はすぐに反映される。", "mixed_sentence_style"),
60
+ ):
61
+ with self.subTest(text=text):
62
+ self.assertIn(category, {
63
+ item["category"] for item in self.analyze_text(text)["findings"]
64
+ })
65
+
66
+ def test_proofreading_preserves_names_code_and_deliberate_styles(self) -> None:
67
+ text = (
68
+ "ののさんは、ももを食べます。操作を確認してください。\n\n"
69
+ "「設定を保存する」と説明します。結果を確認します。\n\n"
70
+ "設定を保存する。\n\n設定を確認します。\n\n"
71
+ "`設定設定(`という識別子を使用します。\n\n"
72
+ "```text\n設定をを確認します(\n```\n"
73
+ )
74
+ categories = {item["category"] for item in self.analyze_text(text)["findings"]}
75
+ self.assertFalse(categories & {
76
+ "duplicate_particle", "duplicate_word", "unbalanced_brackets", "mixed_sentence_style",
77
+ })
78
+
79
+ def test_short_fences_and_fence_info_do_not_expose_code_as_prose(self) -> None:
80
+ for text in (
81
+ "````text\n```python\n設定設定(をを確認する。\n```\n````\n",
82
+ "```text\n```python\n設定設定(をを確認する。\n```\n",
83
+ ):
84
+ with self.subTest(text=text):
85
+ self.assertEqual(self.analyze_text(text)["findings"], [])
86
+
87
+ def test_questions_and_obligations_are_not_style_errors(self) -> None:
88
+ for text in (
89
+ "この結果だけでは、安全だと言えるでしょうか。",
90
+ "この結果だけでは、安全だと言えるだろうか。",
91
+ "管理者は確認しなくてはならない。",
92
+ "この場合は停止せざるを得ない。",
93
+ ):
94
+ with self.subTest(text=text):
95
+ categories = {item["category"] for item in self.analyze_text(text)["findings"]}
96
+ self.assertNotIn("formulaic_conclusion", categories)
97
+ self.assertNotIn("double_negative", categories)
98
+
30
99
  def test_unnatural_fixture_scores_below_natural_fixture(self) -> None:
31
100
  unnatural = analyze(FIXTURES / "unnatural.md", "tech", False)
32
101
  natural = analyze(FIXTURES / "natural.md", "tech", False)
@@ -113,5 +182,104 @@ class DiagnosticTests(unittest.TestCase):
113
182
  self.assertTrue(values.isdisjoint({"npm", "run", "build", "workspace", "core"}))
114
183
 
115
184
 
185
+ class InvariantTests(unittest.TestCase):
186
+ def test_protected_changes_cannot_be_approved_by_semantic_review(self) -> None:
187
+ original = (
188
+ "---\nowner: admin\n---\n# 概要\n\n"
189
+ "REQ-001ではAcmeが3回、30秒で確認します。帯域は3MB/sです。[仕様](https://example.com/spec)\n\n"
190
+ "`MAX_RETRY_COUNT`を使用します。\n\n"
191
+ "| 項目 | 値 |\n| --- | --- |\n| 上限 | 3 |\n\n"
192
+ "````text\n```python\nprint('old')\n```\n````\n"
193
+ )
194
+ changes = (
195
+ ("3回、30秒", "30回、3秒", "numbers_in_order"),
196
+ ("30秒", "30分", "quantities_in_order"),
197
+ ("3MB/s", "3MB/min", "quantities_in_order"),
198
+ ("REQ-001", "REQ-002", "identifiers_in_order"),
199
+ ("https://example.com/spec", "https://example.com/new", "link_destinations"),
200
+ ("MAX_RETRY_COUNT", "RETRY_COUNT", "inline_code"),
201
+ ("| 上限 | 3 |", "| 上限 | 4 |", "protected_blocks"),
202
+ ("owner: admin", "owner: user", "frontmatter"),
203
+ ("# 概要", "## 概要", "heading_levels"),
204
+ ("Acme", "Other", "protected_terms"),
205
+ ("print('old')", "print('new')", "protected_blocks"),
206
+ )
207
+ with tempfile.TemporaryDirectory() as directory:
208
+ before, after, review = (Path(directory) / name for name in ("before.md", "after.md", "review.json"))
209
+ before.write_text(original, encoding="utf-8")
210
+ for old, new, category in changes:
211
+ with self.subTest(category=category):
212
+ after.write_text(original.replace(old, new), encoding="utf-8")
213
+ self.write_review(before, after, review)
214
+ result = check_invariants(before, after, ["Acme"], review)
215
+ self.assertEqual(result["status"], "blocked")
216
+ self.assertIn(category, {
217
+ item["category"] for item in result["mechanical_violations"]
218
+ })
219
+
220
+ def write_review(self, before: Path, after: Path, review: Path) -> None:
221
+ review.write_text(json.dumps({
222
+ "before_sha256": hashlib.sha256(before.read_bytes()).hexdigest(),
223
+ "after_sha256": hashlib.sha256(after.read_bytes()).hexdigest(),
224
+ "reviewer": "test-reviewer",
225
+ "checks": {name: True for name in SEMANTIC_CHECKS},
226
+ }), encoding="utf-8")
227
+
228
+ def test_review_must_cover_exact_edits_and_all_semantic_checks(self) -> None:
229
+ with tempfile.TemporaryDirectory() as directory:
230
+ before, after, review = (Path(directory) / name for name in ("before.md", "after.md", "review.json"))
231
+ before.write_text("管理者だけが設定を確認します。", encoding="utf-8")
232
+ after.write_text("設定を確認できるのは管理者だけです。", encoding="utf-8")
233
+ self.assertEqual(check_invariants(before, before)["status"], "passed")
234
+ self.assertEqual(check_invariants(before, after)["status"], "review_required")
235
+ self.write_review(before, after, review)
236
+ self.assertEqual(check_invariants(before, after, review_path=review)["status"], "passed")
237
+ after.write_text("誰でも設定を確認できます。", encoding="utf-8")
238
+ self.assertEqual(check_invariants(before, after, review_path=review)["status"], "review_required")
239
+ self.write_review(before, after, review)
240
+ value = json.loads(review.read_text(encoding="utf-8"))
241
+ value["checks"]["scope"] = False
242
+ review.write_text(json.dumps(value), encoding="utf-8")
243
+ self.assertEqual(check_invariants(before, after, review_path=review)["status"], "review_required")
244
+
245
+ def test_invariant_cli_exit_codes_and_input_errors(self) -> None:
246
+ with tempfile.TemporaryDirectory() as directory:
247
+ before, after = (Path(directory) / name for name in ("before.md", "after.md"))
248
+ before.write_text("上限は3回です。", encoding="utf-8")
249
+ for text, code in (
250
+ ("上限は3回です。", 0),
251
+ ("上限は30回です。", 2),
252
+ ("3回を上限とします。", 3),
253
+ ):
254
+ with self.subTest(code=code):
255
+ after.write_text(text, encoding="utf-8")
256
+ process = subprocess.run(
257
+ [sys.executable, str(SCRIPT_DIR / "invariants.py"), str(before), str(after), "--json"],
258
+ capture_output=True, text=True,
259
+ )
260
+ self.assertEqual(process.returncode, code, process.stderr)
261
+ self.assertIn("status", json.loads(process.stdout))
262
+ process = subprocess.run(
263
+ [sys.executable, str(SCRIPT_DIR / "invariants.py"), str(before), str(Path(directory) / "missing.md")],
264
+ capture_output=True, text=True,
265
+ )
266
+ self.assertEqual(process.returncode, 1)
267
+ self.assertIn("error:", process.stderr)
268
+
269
+ def test_units_across_line_breaks_and_boolean_review_types(self) -> None:
270
+ with tempfile.TemporaryDirectory() as directory:
271
+ before, after, review = (Path(directory) / name for name in ("before.md", "after.md", "review.json"))
272
+ before.write_text("上限は30\n秒です。", encoding="utf-8")
273
+ after.write_text("上限は30\n分です。", encoding="utf-8")
274
+ self.assertEqual(check_invariants(before, after)["status"], "blocked")
275
+ before.write_text("設定をを保存します。", encoding="utf-8")
276
+ after.write_text("設定を保存します。", encoding="utf-8")
277
+ self.write_review(before, after, review)
278
+ value = json.loads(review.read_text(encoding="utf-8"))
279
+ value["checks"]["negation"] = "true"
280
+ review.write_text(json.dumps(value), encoding="utf-8")
281
+ self.assertEqual(check_invariants(before, after, review_path=review)["status"], "review_required")
282
+
283
+
116
284
  if __name__ == "__main__":
117
285
  unittest.main()
@@ -55,7 +55,9 @@ Never trade technical precision for conversational phrasing.
55
55
  discovered skill,
56
56
  and follow its workflow.
57
57
  2. Ask the loaded skill to review and rewrite only the Japanese prose under
58
- the frozen invariants.
58
+ the frozen invariants. Save an immutable pre-edit copy, pass the frozen
59
+ proper names and domain terms for `--protect`, and keep necessary
60
+ corrections separate from optional wording changes.
59
61
  3. Let `<japanese-prose-dir>` be the directory containing the loaded
60
62
  optimizer's `SKILL.md`. When its diagnostics are available, run the
61
63
  optimizer's scripts with their qualified paths:
@@ -82,6 +84,12 @@ Never trade technical precision for conversational phrasing.
82
84
  violating edit, give the violated invariant back to the optimizer, and
83
85
  retry within the round limit. If the violation cannot be resolved, report
84
86
  `Japanese prose optimization did not converge`.
87
+ Run `<japanese-prose-dir>/scripts/invariants.py` against the immutable
88
+ original and current target. Require a hash-bound contextual semantic
89
+ review for changed prose, following the optimizer's review workflow.
90
+ A `review_required` result or an unavailable/failed invariant gate cannot
91
+ be treated as completion. Rerun after rubber-duck edits; old review hashes
92
+ do not approve new text.
85
93
  7. Re-run kotonoha's doctype checklist and structural lint before
86
94
  rubber-duck review.
87
95
 
@@ -91,9 +99,9 @@ When this pass is in scope (a living, multi-section document whose requested
91
99
  final language is Japanese, in `write` mode), report exactly one status:
92
100
 
93
101
  - `Japanese prose optimization completed`: the registered optimizer ran and
94
- no actionable prose findings remain. Run diagnostics when available; if
95
- `uv` or a diagnostic script is unavailable, note that limitation without
96
- changing this status.
102
+ no actionable prose findings remain, and the invariant gate passed with
103
+ a current semantic review of any edits. A missing optional diagnostic may
104
+ be noted, but a missing or failed invariant gate prevents this status.
97
105
  - `Japanese prose optimization not performed`: the bundled optimizer was
98
106
  missing or unreadable, so no prose review or rewrite ran; include the
99
107
  concrete reason. A standard `npx kotonoha install` should prevent this
@@ -102,8 +110,8 @@ final language is Japanese, in `write` mode), report exactly one status:
102
110
  unresolved actionable findings or invariant violations, or the optimizer
103
111
  started but failed before producing a valid result, a required
104
112
  post-rubber-duck rerun failed or could not start because all 15 handoffs
105
- were already used; list the findings, violations, failure, and attempted
106
- fixes.
113
+ were already used, or the invariant gate/semantic review could not pass;
114
+ list the findings, violations, failure, and attempted fixes.
107
115
 
108
116
  Do not claim natural-language optimization based only on kotonoha's
109
117
  structural lint. Documents outside the scope of this pass require no Japanese