kotonoha 0.1.6-dev.0 → 0.3.0

package/skills/japanese-prose/references/review-workflow.md

@@ -0,0 +1,50 @@
+# Japanese prose review workflow
+
+## Review findings in context
+
+For every diagnostic finding, record one decision:
+
+- `fix`: the expression increases ambiguity, reading load, or mechanical
+  repetition for the intended reader
+- `keep`: the expression is required by accuracy, domain convention, quoted
+  material, or deliberate rhythm
+
+Do not count a finding as resolved merely because the triggering text
+disappeared. Confirm that the replacement preserves meaning.
+
+## Check sentence structure
+
+- The subject and predicate can be identified without rereading.
+- Modifiers sit close to the words they modify.
+- A long sentence contains one main relationship.
+- Negation does not require the reader to reverse the meaning twice.
+- Noun chains expose ownership, purpose, and target relationships.
+
+## Check document rhythm
+
+- Consecutive sentences do not begin with the same two lemmas without reason.
+- Paragraphs do not all begin with the same connective.
+- Sentence lengths vary with information weight.
+- Nominal endings are not used as the default ending for every sentence.
+
+## Check terminology
+
+- Product names, identifiers, values, and citations are unchanged.
+- An unfamiliar term is explained near its first use.
+- One concept uses one preferred term unless a distinction is intentional.
+- An acronym is expanded when the audience cannot be expected to know it.
+
+## Check the whole document
+
+Read only headings and paragraph openings. The argument must still be
+predictable. Then read the finished prose continuously and verify that local
+rewrites did not damage transitions or create repeated explanations.
+
+## Convergence
+
+The review is complete when:
+
+- every finding has a `fix` or `keep` decision
+- rerunning diagnostics produces no new actionable finding
+- frozen invariants match the pre-edit document
+- the reader can reach the requested outcome without an unstated inference

package/skills/japanese-prose/references/scoring.md

@@ -0,0 +1,24 @@
+# Diagnostic scoring
+
+The prose lint starts at 100 and subtracts:
+
+- 10 points for each `critical` finding
+- 4 points for each `warning` finding
+- 1 point for each `info` finding
+
+The minimum is 0. Scores compare repeated runs of the same document and
+configuration; they do not compare authors, genres, or unrelated documents.
+
+Use these bands only as triage:
+
+| Score | Interpretation |
+|---|---|
+| 90–100 | Few mechanically detectable review targets |
+| 75–89 | Several passages need contextual review |
+| 50–74 | Reading load or repeated patterns are widespread |
+| 0–49 | Review the document section by section before publication |
+
+A high score does not prove that facts are correct, the argument is complete,
+or the prose is natural. A low score does not require every finding to be
+changed. Always report the finding categories and contextual decisions beside
+the score.

package/skills/japanese-prose/references/writing-guidelines.md

@@ -0,0 +1,60 @@
+# Japanese prose writing guidelines
+
+Use these rules while drafting or rewriting. They are constraints for
+judgment, not search-and-replace recipes.
+
+## Lead with the reader's required conclusion
+
+State the decision, action, result, or answer before the background that
+supports it. An introductory sentence must earn its place by changing what
+the reader understands or does.
+
+## Keep one main relationship per sentence
+
+A sentence may contain qualifications, but its subject, predicate, and main
+object must remain visible. Split a sentence when the reader must retain one
+unfinished relationship while parsing another.
+
+## Prefer explicit relationships over compressed noun chains
+
+Long noun sequences hide whether terms express ownership, purpose, target, or
+sequence. Add particles or a predicate when GiNZA identifies five or more
+consecutive nouns.
+
+## Use concrete actors and actions
+
+Name who checks, changes, approves, records, or observes something. Avoid
+turning actions into abstract nouns when a direct verb is clearer.
+
+## Treat formulaic expressions as review targets
+
+Expressions such as 「と言えるでしょう」 or 「以下のとおりです」 are not
+automatically wrong. Keep them only when their rhetorical function is
+necessary. Otherwise state the evidence, conclusion, or preview directly.
+
+## Vary rhythm according to information weight
+
+Do not force every sentence or paragraph to the same length. Use a short
+sentence for a decisive result. Use a longer sentence only when its
+relationships remain clear.
+
+## Explain terminology at first use
+
+Introduce an unfamiliar term by function before or beside its name. Preserve
+established product names, protocol names, identifiers, and domain terms.
+
+## Preserve uncertainty accurately
+
+Keep distinctions among confirmed facts, estimates, assumptions, and
+recommendations. Do not make prose smoother by increasing certainty.
+
+## Use emphasis sparingly and safely
+
+Emphasize only the phrase that changes the reader's decision. In Markdown,
+separate `**strong emphasis**` from adjacent prose with half-width spaces.
+
+## End with the consequence
+
+Close a section with what the evidence means for the reader's next decision
+or action. Do not append a generic summary when the consequence is already
+clear.

package/skills/japanese-prose/scripts/core.py

@@ -0,0 +1,192 @@
+"""Shared Markdown and GiNZA utilities for kotonoha's Japanese prose tools."""
+
+from __future__ import annotations
+
+import hashlib
+import re
+from dataclasses import asdict, dataclass
+from functools import lru_cache
+from pathlib import Path
+from typing import Iterable
+
+
+FENCE_RE = re.compile(r"^\s{0,3}(`{3,}|~{3,})")
+HEADING_RE = re.compile(r"^\s{0,3}#{1,6}\s+")
+LIST_RE = re.compile(r"^\s*(?:[-+*]|\d+[.)])\s+")
+BLOCKQUOTE_RE = re.compile(r"^\s*>\s?")
+INLINE_CODE_RE = re.compile(r"`[^`\n]*`")
+LINK_RE = re.compile(r"!?\[([^\]]*)\]\([^)]+\)")
+URL_RE = re.compile(r"https?://\S+")
+HTML_TAG_RE = re.compile(r"<[^>]+>")
+SENTENCE_END_RE = re.compile(r"[。！？!?]+(?:[」』）】〉》]*)")
+JAPANESE_RE = re.compile(r"[ぁ-んァ-ヶ一-龠々]")
+
+
+@dataclass
+class Finding:
+    line: int
+    category: str
+    severity: str
+    message: str
+    excerpt: str
+    evidence: dict | None = None
+    status: str | None = None
+
+    def to_dict(self) -> dict:
+        value = asdict(self)
+        return {key: item for key, item in value.items() if item is not None}
+
+
+@dataclass
+class ProseBlock:
+    text: str
+    start_line: int
+
+
+def read_text(path: Path) -> str:
+    if not path.is_file():
+        raise FileNotFoundError(f"file not found: {path}")
+    return path.read_text(encoding="utf-8")
+
+
+def strip_markdown_inline(text: str) -> str:
+    text = INLINE_CODE_RE.sub(" コード ", text)
+    text = LINK_RE.sub(lambda match: match.group(1), text)
+    text = URL_RE.sub(" ", text)
+    text = HTML_TAG_RE.sub(" ", text)
+    text = text.replace("**", "").replace("__", "")
+    text = text.replace("~~", "")
+    return re.sub(r"[ \t]+", " ", text).strip()
+
+
+def iter_content_lines(text: str) -> Iterable[tuple[int, str]]:
+    """Yield line-numbered Markdown outside metadata, code, and comments."""
+    lines = text.splitlines()
+    fence_marker = ""
+    in_frontmatter = bool(lines and lines[0].strip() == "---")
+    in_comment = False
+
+    for index, raw in enumerate(lines, start=1):
+        stripped = raw.strip()
+        if in_frontmatter:
+            if index > 1 and stripped == "---":
+                in_frontmatter = False
+            yield index, ""
+            continue
+
+        fence = FENCE_RE.match(raw)
+        if fence:
+            marker = fence.group(1)[0]
+            if not fence_marker:
+                fence_marker = marker
+            elif marker == fence_marker:
+                fence_marker = ""
+            yield index, ""
+            continue
+        if fence_marker:
+            yield index, ""
+            continue
+
+        if "<!--" in raw:
+            in_comment = True
+        if in_comment:
+            if "-->" in raw:
+                in_comment = False
+            yield index, ""
+            continue
+        yield index, raw
+
+
+def iter_prose_blocks(text: str) -> list[ProseBlock]:
+    """Return Markdown prose blocks while excluding metadata and code."""
+    blocks: list[ProseBlock] = []
+    buffer: list[str] = []
+    buffer_line = 0
+
+    def flush() -> None:
+        nonlocal buffer, buffer_line
+        if buffer:
+            cleaned = strip_markdown_inline("\n".join(buffer))
+            if cleaned and JAPANESE_RE.search(cleaned):
+                blocks.append(ProseBlock(cleaned, buffer_line))
+        buffer = []
+        buffer_line = 0
+
+    for index, raw in iter_content_lines(text):
+        stripped = raw.strip()
+        if not stripped or stripped.startswith("|") or HEADING_RE.match(raw):
+            flush()
+            continue
+
+        list_item = LIST_RE.match(raw)
+        if list_item:
+            flush()
+        line = LIST_RE.sub("", raw)
+        line = BLOCKQUOTE_RE.sub("", line)
+        if not buffer:
+            buffer_line = index
+        buffer.append(line)
+
+    flush()
+    return blocks
+
+
+def sentence_line(block: ProseBlock, start_char: int) -> int:
+    return block.start_line + block.text[:start_char].count("\n")
+
+
+def split_sentences_fallback(text: str) -> Iterable[tuple[str, int]]:
+    start = 0
+    for match in SENTENCE_END_RE.finditer(text):
+        end = match.end()
+        sentence = text[start:end].strip()
+        if sentence:
+            yield sentence, start
+        start = end
+    tail = text[start:].strip()
+    if tail:
+        yield tail, start
+
+
+@lru_cache(maxsize=2)
+def load_ginza(enable_ner: bool = False):
+    import spacy
+
+    disabled = [] if enable_ner else ["ner"]
+    return spacy.load("ja_ginza", disable=disabled)
+
+
+def content_tokens(span) -> list:
+    return [
+        token
+        for token in span
+        if not token.is_space and not token.is_punct and token.pos_ != "SYM"
+    ]
+
+
+def dependency_depth(token) -> int:
+    depth = 0
+    current = token
+    seen: set[int] = set()
+    while current.head.i != current.i and current.i not in seen and depth < 40:
+        seen.add(current.i)
+        current = current.head
+        depth += 1
+    return depth
+
+
+def excerpt(text: str, limit: int = 100) -> str:
+    value = re.sub(r"\s+", " ", text).strip()
+    return value if len(value) <= limit else f"{value[:limit - 1]}…"
+
+
+def finding_key(finding: Finding) -> str:
+    normalized = re.sub(r"\d+", "#", finding.excerpt.lower())
+    raw = f"{finding.category}\0{normalized}".encode()
+    return hashlib.sha256(raw).hexdigest()[:20]
+
+
+def score_findings(findings: list[Finding]) -> int:
+    weights = {"info": 1, "warning": 4, "critical": 10}
+    deduction = sum(weights.get(item.severity, 4) for item in findings)
+    return max(0, 100 - deduction)

package/skills/japanese-prose/scripts/fixtures/natural.md

@@ -0,0 +1,5 @@
+# 日本語診断の確認
+
+この機能は、設定ミスを公開前に見つけます。管理者が確認画面を開き、対象ユーザーの権限と申請内容を照合してください。
+
+権限が一致すれば承認できます。一致しない場合は、申請者へ差し戻します。

package/skills/japanese-prose/scripts/fixtures/unnatural.md

@@ -0,0 +1,5 @@
+# 日本語診断の確認
+
+結論として、この機能を利用することが可能です。また、この機能は非常に重要なポイントです。また、設定を確認することが可能です。また、結果を確認することが可能です。
+
+利用者権限設定変更承認処理実行結果確認画面を利用する場合においては、設定が正しくないことがないとは言えないため、管理者が確認しなくてはならないということが可能であると言えるでしょう。