asciidoc-pubkit 0.8.7 → 1.0.0

data/docs/architecture.md

@@ -0,0 +1,264 @@
+# How review and scoring work
+
+### Features
+
+- **Structure-aware AsciiDoc parsing.** Asciidoctor parses local includes,
+  conditionals, attributes and section hierarchy before review targets are
+  selected. Prose selection distinguishes paragraphs from lists, tables,
+  quotations and code blocks; supported inline constructs are masked rather
+  than treated as ordinary prose.
+- **Separate heading and prose review.** Each scan selects one
+  scope with its own rules, prompt and edit permissions. Heading review uses
+  the outline and body as context while protecting body text; prose review
+  protects headings. Mixed heading forms are accepted by default. See
+  [Heading review](review-workflow.md#heading-review).
+- **Source-aligned static analysis.** Candidates identify the original source
+  file, line and one-based Unicode column, with a rule ID and review question.
+  Source text is checked against parser locations; unresolved mappings produce
+  coverage notices instead of guessed edit targets. Matches locate passages
+  for review, not proven defects.
+- **Japanese morphological analysis.** MeCab with UTF-8 IPADIC recognizes
+  configured inflected predicates and adjectives, retaining original surfaces,
+  dictionary forms and negative-form information in morphological prose
+  findings. Literal matching is available only through explicit selection.
+- **Contextual review in one prompt.** A single Markdown file carries saved
+  criteria, settings, candidates and manuscript evidence. All selected prose
+  paragraphs appear once, including those without candidates, with heading and
+  neighboring context. The reviewer records decisions and checks meaning;
+  generating the prompt does not edit the manuscript or invoke a model.
+- **Book-specific rules and terminology.** Strict YAML rule sets, a glossary,
+  allow lists and exclusions adapt candidate detection to the manuscript.
+  Heading rules can express a book's preferred form while allowing justified
+  exceptions. Resolved rules and criteria are saved with each session.
+- **Baseline-based preservation checks.** Source snapshots and artifact hashes
+  make the review baseline reproducible and detect stale or altered evidence.
+  Verification checks protected content, source membership, structure and
+  analyzer identity after external edits. It keeps `meaning_verified: false`;
+  semantic correctness remains a review judgment.
+- **Local analysis and distinct scoring paths.** Default analysis runs locally
+  without a model call. Prose scoring reports candidate density; explicit
+  `review score --agent` optionally invokes an installed Codex or Claude CLI
+  for readability ratings. Candidate density, model judgments and development
+  benchmark accuracy remain separate measures.
+
+### Concept
+
+The goal is Japanese technical prose that readers can understand, use, and
+verify, with claims bounded by evidence and technical meaning preserved.
+Clarity includes a paragraph's purpose, concrete operations and referents,
+actors and objects, conditions and consequences, logical connections, and its
+relationship to headings, figures, tables, and code. "Who does what" is one
+part of that goal. Clear prose also retains precise terminology, necessary
+negation, uncertainty, and implementation constraints.
+
+The toolkit supports that goal through shared writing criteria, contextual
+review prompts, source-aligned candidate detection, baseline verification,
+and distinct scoring and development evaluation paths. Mechanical rules locate
+passages to inspect; the reviewer judges their meaning from context and source
+evidence. The writing and review prompts guide a human or external agent in
+planning and revising complete paragraphs. They do not automatically reconstruct
+subjects and objects or rewrite manuscripts. Model-based readability evaluation
+is available only through explicit `review score --agent`.
+
+The table identifies both implemented mechanisms and guidance carried by the
+prompts. A criterion in a prompt is a review instruction, not a guarantee that
+the code can detect or verify it.
+
+| Supported principle or behavior | How it is realized | Implementation or criteria |
+| --- | --- | --- |
+| Write for a technical purpose and the reader's decisions | Shared criteria ask what a paragraph explains and what an engineer can decide or verify; writing prompts include those criteria | [Paragraph purpose](../data/writing/ja/criteria.md#start-with-the-paragraphs-technical-purpose), [Writing](../lib/asciidoc_pubkit/writing.rb) |
+| Make actors, objects, actions, conditions, and results recoverable | Review instructions ask the reviewer to resolve omitted elements from context, while allowing unambiguous omission | [Paragraph revision](../data/writing/ja/criteria.md#revise-the-complete-paragraph), [Review prompt](../lib/asciidoc_pubkit/session.rb) |
+| Replace vague abstraction, weak predicates, and metaphors with concrete explanations | Rules locate limited terms and inflected predicates; the criteria require identifying the actual referent, operation, policy, or effect before revising | [Concrete referents](../data/writing/ja/criteria.md#replace-abstraction-with-the-thing-being-discussed), [Metaphor criteria](../data/writing/ja/criteria.md#explain-metaphorical-operations-from-evidence), [Rules](../lib/asciidoc_pubkit/rules.rb) |
+| Distinguish reader instructions, actual behavior, and available capabilities | The shared criteria preserve the difference between requests, automatic execution, and optional operations | [Instructions, behavior, and capabilities](../data/writing/ja/criteria.md#distinguish-instructions-behavior-and-capabilities) |
+| Connect claims into coherent paragraphs | Prompts include all selected paragraphs and neighboring context; criteria require explicit referents, comparisons, causal relationships, and complete paragraph revision | [Relationships](../data/writing/ja/criteria.md#make-relationships-explicit), [Review prompt](../lib/asciidoc_pubkit/session.rb) |
+| Keep claims within their evidence and preserve meaning | Review guidance asks for implementation, test, or primary-source evidence and checks omissions and unsupported additions; unresolved facts remain unresolved | [Evidence and scope](../data/writing/ja/criteria.md#keep-claims-within-their-evidence-and-purpose), [Paragraph revision](../data/writing/ja/criteria.md#revise-the-complete-paragraph) |
+| Preserve technical distinctions, terminology, and justified prose choices | Criteria protect identifiers, values, conditions, negation, and necessary repetition; glossary rules, allow lists, and contextual questions support project terminology | [Terminology](../data/writing/ja/criteria.md#preserve-exact-technical-terminology), [Packaged rules](../data/review-rules.ja.yml), [Settings](../lib/asciidoc_pubkit/settings.rb) |
+| Review generated-prose patterns without mechanical deletion | Framing, contextual phrases, style, and repeated-ending checks produce candidates; shared criteria reject fixed sentence-length targets and needless synonym changes | [Generated-prose patterns](../data/writing/ja/criteria.md#remove-generated-prose-patterns-without-flattening-the-meaning), [Rules](../lib/asciidoc_pubkit/rules.rb) |
+| Keep headings, illustrations, tables, and code consistent with the explanation | Writing criteria address scope, comparisons, identifiers, and conceptual simplifications; review prompts constrain edits to the selected scope and protect other elements | [Headings, figures, and code](../data/writing/ja/criteria.md#headings-and-the-relation-to-figures-and-code), [Review scope](../lib/asciidoc_pubkit/session.rb) |
+| Make detection traceable and coverage explicit | Asciidoctor source mapping, offset-preserving inline masking, and MeCab/IPADIC produce source-aligned evidence; ambiguous or excluded passages have coverage notices | [Document](../lib/asciidoc_pubkit/document.rb), [Morphology](../lib/asciidoc_pubkit/morphology.rb), [Rules](../lib/asciidoc_pubkit/rules.rb) |
+| Preserve a reproducible and safe review baseline | Sessions save source snapshots, resolved rules, criteria, and analyzer identity; integrity checks and verification protect content outside editable prose | [Session](../lib/asciidoc_pubkit/session.rb), [Rule validation](../lib/asciidoc_pubkit/rule_set.rb) |
+| Keep manuscript indicators, readability judgments, and detector accuracy distinct | Default scoring measures candidate density; optional local CLI evaluation judges readability; annotated development benchmarks measure detection and evaluate preservation separately | [Score](../lib/asciidoc_pubkit/score.rb), [Local evaluator](../lib/asciidoc_pubkit/local_evaluator.rb), [Development benchmark](../benchmark/prose.rb) |
+| Make automation explicit and its limits visible | Default analysis runs locally with no model call; only explicit scoring requests invoke an external CLI. Manuscripts are not edited, and verification retains `meaning_verified: false` | [CLI](../lib/asciidoc_pubkit/cli.rb), [Local evaluator](../lib/asciidoc_pubkit/local_evaluator.rb), [Session](../lib/asciidoc_pubkit/session.rb) |
+
+This section explains the implementation independently of command syntax.
+[Review workflow](review-workflow.md#review-workflow) covers commands and session handling;
+[Score an AsciiDoc manuscript](scoring.md#score-an-asciidoc-manuscript) covers scoring options.
+The diagrams use GitHub-supported Mermaid fenced blocks.
+
+### Shared components and prose selection
+
+Both paths resolve the same settings and parse the AsciiDoc entry file with
+Asciidoctor, including local includes. `Document` selects running-prose paragraphs
+and maps them back to their source files. Ambiguous mappings become coverage
+notices; the analyzer does not guess locations. Headings provide context, while
+code, tables, quotations, lists, and protected blocks are excluded from prose
+analysis. Inline code, quoted spans, links, and other protected constructs are
+masked with spaces that preserve character offsets and newlines.
+
+```mermaid
+flowchart TD
+    A["AsciiDoc entry and local includes"] --> D["Document: parse and map running prose"]
+    C["Settings: options, configuration, resolved rules"] --> D
+    D --> P["Paragraphs with source locations and headings"]
+    D --> N["Coverage notices and parser diagnostics"]
+    P --> M["Rules: mask protected inline content"]
+    C --> M
+    M --> F["Candidate detection"]
+    F --> R["Session: review baseline and evidence"]
+    F --> S["Score: manuscript indicators"]
+    P -->|"Original prose and headings"| S
+    N --> R
+    N --> S
+```
+
+`Settings` resolves rule precedence as command-line `--rules`, configuration
+`review.rules`, then packaged defaults. A custom rule file replaces the whole
+set. `Rules` returns review candidates with one-based Unicode line and column
+positions; a match does not establish a defect. `Session` saves evidence for a
+later review, while `Score` returns a report directly without creating a session.
+Parser diagnostics must be resolved before scanning or scoring.
+
+### Review: collect, judge, and verify
+
+Scanning freezes the source baseline, resolved settings and rules, analyzer
+identity, and writing criteria. The session consists of `manifest.json`,
+`document.json`, `findings.json`, and `baseline/`. Prompt generation checks
+artifact integrity and rejects sources changed since the scan. It emits one
+Markdown file containing the saved criteria, context, every selected paragraph
+in document order, and its candidate evidence, including paragraphs with no
+candidates. Generating that prompt requires no analyzer or model invocation.
+
+```mermaid
+sequenceDiagram
+    actor Reviewer as Human or external agent
+    participant CLI as pubkit CLI
+    participant Sources as Manuscript files
+    participant Session as Saved review session
+    Reviewer->>CLI: review scan FILE
+    CLI->>Sources: Parse prose and detect candidates
+    CLI->>Session: Save evidence and source baseline
+    Reviewer->>CLI: review prompt SESSION
+    CLI->>Session: Check integrity and read saved evidence
+    CLI->>Sources: Check sources still match scan
+    CLI-->>Reviewer: One Markdown review prompt
+    Reviewer->>Sources: Judge context and edit prose externally
+    Reviewer->>CLI: review verify SESSION
+    CLI->>Session: Read original baseline and settings
+    CLI->>Sources: Reparse and compare protected content
+    CLI-->>Reviewer: Mechanical result, notices, remaining candidates
+```
+
+The reviewer decides whether to keep or revise each paragraph and checks source
+evidence before adding claims. `scan`, `prompt`, and `verify` never invoke an AI
+CLI or edit manuscripts. Verification compares the included source set, content
+outside reviewed prose, protected inline tokens, document structure, and analyzer
+identity against the saved baseline. Numeric changes produce manual-review
+notices; remaining candidates alone do not fail verification. A passing result
+means these mechanical checks passed, with `meaning_verified: false`.
+
+### Detection: contextual rules and token boundaries
+
+MeCab with UTF-8 IPADIC is the default analyzer. It supplies surfaces, dictionary
+forms, part of speech, and source-aligned token spans. Explicit literal mode
+matches configured surface strings; there is no silent fallback. Both modes
+apply glossary, configured prose style, and sentence-ending checks.
+
+```mermaid
+flowchart TD
+    P["Masked running prose"] --> B{"Selected tokenizer"}
+    B -->|"mecab, default"| T["MeCab and IPADIC tokens with source offsets"]
+    T --> L["Lemma rules with allow lists and contextual overlap checks"]
+    T --> X["Four enabled contextual predicate patterns"]
+    B -->|"literal, explicit"| E["Exact configured terms"]
+    P --> Q["Exact framing and contextual phrases"]
+    L --> F["Candidate findings"]
+    X --> O["Prefer most specific phrase; apply allow lists"]
+    E --> O
+    Q --> O
+    O --> F
+    P --> G["Glossary, style, and repeated endings"]
+    G --> F
+```
+
+Morphological rules match configured noun and adjective lemmas, canonical verb
+forms, adjacent compound nouns, and sahen nouns followed by `する` or `できる`.
+Predicate spans extend through adjacent auxiliaries, dependent verbs, and
+supported connective particles. Negative auxiliaries preserve polarity;
+negative-only rules require that evidence. Unknown tokens are not guessed.
+
+The four contextual patterns combine an exact prefix (`地味に`, `静かに`,
+`時間を`, or `側に`) with a known independent verb lemma (`効く`, `壊れる`,
+`溶かす`, or `倒す`). The canonical phrase must be enabled in the resolved rules.
+For example, `地味に効かなかった` is collected as one complete candidate with
+lemma `地味に効く` and `negative: true`. Protected text and paragraph boundaries
+cannot bridge the pattern. Longer contextual phrases suppress contained matches,
+including when the longer phrase is allowed. These limited patterns suggest
+reviewing the actual effect or operation; they do not prove a metaphor is wrong.
+
+Sentence-ending analysis splits each paragraph at `。`, `！`, and `？`, retaining
+source offsets. It extracts supported endings such as `検証します` and `です`,
+then checks consecutive windows of three sentences. A sentence without a matching
+ending breaks the run. The first matching run produces one informational
+candidate at the ending of its third sentence. Precise technical repetition may
+be worth retaining; this is not a requirement to vary verbs.
+
+### Scoring: density and optional readability judgment
+
+Scoring first runs the shared candidate detector. For the default
+`candidate-density-v1`, let `N` be all findings, including informational ones,
+and `C` be non-whitespace Unicode characters after inline masking:
+
+```text
+density = N * 1000 / C
+score   = max(0, 100 - 5 * density)
+```
+
+The score is rounded to three decimals. No unmasked prose yields `null`.
+More candidates per 1,000 characters lower the score; short texts are particularly
+sensitive. Changing rules, allow lists, or the detector can change the score of
+an unchanged manuscript. This heuristic is a review indicator, not a calibrated
+quality measure or a probability that AI wrote the text.
+
+```mermaid
+flowchart TD
+    P["Selected prose and detected candidates"] --> D["Compute candidate density and mechanical score"]
+    D --> A{"Explicit agent option and nonempty prose?"}
+    A -->|"No"| R["Text or JSON report"]
+    A -->|"Yes"| I["Paragraph IDs, text, headings, and readability rubric"]
+    I --> J["Installed Codex or Claude CLI; configured model provider"]
+    J --> V["Validate IDs, integer ratings, and reasons"]
+    V --> M["Map ratings to 0, 50, 100 and average"]
+    M --> R
+    D --> K["Retain mechanical score in JSON"]
+    K --> R
+```
+
+With explicit `--agent`, `llm-readability-v1` asks a fresh local CLI invocation to
+rate each paragraph's sentence clarity and paragraph coherence separately from
+1 to 3. The rubric targets engineers familiar with the technical terms. The
+input contains paragraph IDs, original text, and headings, without candidate
+findings or file paths. The CLI may connect to its configured model provider.
+The response must contain every supplied ID exactly once, integer ratings in
+range, and nonempty reasons. Invalid responses fail scoring.
+
+Ratings of 1, 2, and 3 map to 0, 50, and 100. Each dimension averages equally
+across scored paragraphs; the displayed score averages the two dimension means.
+JSON retains both dimensions, paragraph reasons, evaluator identity, and the
+mechanical score. These ordinal averages summarize a fallible judgment. There
+is no reference manuscript or fact checklist in this single-document path, so
+it does not assess preservation or establish technical accuracy. Temporary CLI
+artifacts are removed after the invocation, and manuscripts remain unchanged.
+
+| Result | What it measures | Evidence needed |
+| --- | --- | --- |
+| Candidate-density score | Detected cues per text length | Current prose and resolved rules |
+| Optional readability score | Model judgment of clarity and coherence | Current prose, context, and rubric |
+| Development benchmark accuracy | Detector agreement and location accuracy | Fixed corpus with annotated targets |
+| Review verification | Mechanical preservation after editing | Original session baseline and current files |
+
+See the [development prose benchmark](development.md#development-prose-benchmark) for
+before/after trials with gold targets, separate fact checklists, and evaluator
+calibration. Its precision, recall, and F1 evaluate the detector rather than
+providing a score for an arbitrary manuscript.
+
+[Back to README](../README.md)

data/docs/configuration.md

@@ -0,0 +1,48 @@
+# Configuration
+
+`scan` and `score` search upward from the entrypoint directory for the nearest
+`.asciidoc-pubkit.yml`. Use `--config FILE` to select a different file. CLI options
+override configuration values; unspecified values use built-in defaults.
+The CLI loads one configuration file, not merged book/repository files.
+
+```yaml
+review:
+  language: ja
+  style: desu-masu
+  tokenizer: mecab
+  # heading_rules: heading-rules.yml  # Separate heading rule set
+  # Optional overrides (dictionary paths are relative to this file):
+  # mecab_command: /opt/homebrew/bin/mecab
+  # mecab_dictionary: /opt/homebrew/lib/mecab/dic/ipadic
+  base_dir: .
+  glossary: glossary.yml
+  exclude:
+    - generated/**
+  allows: []
+  attributes:
+    edition: print
+```
+
+Configuration paths are relative to the configuration file. Without configuration,
+the base directory is the entrypoint's directory. Exclusion patterns match source
+paths relative to the base directory; excluded files can still supply attributes
+and are preserved by verification. The default style is `preserve`; explicit
+alternatives are `desu-masu` and `dearu`. Only `ja` is currently supported.
+
+A glossary maps canonical terms to variant strings:
+
+```yaml
+JavaScript:
+  - Javascript
+  - Java Script
+```
+
+Variants are review candidates, not automatic replacement instructions. In MeCab
+mode, `allows` can suppress a canonical dictionary form (all its inflections) or
+an exact matched surface (that occurrence's form only). In literal mode it
+suppresses exact dictionary entries. It does not disable glossary checks.
+Regex patterns are not interpreted in glossary or allow-list entries. Unknown
+configuration keys are rejected. A bare `mecab_command` is resolved through PATH;
+use an absolute path for an explicit executable override.
+
+[Back to README](../README.md)

data/docs/custom-rules.md

@@ -0,0 +1,69 @@
+# Customize review rules
+
+The UTF-8 YAML file [`data/review-rules.ja.yml`](../data/review-rules.ja.yml) is
+included in the gem and loaded by default. Edit that file when running a local
+checkout, or copy it to a project-owned file for custom rules. No Ruby changes
+are required to update candidate terms.
+
+```sh
+asciidoc-pubkit review scan book.adoc --rules ./review-rules.yml
+make run ARGS="review scan book.adoc --rules ./review-rules.yml"
+```
+
+For an installed gem, copy the default file with:
+
+```sh
+ruby -rasciidoc_pubkit -e 'puts File.read(AsciidocPubkit::RuleSet::DEFAULT_PATH)' > review-rules.yml
+```
+
+Alternatively, configure a path in `.asciidoc-pubkit.yml`:
+
+```yaml
+review:
+  rules: review-rules.yml
+```
+
+Precedence is `--rules`, then `review.rules`, then the packaged default.
+CLI paths are relative to the current working directory; configuration paths
+are relative to the configuration file. A custom file replaces the entire rule
+set; it is not merged with defaults. Start by copying the standard file.
+
+### Rule file format (schema version 1)
+
+All top-level fields below are required. Unknown keys and invalid types are
+rejected. YAML aliases and object tags are not supported.
+
+| Field | Format and behavior |
+| --- | --- |
+| `schema_version` | Integer `1` |
+| `terms` | Mapping containing all five categories listed below |
+| `verbs` | Mapping from canonical verb forms to nonempty arrays of MeCab/IPADIC dictionary forms; each dictionary form belongs to only one canonical form |
+| `sahen` | Array of nouns matched with a following `する` or potential `できる` verb |
+| `negative_only` | Array of canonical predicates restricted to negative forms; use the noun plus `する` for sahen predicates |
+| `compound_nouns` | Array of terms matched across contiguous noun tokens; each must also appear in `abstract-reference.terms` |
+
+Each `terms` category must contain `terms` (an array of unique nonempty strings)
+and `question` (a nonempty review instruction string). The required categories
+are `abstract-reference`, `weak-predicate`, `vague-degree`, `contextual-phrase`,
+and `generic-framing`. Empty term arrays disable that category's literal
+candidates. Empty `verbs`, `sahen`, and `compound_nouns` collections disable their
+respective morphological matchers. Keep `negative_only` consistent with the
+configured predicates.
+
+In literal mode, all category term lists use exact phrase matching. In MeCab
+mode, abstract nouns and degree adjectives use dictionary forms, while predicates
+use `verbs` and `sahen`; add a predicate's desired literal surface to
+`weak-predicate.terms` as well if literal mode should detect it. Contextual and
+generic framing phrases use literal matching in both modes, with the
+MeCab exception for the four configured contextual predicate patterns described
+in [rules and coverage](rules-and-coverage.md). Questions apply to
+both detectors. Glossary, style, repeated-ending checks, inline exclusions, and
+morphological suffix handling remain implemented in Ruby.
+
+A scan saves the resolved rule contents and source path in `manifest.json`.
+Prompt generation and verification use the saved contents, even if the original
+YAML file is subsequently edited or removed. Start a new session to apply rule
+changes. Sessions require the same tool version that created them; preserve old
+baselines and start a separate new session when upgrading.
+
+[Back to README](../README.md)

data/docs/development.md

@@ -0,0 +1,138 @@
+# Development
+
+```sh
+bundle install
+bundle exec rake test
+bundle exec ruby -Ilib exe/asciidoc-pubkit --help
+gem build asciidoc-pubkit.gemspec
+```
+
+The tests exercise include-boundary mapping, inline masking, configuration,
+contextual prompts, stale inputs, protected-content verification, and real MeCab
+analysis of inflections, negative predicates, Unicode positions, and long lines.
+Install MeCab and UTF-8 IPADIC before running the complete test suite.
+GitHub Actions is configured for Ruby 3.2, 3.3, 3.4, and 4.0 on Linux.
+
+[`test/fixtures/prose_evaluation.ja.json`](../test/fixtures/prose_evaluation.ja.json)
+contains original Japanese technical prose for candidate regression and manual
+revision evaluation. Automated tests check metaphor candidates in both
+tokenizers, source positions, inline exclusions, and allow lists. The revision
+pairs are manual evaluation cases, not an automated semantic checker or a
+measured AI benchmark. Compare source and revision separately for readability,
+information loss, and unsupported additions, using each case's review axes and
+expected disposition. Cover conditions, negation, numbers, versions, actors,
+causes, implementation requirements, and necessary repetition. Record the
+model, prompt, source evidence, and human judgments when evaluating actual
+generated revisions; fewer findings alone do not establish an improvement.
+
+### Development prose benchmark
+
+The checkout provides `script/evaluate-prose` for measuring future detector and
+prompt changes. This is a development tool, not an installed gem command; the
+benchmark CLI calls are separate from the optional `review score --agent` path.
+The fixed
+[`prose_benchmark.ja.adoc`](../test/fixtures/prose_benchmark.ja.adoc) contains 13
+original editorial paragraphs with intentionally vague framing and metaphors,
+inflection variants, and valid technical prose controls. Its
+[`annotations`](../test/fixtures/prose_benchmark.ja.json) contain 11 review targets
+and separate source fact checklists. It is a small synthetic regression corpus,
+not an AI authorship dataset or a representative measure of prose quality.
+
+Run the deterministic benchmark before and after a change:
+
+```sh
+bundle exec ruby script/evaluate-prose --output .pubkit/evaluation/before
+# After changing the detector or prompt:
+bundle exec ruby script/evaluate-prose --output .pubkit/evaluation/after \
+  --compare .pubkit/evaluation/before/report.json
+```
+
+The report measures candidate precision, recall, F1, and location accuracy on a
+0–100 scale for `generic-framing`, metaphorical operations, and `repeated-ending`.
+Other rule findings are counted separately and are not judged against these
+annotations. Detection matches a target's paragraph and kind; location accuracy
+then checks the one-based source position, represented internally as a zero-based
+Unicode character offset within that paragraph. The annotated repeated-ending
+location is the matching ending of the third sentence in the consecutive run.
+A candidate can be correctly detected even when the reviewer should keep it.
+Zero denominators are reported as `null`, not perfect scores. Duplicate detections
+count as false positives. These scores measure the detector, not the manuscript.
+
+The checked-in [`MeCab baseline`](../benchmark/baseline-mecab.json), captured before
+the detector changes, has precision 100, recall 81.818, F1 90, and location
+accuracy 77.778. It misses two inflected metaphor phrases and mislocates two
+repeated-ending candidates. Reports record source, criteria, rules, corpus, and
+dictionary fingerprints. Comparisons reject changed corpora, scoring versions,
+analyzers, or dictionaries. Reports with configuration fingerprints also reject
+changed benchmark configuration. A fixed empty configuration
+keeps these trials independent of auto-discovered manuscript settings.
+The 0.6.2 location correction raises location accuracy to 100 without
+changing precision, recall, or F1 on this corpus. Adding the four MeCab predicate
+patterns then raises precision, recall, F1, and location accuracy to 100 on these
+11 targets. This is regression evidence for the fixed synthetic corpus, not
+evidence of general performance on unseen manuscripts; literal recall remains
+81.818.
+Record your own baseline when dictionary fingerprints
+differ. `--tokenizer literal` explicitly selects a separate limited-mode run;
+there is no automatic fallback, and literal results do not validate MeCab.
+
+### Optional local AI evaluation
+
+Use an installed, authenticated Codex or Claude CLI to perform rewrite and judge
+trials explicitly. A local CLI may call its configured remote model provider and
+consume account usage; this is not an offline model test. Normal `rake test`
+does not run either CLI. For example:
+
+```sh
+bundle exec ruby script/evaluate-prose --agent codex --repeats 3 \
+  --output .pubkit/evaluation/ai-before
+bundle exec ruby script/evaluate-prose --agent codex --repeats 3 \
+  --output .pubkit/evaluation/ai-after \
+  --compare .pubkit/evaluation/ai-before/report.json
+# Or use Claude:
+bundle exec ruby script/evaluate-prose --agent claude \
+  --output .pubkit/evaluation/claude-before
+```
+
+Each trial generates the actual saved review prompt, requests paragraph revisions
+as JSON, and runs the existing mechanical verifier. A fresh judge invocation
+evaluates both original and revised prose with neutral A/B labels, alternating
+their order across trials. The judge sees source evidence but no findings, tool
+revision labels, or expected calibration answers. The writer receives the source
+fact checklist in addition to the review prompt, so this measures an evidence-
+assisted workflow rather than every real-world manuscript review.
+
+Sentence clarity and paragraph coherence are separately rated 1–3 and displayed
+as 0, 50, or 100 before averaging. These ordinal averages are descriptive
+summaries, not validated interval measurements. Fact retention measures the
+percentage of annotated source facts preserved; addition-free and substitution-
+free paragraph percentages remain separate. Known acceptable, incorrect, and
+unsupported revisions from `prose_evaluation.ja.json` calibrate the judge using
+accuracy and balanced accuracy across the three dispositions. Inspect poor
+calibration before interpreting quality scores. No combined AI-likeness score or
+semantic pass is produced: `meaning_verified` remains `false`.
+Mechanical verification results and their pass rate are also reported separately.
+Quality scores from a trial with preservation violations remain visible for
+diagnosis; a completed evaluation does not authorize accepting that revision.
+
+One trial is a smoke test; use at least three to inspect variation. Reports retain
+individual scores, means, population standard deviations, and ranges. They do
+not perform a significance test. Same-model judging can favor the writer's style;
+`--judge-agent` and `--judge-model` select a separate evaluator. `--model` selects
+the writer model; omitted model options use each CLI's default. AI comparisons
+require the same reported writer and judge models, CLI versions, rubric, and
+calibration corpus. CLI defaults can change, so retain the recorded model IDs
+and use explicit model options for controlled comparisons. Score changes can
+reflect sampling variation as well as implementation changes.
+
+The runner uses argument arrays rather than shell interpolation, read-only Codex
+execution or tool-disabled Claude execution, and a per-invocation timeout
+(`--timeout`, default 300 seconds). The CLI may still load its user configuration;
+use consistent settings for comparisons. Current adapters require the flags
+shown by `codex exec --help` and `claude --help`; older CLI versions may need an
+update. Missing CLIs, authentication errors, timeouts, and malformed responses
+fail explicitly. Existing output directories are never replaced. Partial
+artifacts remain available after failures. Results, prompts, responses, and review
+sessions belong under ignored `.pubkit/`; do not commit private evaluation runs.
+
+[Back to README](../README.md)

data/docs/document-commands.md

@@ -0,0 +1,68 @@
+# Document commands
+
+Version 1.0.0 provides four read-only document commands. They require neither
+MeCab nor an AI CLI, accept documents in any language, and do not create review
+sessions or change manuscripts.
+
+```sh
+asciidoc-pubkit document toc book.adoc --depth 2 --numbered
+asciidoc-pubkit document info book.adoc --json
+asciidoc-pubkit document check-xrefs book.adoc --json
+asciidoc-pubkit document index . --output index.adoc --title "Index"
+```
+
+All four require exactly one input. Use `--help` for command-specific options.
+Output defaults to stdout. `--output FILE` creates a new file and rejects existing
+files and symlinks. Diagnostics go to stderr. These commands do not discover
+`.asciidoc-pubkit.yml` or load review rules. Use repeatable `-a NAME=VALUE` options
+for Asciidoctor attributes and `--base-dir DIR` for the local include boundary.
+The default boundary is the input file's directory, or the scanned directory for
+`index`. Local includes and conditional directives are parsed normally; source
+files and symlink targets must remain within that boundary. Remote includes are
+rejected. `/shared/` has no special mapping or automatic exclusion: migrate such
+references to local includes within the configured boundary.
+
+`toc` prints parsed section titles in document order, with two spaces per nesting
+level. It includes book parts and their child chapters, but excludes the document
+title and apparent headings inside code blocks. `--numbered` uses outline
+positions (`1.`, `1-1.`, etc.), independently of Asciidoctor's publication
+numbering. `--depth` must be a positive integer and limits the Asciidoctor section
+level: chapters are level 1 and book parts are level 0. It does not generate or
+rewrite heading text.
+
+`info` prints `key: value` lines or a JSON object with `--json`. Its fields are
+`doctitle`, `subtitle`, `description`, `keywords`, `lang`, `uuid`, `author`,
+`producer`, and `creator`. Missing attributes become empty strings. Metadata is
+extracted from the parsed document, including active local includes.
+
+`check-xrefs` reports unresolved local `xref:ID[]` and `<<ID,label>>` references,
+including fragments targeting the entry document or an included file. It
+recognizes explicit and generated IDs and attribute-expanded targets. It scans
+macro-enabled block text, list items, table cells, and titles; comments and
+ordinary code blocks are excluded. References to separate documents and remote
+URLs are outside its scope. This is a limited syntax scan, not complete rendered
+link validation: complex inline passthroughs and custom macros are not analyzed.
+Text output has `file:line`, a tab, and the missing ID; JSON output is an array of
+`refid`, `file`, and `line` objects. Locations use Asciidoctor source cursors;
+table cells, list continuations, and multiline titles may identify the enclosing node's start
+rather than the exact token line. Duplicate occurrences of one ID on the same
+source line are reported once.
+
+`index` recursively scans lowercase `.adoc` files, skips hidden subdirectories
+and its output path, and uses each document's ID and title. It produces an
+AsciiDoc alphabetical cross-reference list for English names: parenthesized
+text and subtitles after `:` or `：` are removed; remaining titles must be ASCII
+and contain a letter. Documents without IDs are omitted. Titles are deduplicated
+case-insensitively, preferring the shallowest relative path and then lexical path
+order. `--title` adds a document title with `[#book-index]`; alphabetical groups
+use discrete level-2 headings (`=== A`). This indexes document titles, not all
+section headings. The generated references are useful when those documents are
+included in the consuming book; the command does not verify that membership.
+
+Success returns 0. `check-xrefs` returns 1 when local references are unresolved.
+Invalid inputs/options, include or parse errors, and output failures return 2;
+no output file is created on a parse error. Warnings remain visible on stderr.
+Unlike the original directory index script, an input parse error fails the whole
+index instead of silently omitting the file.
+
+[Back to README](../README.md)