asciidoc-pubkit 1.0.0 → 1.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 784fdf94af7ffc0fc2f4b45fad86b80f57e9d0bfd9049f8fdd90be9c7025fc10
-  data.tar.gz: b5671ae513cb590f05a3bf340d448570c5a92bba66fb9f83f8b296c83537edc7
+  metadata.gz: 11ce21a9f9688e6a959a5a4ffbb4479876739d6f901d0b50f383b1742d1bc620
+  data.tar.gz: 407e838b2f9034dc7c1d085331c23a8fd91eeaa87b6f148dc89b8c732a9ff8af
 SHA512:
-  metadata.gz: 1be2e9f5a2946bbcb9a8c0f94912d26dba36bbc9801bd539d90b4c6c2453b56b1726ffd27792d7bf9972fbee71605d345d47dab84a4828d323e57bb0f68a7ae5
-  data.tar.gz: 238d13ce20da703fc2a83a0f66b213349e7db442af9b05723ec1cc88b7d42fa6632173bf184181a65294171b3845104857c4d954a91dbfb75cadd2db6495bb2d
+  metadata.gz: 69ce04953e4caae4f349bfa9874783588d34b7927fb773f5cd8bf9b1d09980e8ff4bfb66baeb8b02f5e7f2e7d720317d29b21678861ed2ebc2619e069ee03d72
+  data.tar.gz: 3468c4e3b094003f25892d1b5197327df659fe4ea08180e45b93ebdaeedec6ce1a07e832be80526cb0d949219c71f5ccbbf7082c83e765197adf8dae843c45e7

data/CHANGELOG.md

@@ -1,5 +1,25 @@
 # Changelog
 
+## 1.0.1 — 2026-10-05
+
+- Explain the project's publishing background and link the book catalogs without
+  adding publishing functionality.
+
+- Add a dedicated prose evaluation guide covering fixtures, normal test coverage,
+  detector and AI metrics, controlled comparisons, and measured-result limits.
+  Link it from the README and retain setup and test instructions in Development.
+
+- Extend the original development corpus from 13 to 21 paragraphs and the manual
+  revision calibration from 8 to 20 pairs, covering quantity and modifier targets,
+  order, condition scope, parallel work, sentence functions, and style constraints.
+  Identify the expanded corpus as `technical-prose-v2`; preserve the historical
+  v1 MeCab baseline and require a new baseline for comparisons with v2.
+- Adapt yomiyasu v1.0.6 structural comparison and sentence-function guidance into
+  the shared Japanese criteria. Preserve plain or polite style, condition and
+  purpose scope during sentence edits, and meaningful punctuation. Extend the
+  development judge rubric without changing detector rules, scoring formulas,
+  mechanical verification semantics, or existing saved session criteria.
+
 ## 1.0.0 — 2026-10-05
 
 - Release the existing document inspection, explicit prose replacement,

data/README.md

@@ -6,7 +6,17 @@
 
 A toolkit for authoring and reviewing AsciiDoc books.
 
-Version 1.0.0 provides document inspection, explicit prose-only mechanical
+This gem brings together practical lessons from the author's experience writing
+and publishing books, along with a selection of the tools originally developed
+for that work. Those projects include CyberGarage technical books in
+[English](https://www.amazon.com/stores/CyberGarage/author/B0H8M3CPGH) and
+[Japanese](https://www.amazon.co.jp/stores/CyberGarage/author/B0H8M3CPGH),
+and books on health, exercise, and sports science published under
+[WellBeing Lab](https://www.amazon.co.jp/stores/WellBeing-Lab/author/B0H7MB1HVD).
+The experience informs the toolkit's writing criteria, contextual review prompts,
+and checks for preserving manuscript content during revision.
+
+Version 1.0.1 provides document inspection, explicit prose-only mechanical
 replacements, and Japanese manuscript review. Inspect a book's outline,
 metadata, and local cross-references; preview and apply your replacement rules;
 then review and verify edits against a saved baseline.
@@ -16,7 +26,7 @@ Japanese manuscript excerpts and rule terms retain their original text.
 
 ## Available features
 
-The following commands are available in version 1.0.0.
+The following commands are available in version 1.0.1.
 
 | Feature | Commands | Behavior |
 | --- | --- | --- |
@@ -34,6 +44,11 @@ The following commands are available in version 1.0.0.
 | [Manuscript scoring](docs/scoring.md#score-an-asciidoc-manuscript) | `review score` | Report a prose candidate-density score; explicit `--agent codex` or `--agent claude` optionally invokes a local AI CLI for readability ratings |
 | [Writing criteria and prompts](docs/writing.md#writing-commands) | `writing criteria`, `writing prompt` | Print Japanese prose criteria or generate a writing prompt without a manuscript or MeCab |
 
+Version 1.0.1 extends the [shared writing criteria](docs/writing.md#writing-commands)
+with structural comparison before and after revision and preservation of sentence
+functions and plain or polite style. The [development evaluation](docs/prose-evaluation.md)
+includes new cases for these checks.
+
 ## Requirements and scope
 
 Ruby 3.2 or later is required; RubyGems installs Asciidoctor and the other Ruby
@@ -63,7 +78,7 @@ asciidoc-pubkit --help
 ```
 
 To update, run `gem update asciidoc-pubkit`. For Bundler, add
-`gem 'asciidoc-pubkit', '~> 1.0.0'` to your project's `Gemfile` and run commands
+`gem 'asciidoc-pubkit', '~> 1.0.1'` to your project's `Gemfile` and run commands
 through `bundle exec`. See [source installation and local checkout usage](docs/installation.md).
 
 ## Quick start
@@ -103,7 +118,7 @@ asciidoc-pubkit writing prompt --lang ja --output writing-prompt.md
 
 Existing review sessions require the exact tool version that created them.
 Before upgrading, finish ongoing reviews with that version or preserve their
-baselines and start a new review in a separate directory with v1.0.0. A new
+baselines and start a new review in a separate directory with v1.0.1. A new
 scan cannot verify preservation relative to an earlier baseline.
 
 ## Documentation
@@ -121,7 +136,8 @@ scan cannot verify preservation relative to an earlier baseline.
 | [Rules and coverage](docs/rules-and-coverage.md) | Candidate detection, source positions, scope, session compatibility |
 | [Custom review rules](docs/custom-rules.md) | Complete rule replacement, strict YAML schema, saved rules |
 | [Architecture](docs/architecture.md) | Review principles, analysis pipeline, scoring algorithms |
-| [Development](docs/development.md) | Tests, synthetic benchmark, explicit AI evaluation trials |
+| [Development](docs/development.md) | Setup, test suite, development validation |
+| [Prose evaluation](docs/prose-evaluation.md) | Fixed fixtures, detector metrics, AI trials, before/after comparisons |
 | [References](docs/references.md) | Editorial and research foundations |
 | [English review proposal](docs/english-review-research.md) | Future design research; English review is not implemented |
 

data/data/writing/ja/criteria.md

@@ -8,6 +8,7 @@
 - Explain metaphorical operations from evidence
 - Distinguish instructions, behavior, and capabilities
 - Make relationships explicit
+- Compare structure before and after revision
 - Keep claims within their evidence and purpose
 - Remove generated-prose patterns without flattening the meaning
 - Preserve exact technical terminology
@@ -155,6 +156,21 @@ instruction when that fits the document's voice. Do not change `〜します` to
 optional capability are different claims. Name the actor when context does not
 identify it, without repeating an already clear subject in every sentence.
 
+Distinguish advice, obligations, plans, and evaluations as well, including when
+different functions occur in adjacent sentences or clauses. A paragraph can
+combine advice with an explanation of actual behavior; that alone is not an
+ending inconsistency. Do not turn advice into a policy, a plan into a completed
+action, or an evaluation into an implementation goal. Sequence words such as
+`まず` and `次に` establish order, not whether the action is advice or actual
+behavior. When the function or actor is unresolved, retain the uncertainty and
+record what needs confirmation.
+
+Preserve plain or polite style unless the author requests a change or the book's
+explicit style requirements call for one. A technical topic alone does not
+require polite style. Resolve unintended style mixing separately from each
+sentence's function; changing `する` to `します` must not change advice into
+an asserted operation. Do not vary endings just to make these functions uniform.
+
 ## Make relationships explicit
 
 Inspect a sentence when it:
@@ -177,6 +193,40 @@ Use prose for a sequence in which one event causes or conditions the next; use
 a list when the items are genuinely parallel. Do not turn a causal explanation
 into disconnected bullets or impose an order on independent items.
 
+## Compare structure before and after revision
+
+Before replacing words, identify the subject and predicate, each modifier and
+its target, and each referring expression and its referent. Trace conditions,
+exceptions, negation, parallel relationships, quantities and their targets,
+and required order. After revising, compare those relationships with the source,
+including passages without candidates. Keep naturally clear wording when no
+relationship needs repair. Supply missing actors or objects only from evidence;
+leave an unresolved relationship pending rather than inventing one.
+
+Use these checks when splitting, joining, or rearranging sentences:
+
+- Keep a condition or exception attached to the same operations. Joining an
+  unconditional audit-log statement to a conditional data-save statement must
+  not make logging conditional on authentication success.
+- Preserve what each quantity counts and whether it is a total, per-item limit,
+  minimum, maximum, or exact count. `各ワーカーは最大3回再試行します` does not
+  mean `全ワーカーで合計3回まで再試行します`, although the numeral is unchanged.
+- Preserve required order and distinguish it from independent parallel work.
+  Stopping processing, changing settings, and restarting in that order is not
+  equivalent to changing settings before stopping. Do not invent an order or
+  success prerequisite for independent validation steps.
+- Keep modifiers attached to their original targets after replacing verbs or
+  changing word order. In `圧縮されたログを送信するサーバーを監視します`, the
+  logs are compressed and the server is monitored.
+- Preserve the scope and direction of purpose, reason, contrast, and emphasis.
+  A purpose applying to two operations must still apply to both after splitting;
+  necessary contrast or priority must not become simple addition or equality.
+
+Adjust punctuation when it obscures these relationships, not merely to reduce
+comma counts. Keep a colon that maps a label to a value, such as `状態: 正常`,
+and distinguish it from an ornamental ending. Preserve inline syntax and edit
+scope; prose review does not authorize rewriting labels in protected blocks.
+
 ## Keep claims within their evidence and purpose
 
 State whether a claim comes from an implementation, a test, a cited study, or
@@ -304,6 +354,9 @@ condition, cause, and effect.
 Assess readability and meaning preservation separately. Compare the source and
 revision for both missing information and unsupported additions, including
 conditions, negation, numbers, actors, causes, and implementation requirements.
+Include modifier targets, quantity targets, required order, exception scope,
+sentence functions, and requested style in that comparison. Unchanged numbers
+or words do not establish unchanged relationships.
 For example, removing a unique constraint from a duplicate-message check can
 lose a concurrency requirement even when the revised paragraph sounds clearer.
 Record unresolved evidence and keep necessary repetition or qualifications.

data/docs/architecture.md

@@ -256,7 +256,7 @@ artifacts are removed after the invocation, and manuscripts remain unchanged.
 | Development benchmark accuracy | Detector agreement and location accuracy | Fixed corpus with annotated targets |
 | Review verification | Mechanical preservation after editing | Original session baseline and current files |
 
-See the [development prose benchmark](development.md#development-prose-benchmark) for
+See the [prose evaluation guide](prose-evaluation.md) for
 before/after trials with gold targets, separate fact checklists, and evaluator
 calibration. Its precision, recall, and F1 evaluate the detector rather than
 providing a score for an arbitrary manuscript.

data/docs/development.md

@@ -13,126 +13,11 @@ analysis of inflections, negative predicates, Unicode positions, and long lines.
 Install MeCab and UTF-8 IPADIC before running the complete test suite.
 GitHub Actions is configured for Ruby 3.2, 3.3, 3.4, and 4.0 on Linux.
 
-[`test/fixtures/prose_evaluation.ja.json`](../test/fixtures/prose_evaluation.ja.json)
-contains original Japanese technical prose for candidate regression and manual
-revision evaluation. Automated tests check metaphor candidates in both
-tokenizers, source positions, inline exclusions, and allow lists. The revision
-pairs are manual evaluation cases, not an automated semantic checker or a
-measured AI benchmark. Compare source and revision separately for readability,
-information loss, and unsupported additions, using each case's review axes and
-expected disposition. Cover conditions, negation, numbers, versions, actors,
-causes, implementation requirements, and necessary repetition. Record the
-model, prompt, source evidence, and human judgments when evaluating actual
-generated revisions; fewer findings alone do not establish an improvement.
+## Prose evaluation
 
-### Development prose benchmark
-
-The checkout provides `script/evaluate-prose` for measuring future detector and
-prompt changes. This is a development tool, not an installed gem command; the
-benchmark CLI calls are separate from the optional `review score --agent` path.
-The fixed
-[`prose_benchmark.ja.adoc`](../test/fixtures/prose_benchmark.ja.adoc) contains 13
-original editorial paragraphs with intentionally vague framing and metaphors,
-inflection variants, and valid technical prose controls. Its
-[`annotations`](../test/fixtures/prose_benchmark.ja.json) contain 11 review targets
-and separate source fact checklists. It is a small synthetic regression corpus,
-not an AI authorship dataset or a representative measure of prose quality.
-
-Run the deterministic benchmark before and after a change:
-
-```sh
-bundle exec ruby script/evaluate-prose --output .pubkit/evaluation/before
-# After changing the detector or prompt:
-bundle exec ruby script/evaluate-prose --output .pubkit/evaluation/after \
-  --compare .pubkit/evaluation/before/report.json
-```
-
-The report measures candidate precision, recall, F1, and location accuracy on a
-0–100 scale for `generic-framing`, metaphorical operations, and `repeated-ending`.
-Other rule findings are counted separately and are not judged against these
-annotations. Detection matches a target's paragraph and kind; location accuracy
-then checks the one-based source position, represented internally as a zero-based
-Unicode character offset within that paragraph. The annotated repeated-ending
-location is the matching ending of the third sentence in the consecutive run.
-A candidate can be correctly detected even when the reviewer should keep it.
-Zero denominators are reported as `null`, not perfect scores. Duplicate detections
-count as false positives. These scores measure the detector, not the manuscript.
-
-The checked-in [`MeCab baseline`](../benchmark/baseline-mecab.json), captured before
-the detector changes, has precision 100, recall 81.818, F1 90, and location
-accuracy 77.778. It misses two inflected metaphor phrases and mislocates two
-repeated-ending candidates. Reports record source, criteria, rules, corpus, and
-dictionary fingerprints. Comparisons reject changed corpora, scoring versions,
-analyzers, or dictionaries. Reports with configuration fingerprints also reject
-changed benchmark configuration. A fixed empty configuration
-keeps these trials independent of auto-discovered manuscript settings.
-The 0.6.2 location correction raises location accuracy to 100 without
-changing precision, recall, or F1 on this corpus. Adding the four MeCab predicate
-patterns then raises precision, recall, F1, and location accuracy to 100 on these
-11 targets. This is regression evidence for the fixed synthetic corpus, not
-evidence of general performance on unseen manuscripts; literal recall remains
-81.818.
-Record your own baseline when dictionary fingerprints
-differ. `--tokenizer literal` explicitly selects a separate limited-mode run;
-there is no automatic fallback, and literal results do not validate MeCab.
-
-### Optional local AI evaluation
-
-Use an installed, authenticated Codex or Claude CLI to perform rewrite and judge
-trials explicitly. A local CLI may call its configured remote model provider and
-consume account usage; this is not an offline model test. Normal `rake test`
-does not run either CLI. For example:
-
-```sh
-bundle exec ruby script/evaluate-prose --agent codex --repeats 3 \
-  --output .pubkit/evaluation/ai-before
-bundle exec ruby script/evaluate-prose --agent codex --repeats 3 \
-  --output .pubkit/evaluation/ai-after \
-  --compare .pubkit/evaluation/ai-before/report.json
-# Or use Claude:
-bundle exec ruby script/evaluate-prose --agent claude \
-  --output .pubkit/evaluation/claude-before
-```
-
-Each trial generates the actual saved review prompt, requests paragraph revisions
-as JSON, and runs the existing mechanical verifier. A fresh judge invocation
-evaluates both original and revised prose with neutral A/B labels, alternating
-their order across trials. The judge sees source evidence but no findings, tool
-revision labels, or expected calibration answers. The writer receives the source
-fact checklist in addition to the review prompt, so this measures an evidence-
-assisted workflow rather than every real-world manuscript review.
-
-Sentence clarity and paragraph coherence are separately rated 1–3 and displayed
-as 0, 50, or 100 before averaging. These ordinal averages are descriptive
-summaries, not validated interval measurements. Fact retention measures the
-percentage of annotated source facts preserved; addition-free and substitution-
-free paragraph percentages remain separate. Known acceptable, incorrect, and
-unsupported revisions from `prose_evaluation.ja.json` calibrate the judge using
-accuracy and balanced accuracy across the three dispositions. Inspect poor
-calibration before interpreting quality scores. No combined AI-likeness score or
-semantic pass is produced: `meaning_verified` remains `false`.
-Mechanical verification results and their pass rate are also reported separately.
-Quality scores from a trial with preservation violations remain visible for
-diagnosis; a completed evaluation does not authorize accepting that revision.
-
-One trial is a smoke test; use at least three to inspect variation. Reports retain
-individual scores, means, population standard deviations, and ranges. They do
-not perform a significance test. Same-model judging can favor the writer's style;
-`--judge-agent` and `--judge-model` select a separate evaluator. `--model` selects
-the writer model; omitted model options use each CLI's default. AI comparisons
-require the same reported writer and judge models, CLI versions, rubric, and
-calibration corpus. CLI defaults can change, so retain the recorded model IDs
-and use explicit model options for controlled comparisons. Score changes can
-reflect sampling variation as well as implementation changes.
-
-The runner uses argument arrays rather than shell interpolation, read-only Codex
-execution or tool-disabled Claude execution, and a per-invocation timeout
-(`--timeout`, default 300 seconds). The CLI may still load its user configuration;
-use consistent settings for comparisons. Current adapters require the flags
-shown by `codex exec --help` and `claude --help`; older CLI versions may need an
-update. Missing CLIs, authentication errors, timeouts, and malformed responses
-fail explicitly. Existing output directories are never replaced. Partial
-artifacts remain available after failures. Results, prompts, responses, and review
-sessions belong under ignored `.pubkit/`; do not commit private evaluation runs.
+The checkout includes fixed Japanese prose fixtures, detector regression tests,
+and an optional rewrite-and-judge runner. See [Prose evaluation](prose-evaluation.md)
+for the corpus, metrics, test coverage, before/after procedure, and result
+interpretation. Normal `rake test` runs do not invoke an AI CLI.
 
 [Back to README](../README.md)

data/docs/document-commands.md

@@ -1,6 +1,6 @@
 # Document commands
 
-Version 1.0.0 provides four read-only document commands. They require neither
+Version 1.0.1 provides four read-only document commands. They require neither
 MeCab nor an AI CLI, accept documents in any language, and do not create review
 sessions or change manuscripts.
 

data/docs/installation.md

@@ -41,7 +41,7 @@ Add the gem to your project's `Gemfile` to manage its version with Bundler:
 
 ```ruby
 source 'https://rubygems.org'
-gem 'asciidoc-pubkit', '~> 1.0.0'
+gem 'asciidoc-pubkit', '~> 1.0.1'
 ```
 
 Then install dependencies and run the CLI through Bundler:
@@ -66,7 +66,7 @@ git clone https://github.com/cybergarage/asciidoc-pubkit.git
 cd asciidoc-pubkit
 bundle install
 gem build asciidoc-pubkit.gemspec
-gem install ./asciidoc-pubkit-1.0.0.gem
+gem install ./asciidoc-pubkit-1.0.1.gem
 asciidoc-pubkit --version
 ```
 

data/docs/prose-evaluation.md

@@ -0,0 +1,246 @@
+# Prose evaluation
+
+Use a fixed set of Japanese technical paragraphs to compare detector or review
+criteria changes against a project-defined baseline. Keep detection performance,
+readability, and meaning preservation separate so that a clearer rewrite cannot
+hide a lost condition or invented fact.
+
+This guide describes the version 1.0.1 development checkout, including the
+`technical-prose-v2` corpus and criteria update. The evaluation runner is not an
+installed gem command. It does not measure Japanese naturalness or AI authorship
+as a single validated score.
+
+## Three different evaluations
+
+| Evaluation | Input and method | What the number means |
+| --- | --- | --- |
+| `review score` without `--agent` | A manuscript; candidate count normalized by unmasked prose length | Candidate-density indicator, not correctness or naturalness |
+| `script/evaluate-prose` without `--agent` | Fixed corpus and annotated targets; deterministic analysis | Detector precision, recall, F1, and location accuracy |
+| `script/evaluate-prose --agent codex` or `--agent claude` | Fixed corpus; generated review prompt, rewrite, and separate judge invocation | Readability and checklist preservation before and after rewriting |
+
+The default manuscript score is `max(0, 100 - 5 × candidates per 1,000 characters)`.
+A higher score means fewer detected candidates at that text length. Necessary
+repetition can still be a candidate, and fewer candidates alone do not establish
+better prose. See [Manuscript scoring](scoring.md) for the separate optional
+single-manuscript readability rating; that path has no original-versus-revision
+fact checklist.
+
+For a detector change, use the deterministic benchmark. For a writing or review
+criteria change, use the optional rewrite-and-judge trials to measure its effect
+on the actual revisions. Criteria can change while deterministic detector scores
+remain identical.
+
+## Fixed test data
+
+All examples are original project-authored editorial fixtures. They intentionally
+include patterns associated with generated prose, plus valid technical controls.
+They are not a collected dataset of known AI outputs or a representative sample
+of Japanese writing.
+
+| File | Contents and role |
+| --- | --- |
+| [prose_benchmark.ja.adoc](../test/fixtures/prose_benchmark.ja.adoc) | 21 paragraphs used as the actual AsciiDoc input |
+| [prose_benchmark.ja.json](../test/fixtures/prose_benchmark.ja.json) | Matching paragraph IDs and text, 11 annotated detector targets, and preservation checklists |
+| [prose_benchmark.yml](../test/fixtures/prose_benchmark.yml) | Fixed empty configuration that isolates the benchmark from discovered manuscript settings |
+| [prose_evaluation.ja.json](../test/fixtures/prose_evaluation.ja.json) | Detection examples and 20 manual revision pairs with expected `accept`, `reject`, or `needs-evidence` dispositions |
+
+The corpus includes framing such as `重要なのは`, metaphorical operations such
+as `地味に効く`, inflected and negative forms, and repeated endings. Controls
+include literal verbs, established technical terms, necessary repetition,
+concurrency requirements, and protected inline code.
+
+Eight paragraphs added in `technical-prose-v2` cover quantity targets, required
+order, modifier targets, condition scope, parallel checks, advice versus actual
+behavior, plain style, and punctuation and purpose scope. They have no additional
+detector targets but still reach the reviewer and judge. The checklists include
+explicit editorial constraints as well as technical facts.
+
+For example, this pair must be rejected even though its numeral is unchanged:
+
+| Original | Revision | Preservation issue |
+| --- | --- | --- |
+| 各ワーカーは最大3回再試行します。 | 全ワーカーで合計3回まで再試行します。 | A per-worker limit became an aggregate limit |
+
+Other revision pairs cover reversed recovery order, authentication conditions
+incorrectly extended to audit logging, independent checks made conditional, and
+an unsolicited change from plain to polite style. Acceptable pairs and unresolved
+cases prevent calibration from treating every change as an error.
+
+## What normal tests check
+
+Run the complete suite from the repository root with MeCab and UTF-8 IPADIC
+installed:
+
+```sh
+bundle exec rake test
+```
+
+[Prose benchmark tests](../test/prose_benchmark_test.rb) check exact agreement
+between the AsciiDoc input and JSON corpus, source mapping and paragraph order,
+annotated target positions, false positives and duplicates, and both MeCab and
+explicit literal analysis. They also check response validation, comparison
+compatibility, summary calculations, and mechanical verification in a trial
+using a fake evaluator. The fake evaluator makes the workflow test repeatable;
+it does not assess readability or understand the revision pairs.
+
+[Default rule tests](../test/default_rules_test.rb) check metaphor detection,
+allow lists, and protected text using the detection examples.
+[Writing tests](../test/writing_test.rb) and
+[review tests](../test/review_test.rb) check that the shared criteria reach writing
+prompts and saved review prompts. Existing sessions retain their saved criteria.
+
+Normal tests never invoke Codex or Claude. A passing suite establishes these
+contracts and regressions, not a measured improvement in model-generated prose.
+Literal-mode checks do not validate MeCab; there is no automatic fallback.
+
+## Establish a detector baseline
+
+Before changing the detector or review criteria, save a result on the fixed
+corpus. After the change, use a new output directory and compare it:
+
+```sh
+bundle exec ruby script/evaluate-prose --output .pubkit/evaluation/before
+# After changing the detector or review criteria:
+bundle exec ruby script/evaluate-prose --output .pubkit/evaluation/after \
+  --compare .pubkit/evaluation/before/report.json
+```
+
+The report scores only `generic-framing`, the four metaphorical-operation
+patterns, and `repeated-ending`. Other rule findings are counted separately.
+Detection matches a target's paragraph and kind. Location accuracy then checks
+its exact Unicode character offset; CLI source columns are one-based. A repeated
+ending target is at the matching ending of the third consecutive sentence.
+
+| Report field | Definition |
+| --- | --- |
+| `detection.precision` | Matched targets divided by detected scored candidates, × 100 |
+| `detection.recall` | Matched targets divided by annotated targets, × 100 |
+| `detection.f1` | 2 × matched targets divided by the sum of detected and annotated counts, × 100 |
+| `detection.location_accuracy` | Correctly located matches divided by all matched targets, × 100 |
+| `detection_delta` | After minus before for these four metrics, in percentage points |
+
+Duplicate detections count as false positives. Zero denominators are `null`, not
+perfect scores. A correctly detected candidate can still deserve a `keep`
+decision in contextual review.
+
+The historical [MeCab baseline](../benchmark/baseline-mecab.json) belongs to the
+original 13-paragraph `technical-prose-v1` corpus. Its precision was 100, recall
+81.818, F1 90, and location accuracy 77.778. Later detector fixes brought the
+11-target MeCab results to 100 for all four metrics; literal recall remained
+81.818. Keep that historical report unchanged. It cannot be compared directly
+with v2: create a new baseline after expanding the corpus.
+
+## Compare review criteria with AI trials
+
+First fix the corpus, calibration pairs, and judge rubric. Save an AI baseline
+before changing the review criteria, then change only the intended criteria and
+rerun under the same conditions:
+
+```sh
+bundle exec ruby script/evaluate-prose --agent codex --repeats 3 \
+  --output .pubkit/evaluation/ai-before
+# After changing only the review criteria:
+bundle exec ruby script/evaluate-prose --agent codex --repeats 3 \
+  --output .pubkit/evaluation/ai-after \
+  --compare .pubkit/evaluation/ai-before/report.json
+```
+
+These commands use each CLI's default model. For controlled comparisons, use
+`--model` and `--judge-model` to select fixed writer and judge models available
+in your environment. `--judge-agent` selects a separate evaluator CLI; otherwise
+the writer's CLI is used. Same-model judging can favor the writer's style.
+Claude trials use `--agent claude`. AI trials require MeCab; literal mode is
+available only for deterministic detector evaluation.
+
+Each trial performs these steps:
+
+1. Create a fresh review session and generate its actual saved review prompt.
+2. Supply the prompt and source checklists to the writer, requesting one JSON
+   paragraph per corpus ID. Faithful paragraphs can remain unchanged.
+3. Install revisions in the temporary trial manuscript and run mechanical
+   verification for scope, inline preservation, and structure.
+4. Invoke the judge separately with original and revised text labelled A/B,
+   alternating the order across trials. It receives source evidence but no
+   candidate findings or expected calibration answers.
+5. Summarize readability, checklist retention, additions, substitutions,
+   calibration, and mechanical verification separately.
+
+The writer receives evidence checklists, so the trial measures an
+evidence-assisted review workflow. It does not simulate every manuscript review.
+The judge sees original source evidence, so neutral labels reduce presentation
+bias without guaranteeing full blindness.
+
+## Read the AI results
+
+| Metric | Interpretation |
+| --- | --- |
+| `sentence_clarity` | Actors, actions, conditions, and sentence relationships are understandable |
+| `paragraph_coherence` | Sentences form a connected explanation |
+| `fact_retention` | Percentage of supplied checklist entries retained, including explicit editorial constraints |
+| `addition_free_paragraphs` | Percentage with no unsupported additions identified by the judge |
+| `substitution_free_paragraphs` | Percentage with no substitutions identified by the judge |
+| Calibration `accuracy` | Percentage of revision-pair dispositions matching the expected answers |
+| Calibration `balanced_accuracy` | Mean recall across `accept`, `reject`, and `needs-evidence` classes |
+| Mechanical verification pass rate | Percentage of trials passing the existing mechanical verifier |
+
+Readability is rated per paragraph on an ordinal scale: 1 means hard to
+understand, 2 means some ambiguity or distracting framing, and 3 means clear.
+Ratings are mapped to 0, 50, and 100 and averaged. These averages are descriptive
+summaries, not validated interval measurements. Inspect the paragraph reasons
+and actual revisions alongside the numbers.
+
+`ai.trials` contains each trial's `original`, `revision`, and `delta` metrics;
+`delta` is revision minus original. `ai.summary` retains means, population
+standard deviations, minima, and maxima. `ai_delta` compares these summary means
+between runs. In particular, `ai_delta.delta.sentence_clarity` compares the
+within-trial clarity improvement of the new run with that of the baseline.
+
+For an illustrative example, clarity of 50 before rewriting and 75 afterwards
+means a within-trial delta of +25. If the next run's delta is +30, the improvement
+relative to the baseline is +5 points. These are example numbers, not measured
+results. A readability gain with lower retention or new unsupported facts should
+not be accepted merely because it scores higher.
+
+One trial is a smoke test. Use at least three to inspect variation; the runner
+provides no significance test. Poor calibration warrants inspection before
+interpreting quality results. `meaning_verified` remains `false` regardless of
+scores or mechanical pass rate.
+
+## Comparison conditions and artifacts
+
+Comparisons require the same corpus, document, benchmark identity, scoring
+version, analyzer, and dictionary. Reports with configuration fingerprints also
+check the fixed configuration. AI comparisons additionally require matching
+judge rubric and calibration fingerprints, CLI versions, and actual reported
+writer and judge models. Missing actual model identity prevents AI comparison.
+
+Keep CLI user configuration consistent as well. Reports record criteria, rules,
+source, harness, and analyzer fingerprints, but a fingerprint cannot restore old
+content: retain the source revision and any uncommitted changes needed to
+reproduce the run. A changed corpus, judge rubric, or calibration set needs a
+new baseline. Compare the intended criteria or detector change only after the
+other evaluation conditions are fixed.
+
+AI trials require installed, authenticated local Codex or Claude CLIs. They may
+call their configured remote model providers and consume account usage. The
+runner uses argument arrays, read-only Codex execution or tool-disabled Claude
+execution, and a per-invocation timeout (`--timeout`, default 300 seconds).
+Missing CLIs, authentication failures, timeouts, and invalid responses fail
+explicitly. See [the adapter](../lib/asciidoc_pubkit/local_evaluator.rb) for the
+required flags; older CLIs may need an update.
+
+Outputs are saved under the chosen directory, including `report.json`, AI-mode
+`detection.json`, and per-trial review sessions, verification results, prompts,
+and evaluator responses. Existing output directories are never replaced.
+Partial artifacts remain after a failure. Keep results under ignored `.pubkit/`;
+do not commit private evaluation artifacts.
+
+For the version 1.0.1 yomiyasu v1.0.6 adaptation, the expanded v2 deterministic
+benchmark ran before and after the criteria change: precision, recall, F1, and
+location accuracy were all 100, with zero deltas. The suite passed 203 tests and
+2,552 assertions with real MeCab/IPADIC. These are detector and workflow checks.
+External AI rewrite-and-judge trials have not yet been run for this change, so
+its readability and preservation improvement remains unmeasured.
+
+[Development](development.md) · [Manuscript scoring](scoring.md) ·
+[Back to README](../README.md)

data/docs/references.md

@@ -15,6 +15,12 @@ structure, and final edits to the reviewer.
   and separate assessment of meaning preservation and readability. Applied here:
   four limited metaphor patterns and third-sentence repetition locations inform
   candidate detection; the shared criteria require evidence for concrete rewrites.
+  The 1.0.1 update adapts the structural comparison and sentence-function
+  guidance from [v1.0.6, commit 4075fc3](https://github.com/nanaism/yomiyasu/commit/4075fc34fb0333ca1fe42d32cd619338f4a138ba):
+  compare modifier and quantity targets, condition scope, parallel relationships,
+  and order before and after revision; preserve sentence function separately
+  from plain or polite style. The new examples are original project fixtures.
+  Markdown-specific bold fixes and numeric style thresholds are not imported.
 - [日本語技術文書の文章規範](https://gist.github.com/k16shikano/fd287c3133457c4fd8f5601d34aa817d):
   informs paragraph logic, evidence scope, and meaningful uncertainty. Applied
   here: the shared criteria start from each paragraph's technical purpose and

data/docs/review-workflow.md

@@ -151,7 +151,7 @@ manuscripts. Numeric title changes
 are manual-review notices, and remaining candidates do not fail verification.
 `meaning_verified` remains `false`.
 
-Version 1.0.0 uses session schema 3 and requires the exact tool version saved in
+Version 1.0.1 uses session schema 3 and requires the exact tool version saved in
 the session. Preserve or finish an ongoing review with its original tool before
 establishing a new baseline in a separate directory.
 `review score` continues to support prose only and rejects `--scope`.

data/docs/rules-and-coverage.md

@@ -131,13 +131,13 @@ reports analyzer changes instead of treating results from different dictionaries
 as directly comparable. Prompt generation uses saved evidence and does not need
 MeCab. Changed rules or dictionary settings require a new scan.
 
-Version 1.0.0 uses session schema 3.
+Version 1.0.1 uses session schema 3.
 Sessions with a different schema or tool version cannot be loaded. Keep the original baseline for
 an ongoing review and finish it with the original version, or start a new review
 pass in a different directory:
 
 ```sh
-asciidoc-pubkit review scan book.adoc --output .pubkit/review-1.0.0
+asciidoc-pubkit review scan book.adoc --output .pubkit/review-1.0.1
 ```
 
 Severity describes review priority, not proof of an error. There is no AI-authorship

data/docs/writing.md

@@ -9,13 +9,22 @@ it does not supply source facts or authorize edits. Book-specific voice and
 format still come from the book. The OSS-only prose style profile remains with
 the book workflow, not the shared default.
 
+The criteria also ask reviewers to compare source and revision for
+modifier targets, condition and exception scope, quantities and their targets,
+parallel relationships, and required order. Preserve each sentence's function
+(including advice, obligations, plans, and actual behavior) separately from plain
+or polite style. Keep the author's style unless a change is requested or the
+book explicitly requires one. Sentence splitting and punctuation changes must
+retain the original relationships; these are contextual review instructions,
+not automatic semantic checks. Saved sessions retain the criteria from their scan.
+
 ```sh
 asciidoc-pubkit writing criteria --lang ja
 asciidoc-pubkit writing prompt --lang ja --output writing-prompt.md
 ```
 
 For example, `--lang en` exits with an unsupported-language error. No English
-review or writing criteria are shipped in 1.0.0.
+review or writing criteria are shipped in 1.0.1.
 
 For a small trial, use `examples/book.adoc` as the scan input. Its Japanese
 paragraphs deliberately contain review candidates; its code block must remain

data/lib/asciidoc_pubkit.rb

@@ -9,7 +9,7 @@ require 'pathname'
 require 'optparse'
 
 module AsciidocPubkit
-  VERSION = '1.0.0'
+  VERSION = '1.0.1'
   class Error < StandardError; end
 
   def self.hash_text(text)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: asciidoc-pubkit
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.0.1
 platform: ruby
 authors:
 - CyberGarage
@@ -61,6 +61,7 @@ files:
 - docs/english-review-research.md
 - docs/installation.md
 - docs/list-review.md
+- docs/prose-evaluation.md
 - docs/references.md
 - docs/replacements.md
 - docs/review-workflow.md