asciidoc-pubkit 0.1.1 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +34 -0
- data/README.md +210 -17
- data/data/review-rules.ja.yml +271 -0
- data/data/writing/ja/criteria.md +178 -0
- data/lib/asciidoc_pubkit/cli.rb +44 -10
- data/lib/asciidoc_pubkit/document.rb +5 -0
- data/lib/asciidoc_pubkit/language.rb +16 -0
- data/lib/asciidoc_pubkit/rule_set.rb +57 -0
- data/lib/asciidoc_pubkit/rules.rb +50 -45
- data/lib/asciidoc_pubkit/session.rb +85 -17
- data/lib/asciidoc_pubkit/settings.rb +7 -4
- data/lib/asciidoc_pubkit/writing.rb +33 -0
- data/lib/asciidoc_pubkit.rb +4 -1
- metadata +10 -5
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
# Shared criteria for Japanese technical prose
|
|
2
|
+
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
- Start with the paragraph's technical purpose
|
|
6
|
+
- Replace abstraction with the thing being discussed
|
|
7
|
+
- Give weak predicates a concrete action or consequence
|
|
8
|
+
- Make relationships explicit
|
|
9
|
+
- Keep claims within their evidence and purpose
|
|
10
|
+
- Remove generated-prose patterns without flattening the meaning
|
|
11
|
+
- Preserve exact technical terminology
|
|
12
|
+
- Revise the complete paragraph
|
|
13
|
+
- Headings and the relation to figures and code
|
|
14
|
+
|
|
15
|
+
Apply these criteria in context. They identify patterns that deserve inspection,
|
|
16
|
+
not words that must always be removed.
|
|
17
|
+
|
|
18
|
+
## Start with the paragraph's technical purpose
|
|
19
|
+
|
|
20
|
+
Before rewriting sentences, summarize each paragraph in one sentence:
|
|
21
|
+
|
|
22
|
+
1. What implementation, behavior, or decision is this paragraph about?
|
|
23
|
+
2. Who or what performs the action?
|
|
24
|
+
3. Under what condition does it apply?
|
|
25
|
+
4. What changes or follows as a result?
|
|
26
|
+
5. What can an engineer decide or verify from it?
|
|
27
|
+
|
|
28
|
+
If the summary is impossible, repair the paragraph's argument before polishing
|
|
29
|
+
individual phrases. A short sentence is not automatically clear; omitted
|
|
30
|
+
subjects, objects, complements, and referents must still be recoverable from the
|
|
31
|
+
adjacent sentences.
|
|
32
|
+
|
|
33
|
+
## Replace abstraction with the thing being discussed
|
|
34
|
+
|
|
35
|
+
Abstract terms are valid when their referent is stated. Inspect terms such as:
|
|
36
|
+
|
|
37
|
+
| Term | What must be identifiable |
|
|
38
|
+
| --- | --- |
|
|
39
|
+
| 費用・コスト | investigation time, maintenance work, CPU time, memory, network transfer, or money |
|
|
40
|
+
| 契約 | accepted inputs, returned values and errors, compatibility, ownership, concurrency, or another promised behavior |
|
|
41
|
+
| 境界 | the two components and the call, data transfer, or ownership transfer between them |
|
|
42
|
+
| 観点 | the actual comparison axis, such as type safety, shutdown behavior, allocation count, or supported OS |
|
|
43
|
+
| 書き方 | syntax, implementation technique, API shape, or document notation |
|
|
44
|
+
| 範囲・射程 | supported readers, covered subjects, applicable versions, or behavior guaranteed by an API |
|
|
45
|
+
|
|
46
|
+
Do not improve a sentence by replacing one abstraction with another. For
|
|
47
|
+
example, changing `費用` to `コスト` says nothing unless the sentence identifies
|
|
48
|
+
what increases.
|
|
49
|
+
|
|
50
|
+
## Give weak predicates a concrete action or consequence
|
|
51
|
+
|
|
52
|
+
Predicates such as `利用します`, `決めます`, `示します`, `扱います`, `整理します`,
|
|
53
|
+
`表します`, `保てます`, `異なります`, `同じではありません`, `残します`, and
|
|
54
|
+
`評価します` often omit the important information. Ask what is used for what,
|
|
55
|
+
which value or policy is decided, what differs, where evidence is stored, and
|
|
56
|
+
which result drives the decision.
|
|
57
|
+
|
|
58
|
+
Prefer a precise action or consequence:
|
|
59
|
+
|
|
60
|
+
- `検証結果を利用します` becomes `検証結果をAPIの見直し材料にします` when that
|
|
61
|
+
is the actual purpose.
|
|
62
|
+
- `採用する機能を決めます` becomes `サポートするバージョンと互換性を基に採用を
|
|
63
|
+
判断します` when those are the criteria.
|
|
64
|
+
- `結果を残します` becomes `実行したコマンドと結果をプルリクエストへ記録します`
|
|
65
|
+
when that is the storage location.
|
|
66
|
+
|
|
67
|
+
Do not choose a more impressive synonym merely to vary sentence endings. Keep
|
|
68
|
+
the technically correct verb when it names the real operation.
|
|
69
|
+
|
|
70
|
+
## Make relationships explicit
|
|
71
|
+
|
|
72
|
+
Inspect a sentence when it:
|
|
73
|
+
|
|
74
|
+
- begins a new topic without connecting it to the preceding paragraph;
|
|
75
|
+
- says two things differ without naming the differing property;
|
|
76
|
+
- introduces a comparison without naming the alternatives and comparison axis;
|
|
77
|
+
- uses `これ`, `それ`, `前者`, or `後者` when the referent requires rereading;
|
|
78
|
+
- strings three or more abstract nouns together without explaining their
|
|
79
|
+
relationship;
|
|
80
|
+
- uses an abrupt `読者は` even though the effect can be stated directly;
|
|
81
|
+
- attaches `へ` or `から` to an abstract noun in phrases such as `判断へ戻す`;
|
|
82
|
+
- uses a literal translation that is not an established term in the technical
|
|
83
|
+
domain.
|
|
84
|
+
|
|
85
|
+
When a standalone sentence cannot identify its subject or object, use the
|
|
86
|
+
surrounding paragraph to supply the missing noun. Do not repeat the subject in
|
|
87
|
+
every sentence when it is already unambiguous.
|
|
88
|
+
Use prose for a sequence in which one event causes or conditions the next; use
|
|
89
|
+
a list when the items are genuinely parallel. Do not turn a causal explanation
|
|
90
|
+
into disconnected bullets or impose an order on independent items.
|
|
91
|
+
|
|
92
|
+
## Keep claims within their evidence and purpose
|
|
93
|
+
|
|
94
|
+
State whether a claim comes from an implementation, a test, a cited study, or
|
|
95
|
+
the author's inference. Match its scope to that evidence: a single example or
|
|
96
|
+
test establishes what happened under its stated conditions, not a universal
|
|
97
|
+
result. Preserve uncertainty when evidence does not settle the claim; do not
|
|
98
|
+
weaken an established fact merely to sound cautious.
|
|
99
|
+
|
|
100
|
+
Before adding a qualification, ask what decision or interpretation it changes.
|
|
101
|
+
Correct an overbroad main claim at its source. Remove a following disclaimer if
|
|
102
|
+
the preceding explanation already excludes that reading. When a limitation
|
|
103
|
+
affects use, implementation, or verification, state the concrete condition and
|
|
104
|
+
connect it to the relevant action or check. Do not delete a technical boundary
|
|
105
|
+
merely because it uses negative wording.
|
|
106
|
+
|
|
107
|
+
## Remove generated-prose patterns without flattening the meaning
|
|
108
|
+
|
|
109
|
+
Common patterns that merit paragraph-level review include:
|
|
110
|
+
|
|
111
|
+
- repeated `AではなくBです` or `AだけでなくBも` contrasts;
|
|
112
|
+
- paired short slogans that omit the actor, object, condition, or consequence;
|
|
113
|
+
- repeated endings such as `扱います`, `示します`, or `選びます`;
|
|
114
|
+
- `重要なのは`, `ポイントは`, or `〜することです` standing in for the actual
|
|
115
|
+
technical fact;
|
|
116
|
+
- a section opening that only announces what the section will discuss;
|
|
117
|
+
- a section ending that only says the material was explained or organized;
|
|
118
|
+
- metaphors such as `土台`, `器`, `橋渡し`, or `輪が閉じる` used instead of a
|
|
119
|
+
concrete technical operation;
|
|
120
|
+
- a standalone qualification that only denies an overbroad reading already
|
|
121
|
+
excluded by the preceding explanation, such as a redundant
|
|
122
|
+
`という意味ではありません` or `保証ではありません`;
|
|
123
|
+
- generic praise, warnings, or recommendations without the condition that makes
|
|
124
|
+
them true.
|
|
125
|
+
|
|
126
|
+
Keep a contrast when the two alternatives and the difference that affects the
|
|
127
|
+
decision are explicit. Keep necessary repetition when it improves technical
|
|
128
|
+
accuracy.
|
|
129
|
+
Do not shorten sentences or vary endings mechanically. Read the sequence aloud
|
|
130
|
+
and repair the missing relationship rather than replacing one stock pattern
|
|
131
|
+
with another.
|
|
132
|
+
|
|
133
|
+
## Preserve exact technical terminology
|
|
134
|
+
|
|
135
|
+
Check terminology against identifiers, surrounding code, the project's
|
|
136
|
+
glossary, and primary documentation before changing it.
|
|
137
|
+
|
|
138
|
+
- Distinguish an API from a function, method, interface, protocol, or command.
|
|
139
|
+
- Distinguish an error value, operation failure, incident, and defect.
|
|
140
|
+
- Distinguish resource ownership from the responsibility to close or stop it.
|
|
141
|
+
- Use the project's chosen forms consistently, such as `goroutine` versus
|
|
142
|
+
`ゴルーチン`, or `reflection` versus `リフレクション`.
|
|
143
|
+
- Replace unnecessary English words with established Japanese terms, but retain
|
|
144
|
+
identifiers and official names.
|
|
145
|
+
- Do not describe a language operation with a loose everyday verb when the
|
|
146
|
+
domain has a precise term, such as name shadowing, method promotion, parsing,
|
|
147
|
+
serialization, cancellation, or resource release.
|
|
148
|
+
|
|
149
|
+
Natural Japanese must not erase a technical distinction. If the original claim
|
|
150
|
+
is ambiguous, inspect the code or authoritative documentation before editing.
|
|
151
|
+
|
|
152
|
+
## Revise the complete paragraph
|
|
153
|
+
|
|
154
|
+
Use a finding to understand the intended claim, then rewrite the paragraph so
|
|
155
|
+
that every sentence connects to the next. Do not paste an isolated suggested
|
|
156
|
+
sentence into the manuscript without adjusting its context. After editing,
|
|
157
|
+
read the paragraph as continuous prose and verify subject, object, modifier,
|
|
158
|
+
condition, cause, and effect.
|
|
159
|
+
|
|
160
|
+
|
|
161
|
+
## Headings and the relation to figures and code
|
|
162
|
+
|
|
163
|
+
Headings identify the subject, operation, or comparison actually explained.
|
|
164
|
+
Do not promise a broader comparison than the text supports or replace the
|
|
165
|
+
subject with an unexplained metaphor. Heading length and required section
|
|
166
|
+
names belong to the book's format, not to universal prose quality.
|
|
167
|
+
Do not force every heading into a conclusion sentence or the same template;
|
|
168
|
+
make the section's content identifiable in the form the book uses.
|
|
169
|
+
|
|
170
|
+
Introduce tables and figures with enough context to explain what is compared
|
|
171
|
+
or traced, and explain what the reader can establish from them. Keep identifiers,
|
|
172
|
+
values, ordering, and conditions consistent with the code or clearly label a
|
|
173
|
+
simplified conceptual representation. Do not repeat every label as prose.
|
|
174
|
+
|
|
175
|
+
File paths and type names belong where they help explain a specific operation;
|
|
176
|
+
avoid a list of identifiers before the reader knows their roles. Bibliographic
|
|
177
|
+
and implementation-file descriptions introduce the referenced material, rather
|
|
178
|
+
than narrating the author's research activity.
|
data/lib/asciidoc_pubkit/cli.rb
CHANGED
|
@@ -3,9 +3,11 @@
|
|
|
3
3
|
module AsciidocPubkit
|
|
4
4
|
class CLI
|
|
5
5
|
HELP = <<~TEXT
|
|
6
|
-
Usage: asciidoc-pubkit
|
|
6
|
+
Usage: asciidoc-pubkit <group> <command> [options] [input]
|
|
7
7
|
|
|
8
8
|
Commands:
|
|
9
|
+
writing criteria Print the shared Japanese prose criteria
|
|
10
|
+
writing prompt Generate a Japanese technical writing prompt
|
|
9
11
|
review scan FILE Collect Japanese prose and review candidates
|
|
10
12
|
review prompt SESSION Generate an English review prompt with Japanese source excerpts
|
|
11
13
|
review verify SESSION Compare edited sources with the saved baseline
|
|
@@ -14,30 +16,38 @@ module AsciidocPubkit
|
|
|
14
16
|
--help Show help (also supported for each command)
|
|
15
17
|
--version Show the version
|
|
16
18
|
|
|
19
|
+
Use --lang ja with writing commands and review commands. Other review and
|
|
20
|
+
writing languages are not supported yet and return an error.
|
|
17
21
|
No command edits manuscript files or invokes an AI service.
|
|
18
22
|
TEXT
|
|
19
23
|
|
|
20
|
-
def self.run(arguments, out: $stdout, err: $stderr)
|
|
24
|
+
def self.run(arguments, out: $stdout, err: $stderr, input: $stdin)
|
|
21
25
|
args = arguments.dup
|
|
22
26
|
if args == ['--version']
|
|
23
27
|
out.puts VERSION
|
|
24
28
|
return 0
|
|
25
29
|
end
|
|
26
|
-
if args.empty? || args == ['--help'] || args == ['-h'] || args == [
|
|
30
|
+
if args.empty? || args == ['--help'] || args == ['-h'] || %w[review writing].any? { |group| args == [group, '--help'] }
|
|
27
31
|
out.puts HELP
|
|
28
32
|
return 0
|
|
29
33
|
end
|
|
30
34
|
group, command = args.shift(2)
|
|
31
|
-
|
|
35
|
+
valid = (group == 'review' && %w[scan prompt verify].include?(command)) ||
|
|
36
|
+
(group == 'writing' && %w[criteria prompt].include?(command))
|
|
37
|
+
raise Error, 'Unknown command. Use --help.' unless valid
|
|
32
38
|
options = { attributes: {} }
|
|
33
39
|
parser = OptionParser.new do |opts|
|
|
34
|
-
|
|
35
|
-
opts.
|
|
36
|
-
|
|
40
|
+
input_name = group == 'writing' ? '' : (command == 'scan' ? ' FILE' : ' SESSION')
|
|
41
|
+
opts.banner = "Usage: asciidoc-pubkit #{group} #{command} [options]#{input_name}"
|
|
42
|
+
opts.on('-o', '--output PATH', group == 'review' && command == 'scan' ? 'New session directory (default: .pubkit/review)' : 'New output file (default: standard output)') { |v| options[:output] = v }
|
|
43
|
+
opts.on('--lang LANG', 'Language (ja only in this release)') { |v| options[:language] = v }
|
|
44
|
+
if group == 'review' && command == 'scan'
|
|
45
|
+
opts.on('-y', '--yes', 'Answer yes to replacement confirmation') { options[:yes] = true }
|
|
46
|
+
opts.on('--no-input', 'Never prompt; fail on existing output unless --yes') { options[:no_input] = true }
|
|
37
47
|
opts.on('--only FILE', 'Review one included file in the book context') { |v| options[:only] = v }
|
|
38
48
|
opts.on('--config FILE', 'Use an explicit YAML configuration') { |v| options[:config] = v }
|
|
49
|
+
opts.on('--rules FILE', 'Replace default review rules with a YAML rule set') { |v| options[:rules] = v }
|
|
39
50
|
opts.on('--base-dir DIR', 'Set the Asciidoctor base directory') { |v| options[:base_dir] = v }
|
|
40
|
-
opts.on('--lang LANG', 'Prose language (ja only)') { |v| options[:language] = v }
|
|
41
51
|
opts.on('--tokenizer NAME', 'mecab (default) or literal (limited phrase matching)') { |v| options[:tokenizer] = v }
|
|
42
52
|
opts.on('--style STYLE', 'preserve (default), desu-masu, or dearu') { |v| options[:style] = v }
|
|
43
53
|
opts.on('-a', '--attribute NAME=VALUE', 'Set an Asciidoctor attribute; repeat as needed') do |v|
|
|
@@ -45,7 +55,7 @@ module AsciidocPubkit
|
|
|
45
55
|
raise Error, 'Attribute name cannot be empty.' if key.nil? || key.empty?
|
|
46
56
|
options[:attributes][key] = value || ''
|
|
47
57
|
end
|
|
48
|
-
elsif command == 'prompt'
|
|
58
|
+
elsif group == 'review' && command == 'prompt'
|
|
49
59
|
opts.on('--mode MODE', 'revise (default) or diagnose') { |v| options[:mode] = v }
|
|
50
60
|
end
|
|
51
61
|
opts.on('-h', '--help', 'Show command help') { options[:help] = true }
|
|
@@ -55,9 +65,29 @@ module AsciidocPubkit
|
|
|
55
65
|
out.puts parser
|
|
56
66
|
return 0
|
|
57
67
|
end
|
|
68
|
+
if group == 'writing'
|
|
69
|
+
raise Error, 'Writing commands do not accept an input file. Use --help.' unless args.empty?
|
|
70
|
+
language = Language.validate!(options.fetch(:language, Language::DEFAULT), operation: 'writing')
|
|
71
|
+
content = command == 'criteria' ? Writing.criteria(language) : Writing.prompt(language)
|
|
72
|
+
output(content, options[:output], out)
|
|
73
|
+
return 0
|
|
74
|
+
end
|
|
58
75
|
raise Error, 'Exactly one input is required. Use --help.' unless args.length == 1
|
|
76
|
+
Language.validate!(options[:language], operation: 'review') if options[:language]
|
|
59
77
|
if command == 'scan'
|
|
60
|
-
result = Session.scan(args.first, options)
|
|
78
|
+
result = Session.scan(args.first, options) do |destination|
|
|
79
|
+
if options[:yes]
|
|
80
|
+
true
|
|
81
|
+
elsif options[:no_input] || !input.tty?
|
|
82
|
+
raise Error, "Output already exists: #{destination}. Use --yes to replace it or --output for a new session."
|
|
83
|
+
else
|
|
84
|
+
err.print "Output already exists: #{destination}. Replace it? [y/N] "
|
|
85
|
+
err.flush
|
|
86
|
+
answer = input.gets
|
|
87
|
+
raise Error, 'Scan cancelled; existing session was kept.' unless answer && %w[y yes].include?(answer.strip.downcase)
|
|
88
|
+
true
|
|
89
|
+
end
|
|
90
|
+
end
|
|
61
91
|
out.puts "Scanned #{result['paragraphs']} paragraphs; found #{result['findings']} review candidates."
|
|
62
92
|
out.puts "Tokenizer: #{result['tokenizer']}#{result['tokenizer'] == 'literal' ? ' (limited phrase matching; no morphological analysis)' : ' (UTF-8 IPADIC)'}"
|
|
63
93
|
out.puts "Coverage notices: #{result['coverage_notices']}. See document.json for limitations."
|
|
@@ -65,6 +95,10 @@ module AsciidocPubkit
|
|
|
65
95
|
return 0
|
|
66
96
|
end
|
|
67
97
|
session = Session.new(args.first)
|
|
98
|
+
saved_language = Language.validate!(session.language, operation: 'review')
|
|
99
|
+
if options[:language] && options[:language] != saved_language
|
|
100
|
+
raise Error, "Requested language #{options[:language].inspect} does not match session language #{saved_language.inspect}."
|
|
101
|
+
end
|
|
68
102
|
if command == 'prompt'
|
|
69
103
|
output(session.prompt(options.fetch(:mode, 'revise')), options[:output], out)
|
|
70
104
|
return 0
|
|
@@ -26,6 +26,11 @@ module AsciidocPubkit
|
|
|
26
26
|
super(*args)
|
|
27
27
|
end
|
|
28
28
|
doc.parse
|
|
29
|
+
document_language = doc.attr('lang')
|
|
30
|
+
if document_language && document_language != settings.fetch('language')
|
|
31
|
+
Language.validate!(document_language, operation: 'review')
|
|
32
|
+
raise Error, "Document language #{document_language.inspect} does not match review language #{settings.fetch('language').inspect}."
|
|
33
|
+
end
|
|
29
34
|
@line_index = Hash.new { |hash, key| hash[key] = [] }
|
|
30
35
|
@source_lines = @sources.to_h do |path, raw|
|
|
31
36
|
lines = raw.lines.map { |line| line.delete_suffix("\n").delete_suffix("\r") }
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module AsciidocPubkit
|
|
4
|
+
module Language
|
|
5
|
+
WRITING = %w[ja].freeze
|
|
6
|
+
REVIEW = %w[ja].freeze
|
|
7
|
+
DEFAULT = 'ja'
|
|
8
|
+
|
|
9
|
+
def self.validate!(value, operation:)
|
|
10
|
+
supported = { 'writing' => WRITING, 'review' => REVIEW }.fetch(operation)
|
|
11
|
+
return value if supported.include?(value)
|
|
12
|
+
|
|
13
|
+
raise Error, "Unsupported #{operation} language #{value.inspect}; supported: #{supported.join(', ')}."
|
|
14
|
+
end
|
|
15
|
+
end
|
|
16
|
+
end
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module AsciidocPubkit
|
|
4
|
+
class RuleSet
|
|
5
|
+
DEFAULT_PATH = File.expand_path('../../data/review-rules.ja.yml', __dir__)
|
|
6
|
+
CATEGORIES = %w[abstract-reference weak-predicate vague-degree contextual-phrase generic-framing].freeze
|
|
7
|
+
KEYS = %w[schema_version terms verbs sahen negative_only compound_nouns].freeze
|
|
8
|
+
|
|
9
|
+
def self.default_path(language)
|
|
10
|
+
Language.validate!(language, operation: 'review')
|
|
11
|
+
File.expand_path("../../data/review-rules.#{language}.yml", __dir__)
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def self.load(path = DEFAULT_PATH)
|
|
15
|
+
validate(YAML.safe_load(AsciidocPubkit.read_text(path), permitted_classes: [], aliases: false))
|
|
16
|
+
rescue Psych::Exception => e
|
|
17
|
+
raise Error, "Invalid rule YAML in #{path}: #{e.message}"
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def self.mapping(value, keys, label)
|
|
21
|
+
unless value.is_a?(Hash) && (value.keys - keys).empty? && (keys - value.keys).empty?
|
|
22
|
+
raise Error, "#{label} must contain exactly these keys: #{keys.join(', ')}."
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def self.strings(value, label)
|
|
27
|
+
unless value.is_a?(Array) && value.all? { |s| s.is_a?(String) && !s.strip.empty? } && value.uniq == value
|
|
28
|
+
raise Error, "#{label} must be an array of unique nonempty strings."
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def self.validate(data)
|
|
33
|
+
mapping(data, KEYS, 'Rule set')
|
|
34
|
+
raise Error, 'Rule schema_version must be integer 1.' unless data['schema_version'].is_a?(Integer) && data['schema_version'] == 1
|
|
35
|
+
mapping(data['terms'], CATEGORIES, 'Rule terms')
|
|
36
|
+
data['terms'].each do |category, entry|
|
|
37
|
+
mapping(entry, %w[terms question], category)
|
|
38
|
+
strings(entry['terms'], "#{category}.terms")
|
|
39
|
+
raise Error, "#{category}.question must be a nonempty string." unless entry['question'].is_a?(String) && !entry['question'].strip.empty?
|
|
40
|
+
end
|
|
41
|
+
verbs = data['verbs']
|
|
42
|
+
raise Error, 'verbs must be a mapping of canonical forms to lemma arrays.' unless verbs.is_a?(Hash)
|
|
43
|
+
verbs.each do |canonical, forms|
|
|
44
|
+
raise Error, 'Verb canonical forms must be nonempty strings.' unless canonical.is_a?(String) && !canonical.strip.empty?
|
|
45
|
+
strings(forms, "verbs.#{canonical}")
|
|
46
|
+
raise Error, "verbs.#{canonical} must contain at least one lemma." if forms.empty?
|
|
47
|
+
end
|
|
48
|
+
aliases = verbs.values.flatten
|
|
49
|
+
raise Error, 'Verb lemmas must have only one canonical form.' unless aliases.uniq == aliases
|
|
50
|
+
%w[sahen negative_only compound_nouns].each { |key| strings(data[key], key) }
|
|
51
|
+
canonical_forms = verbs.keys + data['sahen'].map { |noun| noun + 'する' }
|
|
52
|
+
raise Error, 'negative_only must refer to configured canonical predicates.' unless (data['negative_only'] - canonical_forms).empty?
|
|
53
|
+
raise Error, 'compound_nouns must also appear in abstract-reference terms.' unless (data['compound_nouns'] - data['terms']['abstract-reference']['terms']).empty?
|
|
54
|
+
data
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
@@ -3,30 +3,14 @@
|
|
|
3
3
|
module AsciidocPubkit
|
|
4
4
|
class Rules
|
|
5
5
|
INLINE = /`[^`\n]*`|\+\+\+.*?\+\+\+|\+\+[^\n]*?\+\+|(?<!\w)\+[^+\n]+\+|「[^」\n]*」|『[^』\n]*』|\{[^}\n]+\}|<<[^>\n]+>>|\[\[[^\]\n]+\]\]|(?:link|xref|image|footnote|pass):[^\s\[]*\[[^\]\n]*\]|https?:\/\/[^\s\[\]<>]+(?:\[[^\]\n]*\])?/m
|
|
6
|
-
TERMS = {
|
|
7
|
-
'abstract-reference' => [%w[コスト 境界 契約 観点 土台 橋渡し 入口 記述 場所 意図 役割 一続き 根拠 部品 開発者], 'Identify the concrete referent, components, or measurable work. Keep established technical meanings.'],
|
|
8
|
-
'weak-predicate' => [%w[利用します 整理します 扱います 示します 変わります 把握します 分けられます そろえます まとまっています 加えます 探します 到達しません 扱いません そろいます 選べます 成り立たせています 確かめます 書き換える 絞れます 渡します あります], 'Check whether the purpose, operation, or result is clear from the surrounding paragraph. Preserve negation and conditions.'],
|
|
9
|
-
'vague-degree' => [%w[浅い 深い], 'Identify the concrete depth, level, scope, or comparison. Keep literal measurements and established technical meanings.'],
|
|
10
|
-
'contextual-phrase' => [%w[これらを であることだけでは あるものとします わけではありません], 'Check the referent, assumption, or qualification against the surrounding explanation. Preserve conditions and negation.'],
|
|
11
|
-
'generic-framing' => [%w[重要なのは ポイントは 本章では ここでは まとめると], 'Check whether this framing adds useful scope or information instead of repeating the explanation.']
|
|
12
|
-
}.freeze
|
|
13
|
-
VERBS = {
|
|
14
|
-
'扱う' => %w[扱う], '示す' => %w[示す], '変わる' => %w[変わる],
|
|
15
|
-
'分ける' => %w[分ける], 'そろえる' => %w[そろえる 揃える],
|
|
16
|
-
'まとまる' => %w[まとまる 纏まる], '加える' => %w[加える],
|
|
17
|
-
'探す' => %w[探す], 'そろう' => %w[そろう 揃う],
|
|
18
|
-
'選ぶ' => %w[選ぶ 選べる], '成り立つ' => %w[成り立つ],
|
|
19
|
-
'確かめる' => %w[確かめる], '書き換える' => %w[書き換える],
|
|
20
|
-
'絞る' => %w[絞る 絞れる], '渡す' => %w[渡す], 'ある' => %w[ある]
|
|
21
|
-
}.freeze
|
|
22
|
-
SAHEN = %w[利用 整理 把握 到達].freeze
|
|
23
|
-
|
|
24
6
|
def self.mask(text)
|
|
25
7
|
# Keep character offsets stable while excluding common inline constructs.
|
|
26
8
|
text.gsub(INLINE) { |match| match.gsub(/[^\n]/, ' ') }
|
|
27
9
|
end
|
|
28
10
|
|
|
29
11
|
def self.scan(paragraphs, settings, tokenizer: nil)
|
|
12
|
+
rules = RuleSet.validate(settings.fetch('rules') { RuleSet.load })
|
|
13
|
+
terms = rules.fetch('terms').transform_values { |entry| entry.values_at('terms', 'question') }
|
|
30
14
|
findings = []
|
|
31
15
|
if settings.fetch('tokenizer', 'mecab') == 'mecab'
|
|
32
16
|
tokenizer ||= Morphology.new(settings)
|
|
@@ -34,14 +18,8 @@ module AsciidocPubkit
|
|
|
34
18
|
token_groups = tokenizer ? tokenize_paragraphs(paragraphs, tokenizer) : []
|
|
35
19
|
paragraphs.each_with_index do |paragraph, index|
|
|
36
20
|
text = mask(paragraph.fetch('text'))
|
|
37
|
-
scan_morphemes(findings, paragraph, text, token_groups[index], settings) if tokenizer
|
|
38
|
-
|
|
39
|
-
next if tokenizer && !%w[generic-framing contextual-phrase].include?(rule)
|
|
40
|
-
terms.each do |term|
|
|
41
|
-
next if settings.fetch('allows').include?(term)
|
|
42
|
-
find_term(findings, paragraph, text, term, rule, 'hint', question)
|
|
43
|
-
end
|
|
44
|
-
end
|
|
21
|
+
scan_morphemes(findings, paragraph, text, token_groups[index], settings, rules, terms) if tokenizer
|
|
22
|
+
scan_literal_terms(findings, paragraph, text, terms, settings, morphological: !!tokenizer)
|
|
45
23
|
settings.fetch('glossary').each do |canonical, variants|
|
|
46
24
|
variants.each do |variant|
|
|
47
25
|
find_term(findings, paragraph, text, variant, 'glossary-variant', 'warning', "Use #{canonical.inspect} when this variant refers to the same concept; preserve identifiers and quotations.")
|
|
@@ -90,8 +68,8 @@ module AsciidocPubkit
|
|
|
90
68
|
end
|
|
91
69
|
end
|
|
92
70
|
|
|
93
|
-
def self.scan_morphemes(findings, paragraph, text, tokens, settings)
|
|
94
|
-
phrase_ranges =
|
|
71
|
+
def self.scan_morphemes(findings, paragraph, text, tokens, settings, rules, terms)
|
|
72
|
+
phrase_ranges = terms['contextual-phrase'][0].flat_map do |phrase|
|
|
95
73
|
text.to_enum(:scan, Regexp.new(Regexp.escape(phrase))).map do
|
|
96
74
|
match = Regexp.last_match
|
|
97
75
|
match.begin(0)...match.end(0)
|
|
@@ -102,40 +80,48 @@ module AsciidocPubkit
|
|
|
102
80
|
lemma = token['lemma']
|
|
103
81
|
rule = nil
|
|
104
82
|
finish_index = index
|
|
105
|
-
compound =
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
83
|
+
compound = rules.fetch('compound_nouns').find do |term|
|
|
84
|
+
surface = +''
|
|
85
|
+
cursor = index
|
|
86
|
+
while (member = tokens[cursor]) && member['pos'] == '名詞' && !member['unknown']
|
|
87
|
+
break if cursor > index && tokens[cursor - 1]['end_offset'] != member['offset']
|
|
88
|
+
surface << member['surface']
|
|
89
|
+
break unless term.start_with?(surface)
|
|
90
|
+
if surface == term
|
|
91
|
+
finish_index = cursor
|
|
92
|
+
break
|
|
93
|
+
end
|
|
94
|
+
cursor += 1
|
|
95
|
+
end
|
|
96
|
+
surface == term
|
|
110
97
|
end
|
|
111
98
|
if compound
|
|
112
99
|
lemma = compound
|
|
113
|
-
finish_index = index + 1
|
|
114
100
|
rule = 'abstract-reference'
|
|
115
|
-
elsif token['pos'] == '名詞' &&
|
|
101
|
+
elsif token['pos'] == '名詞' && token['pos_detail'] == 'サ変接続' && rules.fetch('sahen').include?(lemma) &&
|
|
102
|
+
(following = tokens[index + 1]) && following['pos'] == '動詞' &&
|
|
103
|
+
%w[する できる].include?(following['lemma']) && adjacent?(token, following, text)
|
|
104
|
+
lemma += 'する'
|
|
105
|
+
rule = 'weak-predicate'
|
|
106
|
+
finish_index = predicate_end(tokens, index + 1, text)
|
|
107
|
+
elsif token['pos'] == '名詞' && terms['abstract-reference'][0].include?(lemma)
|
|
116
108
|
rule = 'abstract-reference'
|
|
117
|
-
elsif token['pos'] == '形容詞' &&
|
|
109
|
+
elsif token['pos'] == '形容詞' && terms['vague-degree'][0].include?(lemma)
|
|
118
110
|
rule = 'vague-degree'
|
|
119
111
|
finish_index = predicate_end(tokens, index, text)
|
|
120
|
-
elsif token['pos'] == '動詞' && (entry =
|
|
112
|
+
elsif token['pos'] == '動詞' && (entry = rules.fetch('verbs').find { |_canonical, forms| forms.include?(lemma) })
|
|
121
113
|
lemma = entry[0]
|
|
122
114
|
rule = 'weak-predicate'
|
|
123
115
|
finish_index = predicate_end(tokens, index, text)
|
|
124
|
-
elsif token['pos'] == '名詞' && token['pos_detail'] == 'サ変接続' && SAHEN.include?(lemma)
|
|
125
|
-
following = tokens[index + 1]
|
|
126
|
-
next unless following && following['pos'] == '動詞' && following['lemma'] == 'する' && adjacent?(token, following, text)
|
|
127
|
-
lemma += 'する'
|
|
128
|
-
rule = 'weak-predicate'
|
|
129
|
-
finish_index = predicate_end(tokens, index + 1, text)
|
|
130
116
|
end
|
|
131
117
|
next unless rule
|
|
132
118
|
members = tokens[index..finish_index]
|
|
133
119
|
negative = members.any? { |member| member['pos'] == '助動詞' && %w[ない ぬ ん].include?(member['lemma']) }
|
|
134
|
-
next if
|
|
120
|
+
next if rules.fetch('negative_only').include?(lemma) && !negative
|
|
135
121
|
surface = text[token['offset']...tokens[finish_index]['end_offset']]
|
|
136
122
|
allows = settings.fetch('allows')
|
|
137
123
|
next if [lemma, token['lemma'], surface, surface + '。'].any? { |form| allows.include?(form) }
|
|
138
|
-
add(findings, paragraph, text, token['offset'], surface, rule, 'hint',
|
|
124
|
+
add(findings, paragraph, text, token['offset'], surface, rule, 'hint', terms.fetch(rule)[1])
|
|
139
125
|
findings.last.merge!('lemma' => lemma, 'part_of_speech' => token['pos'], 'negative' => negative,
|
|
140
126
|
'detector' => 'mecab-ipadic')
|
|
141
127
|
end
|
|
@@ -157,6 +143,25 @@ module AsciidocPubkit
|
|
|
157
143
|
finish
|
|
158
144
|
end
|
|
159
145
|
|
|
146
|
+
def self.scan_literal_terms(findings, paragraph, text, terms, settings, morphological:)
|
|
147
|
+
candidates = []
|
|
148
|
+
terms.each do |rule, (surfaces, question)|
|
|
149
|
+
next if morphological && !%w[generic-framing contextual-phrase].include?(rule)
|
|
150
|
+
surfaces.each do |surface|
|
|
151
|
+
text.to_enum(:scan, Regexp.new(Regexp.escape(surface))).each do
|
|
152
|
+
match = Regexp.last_match
|
|
153
|
+
candidates << [match.begin(0), match.end(0), surface, rule, question]
|
|
154
|
+
end
|
|
155
|
+
end
|
|
156
|
+
end
|
|
157
|
+
# Keep the most specific phrase, including when it is explicitly allowed.
|
|
158
|
+
candidates.each do |start, finish, surface, rule, question|
|
|
159
|
+
next if candidates.any? { |left, right, *_| left <= start && right >= finish && right - left > finish - start }
|
|
160
|
+
next if settings.fetch('allows').include?(surface)
|
|
161
|
+
add(findings, paragraph, text, start, surface, rule, 'hint', question)
|
|
162
|
+
end
|
|
163
|
+
end
|
|
164
|
+
|
|
160
165
|
def self.find_term(findings, paragraph, text, term, rule, severity, question)
|
|
161
166
|
text.to_enum(:scan, Regexp.new(Regexp.escape(term))).each do
|
|
162
167
|
match = Regexp.last_match
|