commenter 0.3.0 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 99f441f58237864e8ea0bc5c8682f04d2b15757ef1f28c45317f0181d6621fb4
-  data.tar.gz: 0a48406aa86377612a2722455dad5013457a10805da311c74c1fbfb82cccafd7
+  metadata.gz: 1609bb9eed80e408daebc9c67cda2c657e8484186acce89b823e6b411a5f9ea5
+  data.tar.gz: 46316674e2c639b9256390385d2cb3c59ed8946b604b44fbb0be28728475cd38
 SHA512:
-  metadata.gz: 7c26ede9e2a9e4f5f060c0b8679cc9afcb36214a4c5b1a2f02c15a44cd6671d7b104954f4cf4b6df3df9f3806ab7c4d0418982a75371805c773c90064b82c8b9
-  data.tar.gz: 32dd2e31a603391bef5a112d0b10da62b7c29e249bcc76c3e5456aab939015ff51d21036108c3366743db11e01411a2bd285b7992f1a534009d5d4cc184e9d31
+  metadata.gz: 887aa19ab8ca3559fd20caf3e19a9941b631c677faeda86553597671883fc8dbae8c1e1f6d83bd0014e036774743addc16bc538b7404f6939be6636e1d623704
+  data.tar.gz: 3b850d0872f2f572b599d0bf7a70a977e1e5e7e797f5fb3447905c09cc7578cff18d99de6b32ae3b539758b7319b07590fdfd9d5249f74024ffdcd842d48d09d

data/.rubocop.yml

@@ -12,3 +12,7 @@ Style/StringLiteralsInInterpolation:
 Metrics/ModuleLength:
   Exclude:
     - "spec/**/*"
+
+Metrics/BlockLength:
+  Exclude:
+    - "spec/**/*"

data/CLAUDE.md

@@ -18,13 +18,23 @@ bin/console                      # interactive prompt
 
 CI (metanorma/ci `generic-rake.yml`) runs `bundle exec rake` on Ruby 3.2/3.4/4.0 across macOS/Ubuntu/Windows. RuboCop offenses fail CI — run `bundle exec rake`, not just rspec, before pushing. `Gemfile.lock` is gitignored; dependencies resolve fresh in CI.
 
-Specs generate XLSX fixtures at runtime via `spec/support/xlsx_builder.rb` (plain rubyzip, no spreadsheet-writing dependency) plus row data in `spec/support/osd_fixtures.rb` — there are no binary fixture files in the repo.
+Specs generate XLSX fixtures at runtime via `spec/support/xlsx_builder.rb` (plain rubyzip, no spreadsheet-writing dependency) plus row data in `spec/support/osd_fixtures.rb` — there are no binary fixture files in the repo. Redline DOCX fixtures are built at runtime via `spec/support/redline_docx_builder.rb` (same approach).
+
+## Release
+
+NEVER bump `lib/commenter/version.rb` by hand and never push tags. Releases go through the GHA workflow, which does the version bump, `v*` tag, and gem push itself:
+
+```sh
+gh workflow run release.yml --ref main -f next_version=X.Y.Z
+```
+
+The version number is the maintainer's decision — ask, never infer it from semver reasoning. PRs are rebase-merged.
 
 ## What this gem does
 
 Converts ISO comment sheets to structured YAML and back, and syncs comments to GitHub issues:
 
-- **Import**: DOCX (ISO 2012-03 balloting template) or XLSX (ISO Online Standards Development exports) → YAML + schema file. Format auto-detected from extension.
+- **Import**: DOCX (ISO 2012-03 balloting template) or XLSX (ISO Online Standards Development exports) → YAML + schema file. Format auto-detected from extension. Redline DOCX with tracked changes is dispatched to `Parser::TrackChangeDocxParser` via `--format redline`.
 - **Fill**: YAML → filled DOCX comment sheet (`data/iso_comment_template_2012-03.docx`), optional status-based cell shading.
 - **GitHub round-trip**: `github-create` makes issues from YAML via Liquid templates; `github-retrieve` pulls `> **OBSERVATION:**` blockquotes from closed issues back into the YAML's observations field.
 
@@ -34,11 +44,12 @@ Plain text only — formulas/images/complex formatting are unsupported (docx gem
 
 Core flow: `Parser` → `CommentSheet` (metadata + `Comment` objects) → YAML / DOCX / GitHub issues.
 
-- `Commenter::Comment` / `Commenter::CommentSheet` — data model. `Comment` carries the common fields plus optional OSD-specific fields (`user_name`, `resolution_status`, `motivation`, etc.) and a `github` sub-hash tracking issue state. The sheet's `version` field (`"2012-03"` vs `"osd"`) selects the output schema.
-- `Commenter::Parser` — dispatches by format: `.docx` parsed inline with the `docx` gem; `.xlsx` delegated to `Parser::OsdXlsxParser` (uses `roo`).
+- `Commenter::Comment` / `Commenter::CommentSheet` — data model. `Comment` carries the common fields plus optional OSD-specific fields (`user_name`, `resolution_status`, `motivation`, etc.) and a `github` sub-hash tracking issue state. Comment types: short codes (`ge`/`te`/`ed`) are expanded to full names (`general`/`technical`/`editorial`) when a `Comment` is loaded; YAML output stores the expanded form. The sheet's `version` field (`"2012-03"` vs `"osd"`) selects the output schema.
+- `Commenter::Parser` — dispatches by format: `.docx` parsed inline with the `docx` gem; `.xlsx` delegated to `Parser::OsdXlsxParser` (uses `roo`); `redline` delegated to `Parser::TrackChangeDocxParser`.
 - `Parser::OsdXlsxParser` — auto-detects two OSD export variants by header row ("resolved" 17-col starting `Comment ID`; "unresolved" 15-col starting `User name`), extracts metadata (date/reference/stage/titles) from header rows 1–2, maps columns by header name into `Comment` attributes, and synthesizes `observations` from `resolution_status` + `motivation`.
-- `Commenter::Filler` — writes comments into the DOCX template table.
-- `Commenter::GitHubIssueCreator` (`lib/commenter/github_integration.rb`) — Octokit-based; duplicate detection searches for a unique ID rendered from a configurable Liquid `unique_id` template (stage-aware by default: `[DIS] GB-001`), so the same comment ID at different ballot stages creates separate issues.
+- `Parser::TrackChangeDocxParser` — redline DOCX import: streams `word/document.xml` with Nokogiri XML::Reader (redlines can exceed 100 MB), captures `w:ins`/`w:del`/`w:moveFrom`/`w:moveTo` as comment entries (`proposed_change` renders the change itself, e.g. `Insert: "text"`; clause resolved from the nearest preceding heading including sub-clauses; element resolved from Table/Figure/Formula/NOTE references in the same clause; self-closing paragraph-mark markers skipped), plus reviewer `w:comment` threads as `-CNNN` remark entries (remark verbatim in `comments`, instruction reworded in `proposed_change`, empty `observations` for the owner). Stamps `observations` from the `observations`/`accept_all` options.
+- `Commenter::Filler` — writes comments into the DOCX template table; maps observation status text (`accept(ed)?`, `noted`, `reject(ed)?`, …) to shading colors.
+- GitHub integration (`lib/commenter/github_integration.rb`) — two classes with parallel structure: `GitHubIssueCreator` (`github-create`) and `GitHubIssueRetriever` (`github-retrieve`). Each loads its own config YAML and Octokit client, and rewrites the comments YAML in place after the run. Duplicate detection searches issue titles for a unique ID rendered from a configurable Liquid `unique_id` template (stage-aware by default: `[DIS] GB-001`), so the same comment ID at different ballot stages creates separate issues.
 - `Commenter::Cli` (`lib/commenter/cli.rb`) — Thor CLI with subcommands `import`, `fill`, `github-create`, `github-retrieve`.
 
 ### Schemas
@@ -55,4 +66,3 @@ Core flow: `Parser` → `CommentSheet` (metadata + `Comment` objects) → YAML /
 - `data/` files (DOCX template, Liquid templates, sample config) are shipped with the gem and loaded relative to `__dir__` — do not treat them as disposable.
 - Testing GitHub integration: use `--dry-run` with `GITHUB_TOKEN=dummy_token` — exercises template rendering without API calls.
 - `sig/commenter.rbs` is a minimal stub; RBS coverage is not currently maintained.
-- Release: pushing a `v*` tag triggers `.github/workflows/release.yml`. Version lives in `lib/commenter/version.rb`.

data/README.adoc

@@ -19,6 +19,8 @@ The supported input formats are:
   (DOCX)
 * ISO Online Standards Development (OSD) XLSX exports (resolved and unresolved
   comment sheets)
+* Redlined Word documents containing tracked changes (e.g. ISO/CS editor
+  redlines), imported with `--format redline`
 
 This gem only supports plain text comment extraction and filling. Only use this
 to handle plain text comments and resolutions.
@@ -56,7 +58,9 @@ $ gem install commenter
 === Importing comments
 
 Convert an ISO comment sheet (DOCX or XLSX) to structured YAML.
-The format is auto-detected from the file extension.
+The format is auto-detected from the file extension; redline DOCX must be
+selected explicitly with `--format redline` because the extension cannot
+distinguish a redline from a comment sheet.
 
 [source,shell]
 ----
@@ -68,8 +72,28 @@ $ commenter import "91855-Comments-resolved.xlsx" -o comments.yaml
 
 # From XLSX (ISO OSD comments-only export)
 $ commenter import "ISO 5843-6-Comments.xlsx" -o comments.yaml
+
+# From a redlined DOCX with tracked changes (ISO/CS editor redline)
+$ commenter import "ISO_DIS 2533 Enquiry Trackchange Word.docx" --format redline \
+    --body CS --document "ISO 2533:2026" --stage DIS \
+    --observations "Accepted. ISO/CS tracked change accepted." \
+    -o comments-cs.yaml
 ----
 
+Each tracked change (`w:ins`, `w:del`, `w:moveFrom`, `w:moveTo`) becomes a
+comment entry whose `proposed_change` renders the change itself
+(`Insert: "..."` / `Delete: "..."` / `Move ...: "..."`) and whose
+`locality.clause` is resolved from the nearest preceding heading (including
+sub-clauses such as `4.2.1` and `Annex A`), with `locality.element` resolved
+from the nearest `Table`/`Figure`/`Formula`/`NOTE` reference in the same
+clause. Self-closing paragraph-mark markers are skipped because they carry
+no content.
+
+Reviewer comment threads (`w:comment`) are emitted after the track changes
+with `-CNNN` ids: the remark verbatim in `comments`, its instruction
+reworded as the `proposed_change` (a leading "Please " is dropped), and
+empty `observations` for the document owner to draft.
+
 This will create two files:
 
 `comments.yaml`:: The structured comment data
@@ -87,10 +111,15 @@ Options:
 `-o, --output FILE`:: Output YAML file (default: comments.yaml)
 `-e, --exclude-observations`:: Skip the observations column
 `--schema-dir DIR`:: Directory for schema file (default: schema)
-`--format FORMAT`:: Force input format (`docx` or `xlsx`; auto-detected by default)
-`--sheet NAME`:: XLSX sheet name to parse (default: first sheet)
+`--format FORMAT`:: Force input format (`docx`, `xlsx` or `redline`; auto-detected by default)
+`--sheet NAME`:: XLSX: sheet name to parse (default: first sheet)
 `--resolved-only`:: XLSX: use resolved comments sheet only
 `--unresolved-only`:: XLSX: use unresolved comments sheet only
+`--body CODE`:: Redline: member body code for comment IDs (default: `CS`)
+`--observations TEXT`:: Redline: observations stamped on all track changes
+`--accept-all`:: Redline: stamp `Accepted. Tracked change accepted.` on all track changes (an explicit `--observations` wins)
+`--document ID`:: Redline: document identifier (e.g. `ISO 2533:2026`)
+`--stage STAGE`:: Redline: approval stage (e.g. `DIS`)
 
 ==== Metadata (extraction limitation)
 
@@ -134,7 +163,7 @@ comments:
       line_number:
       clause: "_whole document"
       element:
-    type: ge
+    type: general
     comments: |
       The document should include more examples
       to clarify the implementation requirements.
@@ -150,7 +179,7 @@ comments:
       line_number: "45"
       clause: "5.2.1"
       element: "Table 3"
-    type: te
+    type: technical
     comments: |
       The values in Table 3 appear to be
       inconsistent with the formula in 5.1.
@@ -471,7 +500,7 @@ comments:
     locality:
       clause: "5.2.1"
       element: "Table 3"
-    type: te
+    type: technical
     comments: "The values in Table 3 appear inconsistent..."
     proposed_change: "Correct the values in column 2..."
     observations: ""

data/lib/commenter/cli.rb

@@ -3,20 +3,23 @@
 require "thor"
 require "yaml"
 require "fileutils"
-require "commenter/parser"
-require "commenter/filler"
-require "commenter/github_integration"
+require "commenter"
 
 module Commenter
   class Cli < Thor
-    desc "import INPUT", "Convert comment sheet (DOCX or XLSX) to YAML"
+    desc "import INPUT", "Convert comment sheet (DOCX or XLSX) or redline DOCX to YAML"
     option :output, type: :string, aliases: :o, default: "comments.yaml", desc: "Output YAML file"
     option :exclude_observations, type: :boolean, aliases: :e, desc: "Exclude observations column"
     option :schema_dir, type: :string, default: "schema", desc: "Directory for schema file"
-    option :format, type: :string, desc: "Force input format (docx or xlsx)"
+    option :format, type: :string, desc: "Force input format (docx, xlsx or redline)"
     option :sheet, type: :string, desc: "XLSX sheet name to parse (default: first sheet)"
     option :resolved_only, type: :boolean, desc: "XLSX: use resolved comments sheet only"
     option :unresolved_only, type: :boolean, desc: "XLSX: use unresolved comments sheet only"
+    option :body, type: :string, desc: "Redline: member body code for comment IDs (default: CS)"
+    option :observations, type: :string, desc: "Redline: observations stamped on all track changes"
+    option :accept_all, type: :boolean, desc: "Redline: stamp 'Accepted. Tracked change accepted.' on all track changes"
+    option :document, type: :string, desc: "Redline: document identifier (e.g. ISO 2533:2026)"
+    option :stage, type: :string, desc: "Redline: approval stage (e.g. DIS)"
     def import(input_file)
       output_yaml = options[:output]
       schema_dir = options[:schema_dir]
@@ -28,14 +31,11 @@ module Commenter
       parser = Parser.new
       comment_sheet = parser.parse(input_file, options)
 
-      # Determine which schema to use based on version
-      schema_name = comment_sheet.version == "osd" ? "iso_comment_osd.yaml" : "iso_comment_2012-03.yaml"
-
       # Write the YAML data file with schema reference
-      yaml_content = generate_yaml_with_header(comment_sheet.to_yaml_h, schema_dir, schema_name)
-      File.write(output_yaml, yaml_content)
+      File.write(output_yaml, comment_sheet.to_yaml_document(schema_dir))
 
       # Copy schema file to output directory
+      schema_name = comment_sheet.schema_name
       schema_source = File.join(__dir__, "../../schema/#{schema_name}")
       schema_target = File.join(schema_dir, schema_name)
 
@@ -212,13 +212,5 @@ module Commenter
     def self.exit_on_failure?
       true
     end
-
-    private
-
-    def generate_yaml_with_header(data, schema_dir, schema_name = "iso_comment_2012-03.yaml")
-      schema_path = File.join(schema_dir, schema_name)
-      header = "# yaml-language-server: $schema=#{schema_path}\n\n"
-      header + data.to_yaml
-    end
   end
 end

data/lib/commenter/comment.rb

@@ -13,7 +13,7 @@ module Commenter
       @id = attrs[:id]
       @body = attrs[:body]
       @locality = symbolize_keys(attrs[:locality] || {})
-      @type = expand_comment_type(attrs[:type])
+      @type = CommentType.full_name(attrs[:type])
       @comments = attrs[:comments]
       @proposed_change = attrs[:proposed_change]
       @observations = attrs[:observations]
@@ -30,16 +30,8 @@ module Commenter
       @stage_code = attrs[:stage_code]
     end
 
-    # Expands short type codes to full names. One-way by design: already
-    # expanded values pass through unchanged so that reloading YAML output
-    # stays stable.
     def expand_comment_type(type)
-      case type&.downcase
-      when "ge" then "general"
-      when "te" then "technical"
-      when "ed" then "editorial"
-      else type
-      end
+      CommentType.full_name(type)
     end
 
     def line_number

data/lib/commenter/comment_sheet.rb

@@ -45,6 +45,14 @@ module Commenter
       stringify_keys(hash)
     end
 
+    def schema_name
+      version == "osd" ? "iso_comment_osd.yaml" : "iso_comment_2012-03.yaml"
+    end
+
+    def to_yaml_document(schema_dir = "schema")
+      "# yaml-language-server: $schema=#{File.join(schema_dir.to_s, schema_name)}\n\n#{to_yaml_h.to_yaml}"
+    end
+
     def self.from_hash(hash)
       new(hash)
     end

data/lib/commenter/comment_type.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+module Commenter
+  # Single home for the comment type vocabulary: short codes (ge/te/ed) and
+  # their expanded names (general/technical/editorial). Unrecognized values
+  # pass through unchanged; display_name capitalizes recognized types.
+  module CommentType
+    FULL_NAMES = {
+      "ge" => "general",
+      "te" => "technical",
+      "ed" => "editorial"
+    }.freeze
+
+    CODES = FULL_NAMES.invert.freeze
+
+    DISPLAY_NAMES = FULL_NAMES.each_with_object({}) do |(code, full), display|
+      display[code] = full.capitalize
+      display[full] = full.capitalize
+    end.freeze
+
+    module_function
+
+    def code(type)
+      CODES.fetch(type.to_s.strip.downcase) { type&.to_s&.strip }
+    end
+
+    def full_name(type)
+      FULL_NAMES.fetch(type.to_s.strip.downcase) { type&.to_s&.strip }
+    end
+
+    def display_name(type)
+      DISPLAY_NAMES.fetch(type.to_s.strip.downcase) { type || "Unknown" }
+    end
+  end
+end

data/lib/commenter/github_integration.rb

@@ -8,11 +8,10 @@ require "dotenv/load"
 module Commenter
   class GitHubIssueCreator
     def initialize(config_path, title_template_path = nil, body_template_path = nil)
-      @config = load_config(config_path)
-      @github_client = create_github_client
-      @repo = @config.dig("github", "repository")
-
-      raise "GitHub repository not specified in config" unless @repo
+      @session = GitHubSession.new(config_path)
+      @config = @session.config
+      @github_client = @session.client
+      @repo = @session.repo
 
       @title_template = load_liquid_template(title_template_path || default_title_template_path)
       @body_template = load_liquid_template(body_template_path || default_body_template_path)
@@ -43,19 +42,6 @@ module Commenter
 
     private
 
-    def load_config(config_path)
-      YAML.load_file(config_path)
-    rescue Errno::ENOENT
-      raise "Configuration file not found: #{config_path}"
-    end
-
-    def create_github_client
-      token = @config.dig("github", "token") || ENV["GITHUB_TOKEN"]
-      raise "GitHub token not found. Set GITHUB_TOKEN environment variable or specify in config file." unless token
-
-      Octokit::Client.new(access_token: token)
-    end
-
     def default_title_template_path
       File.join(__dir__, "../../data/github_issue_title_template.liquid")
     end
@@ -103,7 +89,7 @@ module Commenter
         "comment_id" => comment.id || "",
         "body" => comment.body || "",
         "type" => comment.type || "",
-        "type_full_name" => expand_comment_type(comment.type),
+        "type_full_name" => CommentType.display_name(comment.type),
         "comments" => comment.comments || "",
         "proposed_change" => comment.proposed_change || "",
         "observations" => comment.observations || "",
@@ -134,15 +120,6 @@ module Commenter
       @unique_id_template.render(base_variables).strip
     end
 
-    def expand_comment_type(type)
-      case type&.downcase
-      when "ge", "general" then "General"
-      when "te", "technical" then "Technical"
-      when "ed", "editorial" then "Editorial"
-      else type || "Unknown"
-      end
-    end
-
     def format_locality(comment)
       parts = []
       parts << "Clause #{comment.clause}" if comment.clause && !comment.clause.strip.empty?
@@ -314,23 +291,16 @@ module Commenter
 
       # Write updated YAML
       output_file = options[:output] || yaml_file
-      yaml_content = generate_yaml_with_header(comment_sheet.to_yaml_h)
-      File.write(output_file, yaml_content)
-    end
-
-    def generate_yaml_with_header(data)
-      header = "# yaml-language-server: $schema=schema/iso_comment_2012-03.yaml\n\n"
-      header + data.to_yaml
+      File.write(output_file, comment_sheet.to_yaml_document)
     end
   end
 
   class GitHubIssueRetriever
     def initialize(config_path)
-      @config = load_config(config_path)
-      @github_client = create_github_client
-      @repo = @config.dig("github", "repository")
-
-      raise "GitHub repository not specified in config" unless @repo
+      @session = GitHubSession.new(config_path)
+      @config = @session.config
+      @github_client = @session.client
+      @repo = @session.repo
     end
 
     def retrieve_observations_from_yaml(yaml_file, options = {})
@@ -357,19 +327,6 @@ module Commenter
 
     private
 
-    def load_config(config_path)
-      YAML.load_file(config_path)
-    rescue Errno::ENOENT
-      raise "Configuration file not found: #{config_path}"
-    end
-
-    def create_github_client
-      token = @config.dig("github", "token") || ENV["GITHUB_TOKEN"]
-      raise "GitHub token not found. Set GITHUB_TOKEN environment variable or specify in config file." unless token
-
-      Octokit::Client.new(access_token: token)
-    end
-
     def retrieve_observation(comment, options)
       issue_number = comment.github_issue_number
 
@@ -486,13 +443,7 @@ module Commenter
 
     def update_yaml_with_observations(yaml_file, comment_sheet, options)
       output_file = options[:output] || yaml_file
-      yaml_content = generate_yaml_with_header(comment_sheet.to_yaml_h)
-      File.write(output_file, yaml_content)
-    end
-
-    def generate_yaml_with_header(data)
-      header = "# yaml-language-server: $schema=schema/iso_comment_2012-03.yaml\n\n"
-      header + data.to_yaml
+      File.write(output_file, comment_sheet.to_yaml_document)
     end
   end
 end

data/lib/commenter/github_session.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+require "octokit"
+require "yaml"
+
+module Commenter
+  # Shared connection setup for the GitHub integration commands: loads the
+  # configuration file, builds the Octokit client, and resolves the target
+  # repository. One constructor, one test surface.
+  class GitHubSession
+    attr_reader :config, :client, :repo
+
+    def initialize(config_path)
+      @config = load_config(config_path)
+      @client = Octokit::Client.new(access_token: required_token)
+      @repo = @config.dig("github", "repository")
+      raise "GitHub repository not specified in config" unless @repo
+    end
+
+    private
+
+    def load_config(config_path)
+      YAML.load_file(config_path)
+    rescue Errno::ENOENT
+      raise "Configuration file not found: #{config_path}"
+    end
+
+    def required_token
+      token = @config.dig("github", "token") || ENV["GITHUB_TOKEN"]
+      raise "GitHub token not found. Set GITHUB_TOKEN environment variable or specify in config file." unless token
+
+      token
+    end
+  end
+end

data/lib/commenter/parser/osd_xlsx_parser.rb

@@ -201,7 +201,7 @@ module Commenter
             clause: clause.empty? ? nil : clause,
             element: clause_title.empty? ? nil : clause_title
           }.compact,
-          type: normalize_comment_type(cell_value_str(row, col_map[:comment_type]).to_s),
+          type: CommentType.code(cell_value_str(row, col_map[:comment_type]).to_s),
           comments: comment_text.to_s.strip,
           proposed_change: cell_value_str(row, col_map[:proposed_change]) ||
             cell_value_str(row, col_map[:proposal_on_text])
@@ -227,15 +227,6 @@ module Commenter
         parts.empty? ? nil : parts.join(". ")
       end
 
-      def normalize_comment_type(type_str)
-        case type_str.strip.downcase
-        when "editorial", "ed" then "ed"
-        when "technical", "te" then "te"
-        when "general", "ge" then "ge"
-        else type_str.strip
-        end
-      end
-
       def cell_value(row, index)
         return nil if index.nil? || row.length <= index