commenter 0.6.1 → 0.7.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 116bd5694762b20a730aa2d67d9950e3acedecf5e0e5d2f95a6d183d376c0e97
-  data.tar.gz: d1c34482394b44f5195a91268eb3b46f4db58c986afc214a841d74b908f98edb
+  metadata.gz: dbcfbae1636268a1f6577337f999f47271e93b7943fd6e9a28a3ddb93603ad04
+  data.tar.gz: dcca1817d391d94ce98fbb1469499fd480944cac137ea86ef9c468f1873347fd
 SHA512:
-  metadata.gz: 1d755c6bfefe119528e4ddb9e65033263f5c03550d972ae29bd62aaaea8df3fa571439305f80429eefb017856d387e5dde23c18a744ecdfb03d2f77d6a6e486e
-  data.tar.gz: 6d297683a681dc1918be0db3a5dd63aded21fca0fce0a63799b36e1342d60f927b9690ca9757c8935c74c530edc3205acb7b510e4930360a7f4a99d595a865fc
+  metadata.gz: 0fc597c5fe0c6d0f9db6366569d20cad1c93a78170187d3d9a67ba0e8911ba64b355938d11a40fe752b6ccd79ea5e3d7f702af5b61c1848a996962df3d57b1e9
+  data.tar.gz: 50e641728f5d07c496388fdc073c2a659ff212f21e8d0d23c44de7c449270052f7bc56dba427bd35f1a68125151d62c61b95a913e7e1aad9e9f4ba4100aebf37

data/CLAUDE.md

@@ -50,7 +50,7 @@ Core flow: `Parser` → `CommentSheet` (metadata + `Comment` objects) → YAML /
 - `Parser::TrackChangeDocxParser` — redline DOCX import: streams `word/document.xml` with Nokogiri XML::Reader (redlines can exceed 100 MB), captures `w:ins`/`w:del`/`w:moveFrom`/`w:moveTo` as comment entries (`proposed_change` renders the change itself, e.g. `Insert: "text"`; clause resolved from the nearest preceding heading including sub-clauses; element resolved from Table/Figure/Formula/NOTE references in the same clause; self-closing paragraph-mark markers skipped), plus reviewer `w:comment` threads as `-CNNN` remark entries (remark verbatim in `comments`, instruction reworded in `proposed_change`, empty `observations` for the owner). Stamps `observations` from the `observations`/`accept_all` options.
 - `Commenter::Filler` — writes comments into the DOCX template table; maps observation status text (`accept(ed)?`, `noted`, `reject(ed)?`, …) to shading colors.
 - GitHub integration — `GitHubIssueCreator` (`github-create`, `lib/commenter/github_integration.rb`), `GitHubIssueRetriever` (`github-retrieve`, `lib/commenter/github_issue_retriever.rb`), and `GitHubSync` (`github-sync`, subclass of the creator). Each loads its own config YAML and Octokit client, and rewrites the comments YAML in place after the run. Duplicate detection verifies issue numbers already recorded in the YAML, then searches issue bodies for a deterministic `urn:commenter:` marker appended to every created issue, then falls back to a title search over a configurable Liquid `unique_id` template (stage-aware by default: `[DIS] GB-001`). The same comment ID at different ballot stages creates separate issues. github-retrieve fills observations from open and closed issues alike.
-- `Commenter::Cli` (`lib/commenter/cli.rb`) — Thor CLI with subcommands `import`, `merge`, `stats`, `diff`, `fill`, `export`, `github-create`, `github-sync`, `github-retrieve`.
+- `Commenter::Cli` (`lib/commenter/cli.rb`) — Thor CLI with subcommands `import`, `merge`, `stats`, `diff`, `validate`, `fill`, `export`, `github-create`, `github-sync`, `github-retrieve`.
 
 ### Schemas
 

data/README.adoc

@@ -234,6 +234,23 @@ issue counts as *open*; no disposition at all counts as *undecided*). Use
 `--format yaml` for machine-readable output, and `-o report.md` to write to a
 file — the Markdown pastes directly into a GitHub issue or WG minutes.
 
+=== Validating a comment sheet
+
+The shipped schemas power IDE validation; `commenter validate` checks the
+same structure from the command line — unknown versions, missing ids or
+comment text, duplicate ids, and unrecognized comment types — and exits
+non-zero when errors are found, so it fits CI and pre-flight checks:
+
+[source,shell]
+----
+commenter validate ballot.yaml
+----
+
+Known stages are AWI, WD, CD, DIS, FDIS, PRF, and PUB. A sheet at an
+unusual stage (or with an unrecognized comment type) produces a WARNING,
+which is informational only — a non-zero exit always means errors were
+found.
+
 === Comparing ballots between stages
 
 The same comment ID (`DE-001`) recurs at every ballot stage; comparing two
@@ -319,6 +336,8 @@ github:
   token: "ghp_xxxxxxxxxxxx"
   milestone:
     name: "ISO 80000-2 DIS Review"
+    # Creates the milestone when it does not exist yet
+    create_if_missing: true
   default_labels: ["comment-review", "iso-standard"]
   stage_labels:
     WD: ["working-draft"]
@@ -338,7 +357,7 @@ templates are provided, but you can customize them.
 
 The title template (`data/github_issue_title_template.liquid`) supports these variables:
 
-* `stage` - Approval stage (WD/CD/DIS/FDIS/PRF/PUB)
+* `stage` - Approval stage (AWI/WD/CD/DIS/FDIS/PRF/PUB)
 * `document` - Document identifier
 * `comment_id` - Comment identifier
 * `brief_summary` - Generated summary combining locality and description
@@ -435,13 +454,22 @@ Every created issue body also carries a deterministic marker line, e.g.
 +
 [source]
 ----
-`urn:commenter:owner/repo:dis:GB-001`
+`urn:commenter:owner/repo:dis:iso1234-2:2019:gb-001`
 ----
 +
-Duplicate detection searches the issue bodies for this marker first — exact
-and template-independent — and falls back to the title search for issues
-created before the marker existed. Comment numbers already recorded in the
-comments YAML are verified with a direct read before any search runs.
+The marker includes the sheet's document (compacted and lowercased), so
+the same comment ID (`GB-001`) posted for two different documents of one
+repository never collides. Sheets without a `document` field keep the
+older, document-less marker shape.
+
+Duplicate detection searches the issue bodies for this marker first —
+exact and template-independent. It then falls back to the document-less
+marker for issues created before the document segment existed, and to the
+title search for issues created before markers existed at all; those two
+fallbacks are ambiguous when the same comment ID exists for several
+documents, so a fallback hit counts only when the issue itself names the
+sheet's document. Comment numbers already recorded in the comments YAML
+are verified with a direct read before any search runs.
 
 ===== Unique ID template
 
@@ -530,7 +558,10 @@ By default an open issue whose comment has a recorded disposition
 `github-retrieve` round-trip working, since observations are retrieved from
 closed issues. Disable with `--no-close-on-disposition`.
 
-The milestone is resolved once per run and reused for every issue.
+The milestone is resolved once per run and reused for every issue. When
+`create_if_missing` is set and no milestone carries that name, the
+milestone is created on first use (honoring `description` and `due_date`
+from the same config block).
 
 ==== Retrieving observations from GitHub issues
 

data/lib/commenter/cli.rb

@@ -86,6 +86,30 @@ module Commenter
       end
     end
 
+    desc "validate INPUT.yaml", "Check a comment sheet for structural problems"
+    def validate(input_yaml)
+      comment_sheet = CommentSheet.from_yaml(File.read(input_yaml))
+      problems = SheetValidator.call(comment_sheet)
+
+      problems.each do |problem|
+        where = problem[:comment] ? "#{problem[:comment]}: " : ""
+        puts "#{problem[:severity].to_s.upcase}: #{where}#{problem[:message]}"
+      end
+
+      errors = problems.count { |problem| problem[:severity] == :error }
+      warnings = problems.count { |problem| problem[:severity] == :warning }
+
+      # Warnings surface unusual-but-legal sheets (e.g. an early-stage draft)
+      # and must not fail CI on their own; only errors are fatal.
+      raise Commenter::Error, "#{errors} error(s), #{warnings} warning(s) found in #{input_yaml}" if errors.positive?
+
+      if problems.empty?
+        puts "OK: #{comment_sheet.comments.length} comments, no problems found"
+      else
+        puts "#{comment_sheet.comments.length} comments valid, #{warnings} warning(s)"
+      end
+    end
+
     desc "fill INPUT.yaml", "Fill DOCX template from YAML comments"
     option :output, type: :string, aliases: :o, default: "filled_comments.docx", desc: "Output DOCX file"
     option :template, type: :string, aliases: :t, desc: "Custom template file"

data/lib/commenter/comment.rb

@@ -100,9 +100,21 @@ module Commenter
       locality.element = value
     end
 
+    # Public so every consumer (summaries, issue templates) renders locality
+    # through one implementation.
+    def locality_summary
+      parts = []
+      if clause && !clause.strip.empty?
+        parts << (clause.strip.match?(/\Aclause\b/i) ? clause : "Clause #{clause}")
+      end
+      parts << element if element && !element.strip.empty?
+      parts << "Line #{line_number}" if line_number && !line_number.strip.empty?
+      parts.join(", ")
+    end
+
     def brief_summary(max_length = 60)
       description = comments.to_s.split(/[.!?\n]/).map(&:strip).reject(&:empty?).first.to_s
-      locality_text = format_locality
+      locality_text = locality_summary
 
       if description.empty?
         locality_text.empty? ? "No description" : locality_text
@@ -143,15 +155,5 @@ module Commenter
       self.github = GithubInfo.new(issue_number: issue_number, issue_url: issue_url,
                                    status: status, created_at: created_at)
     end
-
-    private
-
-    def format_locality
-      parts = []
-      parts << "Clause #{clause}" if clause && !clause.strip.empty?
-      parts << element if element && !element.strip.empty?
-      parts << "Line #{line_number}" if line_number && !line_number.strip.empty?
-      parts.join(", ")
-    end
   end
 end

data/lib/commenter/github_integration.rb

@@ -10,7 +10,6 @@ module Commenter
     THROTTLE_SECONDS = 1
     RATE_LIMIT_RETRY_DELAY = 60
     RATE_LIMIT_MAX_RETRIES = 3
-    MARKER_PREFIX = "urn:commenter"
 
     def initialize(config_path, title_template_path = nil, body_template_path = nil, client: nil)
       @session = GitHubSession.new(config_path)
@@ -108,7 +107,7 @@ module Commenter
         "unique_id" => unique_id,
         "has_observations" => !comment.observations.nil? && !comment.observations.strip.empty?,
         "has_proposed_change" => !comment.proposed_change.nil? && !comment.proposed_change.strip.empty?,
-        "locality_summary" => format_locality(comment)
+        "locality_summary" => comment.locality_summary
       }
     end
 
@@ -124,14 +123,6 @@ module Commenter
       @unique_id_template.render(base_variables).strip
     end
 
-    def format_locality(comment)
-      parts = []
-      parts << "Clause #{comment.clause}" if comment.clause && !comment.clause.strip.empty?
-      parts << comment.element if comment.element && !comment.element.strip.empty?
-      parts << "Line #{comment.line_number}" if comment.line_number && !comment.line_number.strip.empty?
-      parts.join(", ")
-    end
-
     def create_issue(comment, comment_sheet, options = {})
       puts "[GitHubIssueCreator] Creating issue for comment ID: #{comment.id}"
       begin
@@ -181,16 +172,9 @@ module Commenter
       end
     end
 
-    # Deterministic marker embedded in every created issue body, independent
-    # of the user's templates, so duplicate detection never has to rely on
-    # title matching alone (#1).
-    def issue_marker(comment, comment_sheet)
-      "#{MARKER_PREFIX}:#{@repo.downcase}:#{comment_sheet.stage.to_s.downcase}:#{comment.id}"
-    end
-
     def body_with_marker(comment, comment_sheet)
       rendered = @body_template.render(template_variables(comment, comment_sheet))
-      "#{rendered}\n\n---\n`#{issue_marker(comment, comment_sheet)}`"
+      "#{rendered}\n\n---\n`#{IssueMarker.for(comment, comment_sheet, @repo)}`"
     end
 
     def recorded_issue(comment, comment_sheet)
@@ -200,14 +184,14 @@ module Commenter
       puts "[GitHubIssueCreator] Verifying recorded issue ##{number} for comment ID: #{comment.id}"
       issue = @github_client.issue(@repo, number)
       if issue.title.include?(render_unique_id(comment, comment_sheet)) ||
-         issue.body.to_s.include?(issue_marker(comment, comment_sheet))
+         issue.body.to_s.include?(IssueMarker.for(comment, comment_sheet, @repo))
         issue
       else
-        puts "[GitHubIssueCreator] Recorded issue ##{number} does not match, falling back to title search."
+        puts "[GitHubIssueCreator] Recorded issue ##{number} does not match, falling back to search."
         nil
       end
     rescue Octokit::NotFound
-      puts "[GitHubIssueCreator] Recorded issue ##{number} no longer exists, falling back to title search."
+      puts "[GitHubIssueCreator] Recorded issue ##{number} no longer exists, falling back to search."
       nil
     end
 
@@ -257,19 +241,29 @@ module Commenter
       }
     end
 
+    # Duplicate detection runs in three tiers: the document-qualified marker
+    # is exact; the legacy marker and the title match are ambiguous when two
+    # documents share a comment id, so those hits count only when the issue
+    # itself names this document.
     def find_existing_issue(comment, comment_sheet)
       unique_id = render_unique_id(comment, comment_sheet)
       puts "[GitHubIssueCreator] Searching for existing issue with unique_id: #{unique_id}"
 
-      # The marker search is exact and template-independent (#1); the title
-      # search remains as a fallback for issues created before the marker.
-      marker = issue_marker(comment, comment_sheet)
-      marker_results = @github_client.search_issues("repo:#{@repo} in:body \"#{marker}\"")
-      return marker_results.items.first if marker_results.items.any?
+      search_issues("body", IssueMarker.for(comment, comment_sheet, @repo)) ||
+        search_issues("body", IssueMarker.legacy_for(comment, comment_sheet, @repo), verify: comment_sheet) ||
+        search_issues("title", unique_id, verify: comment_sheet)
+    end
 
-      query = "repo:#{@repo} in:title \"#{unique_id}\""
-      results = @github_client.search_issues(query)
-      results.items.first
+    def search_issues(field, term, verify: nil)
+      issues = @github_client.search_issues("repo:#{@repo} in:#{field} \"#{term}\"").items
+      return issues.first if verify.nil?
+      return nil if IssueMarker.document_segment(verify).nil?
+
+      issues.find { |issue| names_document?(issue, verify.document) }
+    end
+
+    def names_document?(issue, document)
+      issue.title.to_s.include?(document) || issue.body.to_s.include?(document)
     end
 
     def determine_labels(comment, comment_sheet)
@@ -307,48 +301,13 @@ module Commenter
     end
 
     def determine_milestone(_comment, _comment_sheet, _options)
-      # # Check for stage-specific milestone
-      # if comment_sheet.stage
-      #   stage_milestone = @config.dig("github", "stage_milestones", comment_sheet.stage)
-      #   if stage_milestone
-      #     milestone_number = resolve_milestone_by_name_or_number(stage_milestone)
-      #     return milestone_number if milestone_number
-      #   end
-      # end
-
-      # Use configured milestone
       milestone_config = @config.dig("github", "milestone")
       return nil unless milestone_config
 
-      if milestone_config["number"]
-        puts "[GitHubIssueCreator] Using milestone number: #{milestone_config["number"]}"
-        milestone_config["number"]
-      elsif milestone_config["name"]
-        # Resolved once per run and memoized: resolving per comment caused
-        # duplicated milestone fetches (see 3a06c3d).
-        @milestone_number ||= begin
-          puts "[GitHubIssueCreator] Using milestone name: #{milestone_config["name"]}"
-          resolve_milestone_by_name_or_number(milestone_config["name"])
-        end
-      end
-    end
-
-    def resolve_milestone_by_name_or_number(milestone_identifier)
-      # If it's a number, return it directly
-      return milestone_identifier.to_i if milestone_identifier.to_s.match?(/^\d+$/)
-
-      # Otherwise, search by name
-      find_milestone_by_name(milestone_identifier)
-    end
-
-    def find_milestone_by_name(name)
-      milestones = @github_client.milestones(@repo, state: "all")
-
-      puts "[GitHubIssueCreator] Found #{milestones.size} milestones in repository #{@repo}" if milestones.any?
-      milestone = milestones.find { |m| m.title == name }
-      milestone&.number
-    rescue Octokit::Error
-      nil
+      # Memoized: resolving per comment caused duplicated milestone fetches
+      # (see 3a06c3d).
+      @milestone_number ||= MilestoneResolver.new(@github_client, @repo, milestone_config).number
+      @milestone_number
     end
 
     def update_yaml_with_github_info(yaml_file, comment_sheet, results, options)

data/lib/commenter/issue_marker.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Commenter
+  # Deterministic issue markers for duplicate detection: exact, independent
+  # of the user's templates, and stable across runs. The document segment
+  # keeps the marker unambiguous when the same comment id exists for several
+  # documents in one repository; sheets without a document keep the earlier
+  # marker shape.
+  module IssueMarker
+    PREFIX = "urn:commenter"
+
+    module_function
+
+    def for(comment, comment_sheet, repo)
+      segments = [PREFIX, repo.to_s.downcase, comment_sheet.stage.to_s.downcase]
+      document = document_segment(comment_sheet)
+      segments << document unless document.nil?
+      segments << comment.id
+      segments.join(":")
+    end
+
+    # The marker shape used before markers carried the document. Ambiguous
+    # when two documents share a comment id, so callers must verify a hit
+    # against the sheet's document (see ExistingIssueFinder).
+    def legacy_for(comment, comment_sheet, repo)
+      "#{PREFIX}:#{repo.to_s.downcase}:#{comment_sheet.stage.to_s.downcase}:#{comment.id}"
+    end
+
+    def document_segment(comment_sheet)
+      document = comment_sheet.document.to_s.gsub(/\s+/, "").downcase
+      document.empty? ? nil : document
+    end
+  end
+end

data/lib/commenter/milestone_resolver.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module Commenter
+  # Resolves (and optionally creates) the GitHub milestone issues are
+  # assigned to: a `number` config passes through; a `name` resolves
+  # against the repository; `create_if_missing` mints the milestone on
+  # first use, honoring `description` and `due_date`.
+  class MilestoneResolver
+    def initialize(client, repo, config)
+      @client = client
+      @repo = repo
+      @config = config
+    end
+
+    def number
+      return @config["number"] if @config["number"]
+      return nil unless @config["name"]
+
+      @number ||= resolve
+      @number ||= create if @config["create_if_missing"]
+      @number
+    end
+
+    private
+
+    def resolve
+      return @config["name"].to_i if @config["name"].to_s.match?(/\A\d+\z/)
+
+      milestones = @client.milestones(@repo, state: "all")
+      puts "[GitHubIssueCreator] Found #{milestones.size} milestones in repository #{@repo}" if milestones.any?
+      milestones.find { |milestone| milestone.title == @config["name"] }&.number
+    rescue Octokit::Error
+      nil
+    end
+
+    def create
+      options = {}
+      options[:description] = @config["description"] if @config["description"]
+      if (due = @config["due_date"])
+        options[:due_on] = due.include?("T") ? due : "#{due}T00:00:00Z"
+      end
+      milestone = @client.create_milestone(@repo, @config["name"], options)
+      puts "[GitHubIssueCreator] Created milestone ##{milestone.number}: #{@config["name"]}"
+      milestone.number
+    rescue Octokit::Error => e
+      puts "[GitHubIssueCreator] Could not create milestone #{@config["name"].inspect}: #{e.message}"
+      nil
+    end
+  end
+end

data/lib/commenter/sheet_validator.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module Commenter
+  # Structural validation of a comment sheet — the checks the CLI schemas
+  # express, living next to the model so the vocabulary stays in one place.
+  # Returns a list of problems; entries are hashes with :severity (:error or
+  # :warning), :comment (id when the problem belongs to one comment), and
+  # :message. An empty list means the sheet is valid.
+  module SheetValidator
+    VERSIONS = %w[2012-03 osd].freeze
+    STAGES = %w[AWI WD CD DIS FDIS PRF PUB].freeze
+
+    module_function
+
+    def call(sheet)
+      problems = []
+      unless VERSIONS.include?(sheet.version)
+        problems << { severity: :error,
+                      message: "unknown version #{sheet.version.inspect} (expected #{VERSIONS.join(" or ")})" }
+      end
+      if sheet.stage && !STAGES.include?(sheet.stage)
+        problems << { severity: :warning,
+                      message: "unusual stage #{sheet.stage.inspect} (expected one of #{STAGES.join(", ")})" }
+      end
+
+      sheet.comments.each_with_index do |comment, index|
+        problems.concat(comment_problems(comment, index))
+      end
+
+      duplicates = sheet.comments.group_by { |comment| comment.id.to_s }
+                        .reject { |id, _| id.empty? }
+                        .select { |_, comments| comments.length > 1 }
+                        .keys
+      duplicates.sort.each do |id|
+        problems << { severity: :error, comment: id, message: "duplicate comment id #{id}" }
+      end
+
+      problems
+    end
+
+    def comment_problems(comment, index)
+      problems = []
+      label = comment.id.to_s.empty? ? "comment ##{index + 1}" : comment.id
+
+      problems << { severity: :error, comment: label, message: "missing id" } if comment.id.to_s.strip.empty?
+      problems << { severity: :error, comment: label, message: "missing comment text" } if comment.comments.to_s.strip.empty?
+      unless comment.type.to_s.strip.empty? || CommentType.known?(comment.type)
+        problems << { severity: :warning, comment: label,
+                      message: "unrecognized comment type #{comment.type.inspect} (no label will be minted for it)" }
+      end
+
+      problems
+    end
+  end
+end

data/lib/commenter/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Commenter
-  VERSION = "0.6.1"
+  VERSION = "0.7.1"
 end

data/lib/commenter.rb

@@ -14,12 +14,15 @@ module Commenter
   autoload :Ballot, "commenter/ballot"
   autoload :BallotReport, "commenter/ballot_report"
   autoload :BallotDiff, "commenter/ballot_diff"
+  autoload :SheetValidator, "commenter/sheet_validator"
   autoload :Comment, "commenter/comment"
   autoload :CommentSheet, "commenter/comment_sheet"
   autoload :GitHubSession, "commenter/github_session"
   autoload :Parser, "commenter/parser"
   autoload :Filler, "commenter/filler"
   autoload :GitHubIssueCreator, "commenter/github_integration"
+  autoload :IssueMarker, "commenter/issue_marker"
+  autoload :MilestoneResolver, "commenter/milestone_resolver"
   autoload :GitHubIssueRetriever, "commenter/github_issue_retriever"
   autoload :GitHubSync, "commenter/github_sync"
 end

data/spec/commenter/cli_spec.rb

@@ -101,3 +101,41 @@ RSpec.describe Commenter::Cli do
     end
   end
 end
+
+RSpec.describe Commenter::Cli do
+  describe "#validate" do
+    def sheet_file(data)
+      file = Tempfile.new(["sheet", ".yaml"])
+      file.write(data.to_yaml)
+      file.close
+      file
+    end
+
+    it "passes a sheet at the AWI stage" do
+      file = sheet_file({ "version" => "2012-03", "stage" => "AWI",
+                          "comments" => [{ "id" => "DE-001", "body" => "DE", "comments" => "Text" }] })
+
+      expect { described_class.start(["validate", file.path]) }
+        .to output(a_string_including("OK:", "no problems found")).to_stdout
+      file.unlink
+    end
+
+    it "does not fail when only warnings are found" do
+      file = sheet_file({ "version" => "2012-03", "stage" => "CUSTOM",
+                          "comments" => [{ "id" => "DE-001", "body" => "DE", "comments" => "Text" }] })
+
+      expect { described_class.start(["validate", file.path]) }
+        .to output(a_string_including('WARNING: unusual stage "CUSTOM"', "1 warning(s)")).to_stdout
+      file.unlink
+    end
+
+    it "fails when errors are found" do
+      file = sheet_file({ "version" => "2012-03",
+                          "comments" => [{ "id" => "DE-001", "body" => "DE" }] })
+
+      expect { described_class.start(["validate", file.path]) }
+        .to raise_error(Commenter::Error, a_string_including("1 error(s)"))
+      file.unlink
+    end
+  end
+end

data/spec/commenter/comment_spec.rb

@@ -81,6 +81,18 @@ RSpec.describe Commenter::Comment do
         expect(summary).to include("Line 42")
       end
 
+      it "does not prefix Clause when the clause already names it" do
+        comment = described_class.new(
+          locality: { clause: "Clause 4 and document" },
+          comments: "Comment text"
+        )
+
+        summary = comment.brief_summary
+
+        expect(summary).to start_with("Clause 4 and document:")
+        expect(summary).not_to include("Clause Clause")
+      end
+
       it "handles empty locality gracefully" do
         comment = described_class.new(
           locality: { clause: "" },
@@ -231,3 +243,19 @@ RSpec.describe Commenter::Comment do
     end
   end
 end
+
+RSpec.describe Commenter::Comment do
+  describe "#locality_summary" do
+    it "formats clause, element, and line number" do
+      comment = described_class.new(locality: { clause: "5.1", element: "Table 1", line_number: "42" })
+
+      expect(comment.locality_summary).to eq("Clause 5.1, Table 1, Line 42")
+    end
+
+    it "does not double the Clause prefix when the clause already names it" do
+      comment = described_class.new(locality: { clause: "Clause 4 and document" })
+
+      expect(comment.locality_summary).to eq("Clause 4 and document")
+    end
+  end
+end

data/spec/commenter/github_integration_spec.rb

@@ -17,13 +17,14 @@ RetrieverComment = Struct.new(:body, keyword_init: true)
 # exercised against an object with the same protocol as Octokit::Client.
 class FakeGitHubClient
   attr_accessor :rate_limit_failures, :search_error
-  attr_reader :created
+  attr_reader :created, :created_milestones
 
   def initialize(existing_issues = [])
     @issues = {}
     existing_issues.each { |issue| @issues[issue.number] = issue }
     @next_number = @issues.keys.max.to_i
     @created = []
+    @created_milestones = []
     @rate_limit_failures = 0
   end
 
@@ -46,6 +47,12 @@ class FakeGitHubClient
     [FakeMilestone.new(number: 1, title: "Test Milestone")]
   end
 
+  def create_milestone(_repo, title, _options = {})
+    milestone = FakeMilestone.new(number: 100 + @created_milestones.length + 1, title: title)
+    @created_milestones << milestone
+    milestone
+  end
+
   def create_issue(_repo, title, body, _options = {})
     if @rate_limit_failures.positive?
       @rate_limit_failures -= 1
@@ -175,6 +182,36 @@ RSpec.describe Commenter::GitHubIssueCreator do
     end
   end
 
+  describe "milestone handling" do
+    let(:client) { FakeGitHubClient.new }
+    let(:creator) do
+      described_class.new(config_file.path, title_template_file.path, body_template_file.path, client: client)
+    end
+
+    it "creates the milestone when create_if_missing is set" do
+      config_data["github"]["milestone"] = {
+        "name" => "Fresh Milestone",
+        "create_if_missing" => true,
+        "description" => "Created by specs",
+        "due_date" => "2026-12-31"
+      }
+
+      result = creator.create_issues_from_yaml(yaml_file.path, dry_run: true).first
+
+      expect(result[:milestone]).to eq(101)
+      expect(client.created_milestones.map(&:title)).to eq(["Fresh Milestone"])
+    end
+
+    it "leaves the milestone unset when missing without create_if_missing" do
+      config_data["github"]["milestone"] = { "name" => "Absent Milestone" }
+
+      result = creator.create_issues_from_yaml(yaml_file.path, dry_run: true).first
+
+      expect(result[:milestone]).to be_nil
+      expect(client.created_milestones).to be_empty
+    end
+  end
+
   describe "#create_issues_from_yaml" do
     let(:creator) { described_class.new(config_file.path, title_template_file.path, body_template_file.path) }
 
@@ -240,7 +277,7 @@ RSpec.describe Commenter::GitHubIssueCreator do
       end
 
       it "falls back to search when the recorded issue no longer exists" do
-        client = FakeGitHubClient.new([seeded_issue(5, "[DIS] US-001: posted earlier")])
+        client = FakeGitHubClient.new([seeded_issue(5, "[DIS] US-001: posted earlier (Test Document)")])
         sheet = write_sheet([yaml_data["comments"].first.merge("github" => { "issue_number" => 99 })])
         creator = described_class.new(config_file.path, title_template_file.path,
                                       body_template_file.path, client: client)
@@ -256,7 +293,7 @@ RSpec.describe Commenter::GitHubIssueCreator do
 
       it "falls back to search when the recorded issue title does not match the stage" do
         client = FakeGitHubClient.new([seeded_issue(7, "[WD] US-001: different stage"),
-                                       seeded_issue(5, "[DIS] US-001: posted earlier")])
+                                       seeded_issue(5, "[DIS] US-001: posted earlier (Test Document)")])
         sheet = write_sheet([yaml_data["comments"].first.merge("github" => { "issue_number" => 7 })])
         creator = described_class.new(config_file.path, title_template_file.path,
                                       body_template_file.path, client: client)
@@ -270,7 +307,7 @@ RSpec.describe Commenter::GitHubIssueCreator do
       end
 
       it "records the issue in the YAML when skipping via title search" do
-        client = FakeGitHubClient.new([seeded_issue(5, "[DIS] US-001: posted earlier")])
+        client = FakeGitHubClient.new([seeded_issue(5, "[DIS] US-001: posted earlier (Test Document)")])
         sheet = write_sheet(yaml_data["comments"])
         creator = described_class.new(config_file.path, title_template_file.path,
                                       body_template_file.path, client: client)
@@ -356,6 +393,14 @@ RSpec.describe Commenter::GitHubIssueCreator do
       file
     end
 
+    it "does not double the Clause prefix in locality_summary" do
+      yaml_data["comments"].first["locality"]["clause"] = "Clause 4 and document"
+
+      result = creator.create_issues_from_yaml(yaml_file.path, dry_run: true).first
+
+      expect(result[:title]).to include("Clause 4 and document|")
+    end
+
     it "exposes sheet, comment, and computed variables to templates" do
       result = creator.create_issues_from_yaml(yaml_file.path, dry_run: true).first
       variables = result[:title].split("|")
@@ -505,12 +550,57 @@ RSpec.describe Commenter::GitHubIssueCreator do
       file.unlink
     end
 
-    def write_yaml_with_github(issue_number)
+    it "embeds the document in the marker when the sheet has one" do
+      client = FakeGitHubClient.new
+      creator = described_class.new(config_file.path, client: client)
+
+      file = write_yaml_with_github(nil, document: "ISO 80000-2:2019")
+      results = creator.create_issues_from_yaml(file)
+
+      expect(results.first[:status]).to eq(:created)
+      expect(client.created.first.body)
+        .to match(%r{urn:commenter:test-org/test-repo:dis:iso80000-2:2019:US-001})
+      file.unlink
+    end
+
+    it "creates a new issue when a same-id issue exists for another document (legacy marker)" do
+      other = FakeIssue.new(number: 9, title: "[DIS] US-001 (Doc A)",
+                            body: "x `urn:commenter:test-org/test-repo:dis:US-001`",
+                            html_url: "u", state: "open", created_at: "2026-01-01T00:00:00Z")
+      client = FakeGitHubClient.new([other])
+      creator = described_class.new(config_file.path, client: client)
+
+      file = write_yaml_with_github(nil, document: "Doc B")
+      results = creator.create_issues_from_yaml(file)
+
+      expect(results.first[:status]).to eq(:created)
+      expect(results.first[:issue_number]).to eq(10)
+      file.unlink
+    end
+
+    it "skips via the legacy marker only when the existing issue names this document" do
+      legacy = FakeIssue.new(number: 9, title: "[DIS] US-001 Doc B",
+                             body: "x `urn:commenter:test-org/test-repo:dis:US-001`",
+                             html_url: "u", state: "open", created_at: "2026-01-01T00:00:00Z")
+      client = FakeGitHubClient.new([legacy])
+      creator = described_class.new(config_file.path, client: client)
+
+      file = write_yaml_with_github(nil, document: "Doc B")
+      results = creator.create_issues_from_yaml(file)
+
+      expect(results.first[:status]).to eq(:skipped)
+      expect(results.first[:issue_number]).to eq(9)
+      file.unlink
+    end
+
+    def write_yaml_with_github(issue_number, document: nil)
       github = issue_number ? { "issue_number" => issue_number, "status" => "open" } : nil
+      sheet = { "version" => "2012-03", "stage" => "DIS",
+                "comments" => [{ "id" => "US-001", "body" => "US", "comments" => "Text",
+                                 "github" => github }.compact] }
+      sheet["document"] = document unless document.nil?
       file = Tempfile.new(["comments", ".yaml"])
-      file.write({ "version" => "2012-03", "stage" => "DIS",
-                   "comments" => [{ "id" => "US-001", "body" => "US", "comments" => "Text",
-                                    "github" => github }.compact] }.to_yaml)
+      file.write(sheet.to_yaml)
       file.close
       file
     end

data/spec/commenter/sheet_validator_spec.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require "spec_helper"
+
+RSpec.describe Commenter::SheetValidator do
+  def sheet(comments, version: "2012-03", stage: nil)
+    Commenter::CommentSheet.new(version: version, stage: stage, comments: comments)
+  end
+
+  def comment(id, text: "Text", type: "technical")
+    Commenter::Comment.new(id: id, body: id.to_s.split("-").first, comments: text, type: type)
+  end
+
+  it "accepts a clean sheet" do
+    expect(described_class.call(sheet([comment("DE-001")]))).to eq([])
+  end
+
+  it "reports unknown versions and unusual stages" do
+    problems = described_class.call(sheet([comment("DE-001")], version: "2018-01", stage: "CUSTOM"))
+
+    expect(problems).to include(severity: :error, message: a_string_including("unknown version"))
+    expect(problems).to include(severity: :warning, message: a_string_including("unusual stage"))
+  end
+
+  it "accepts the AWI stage without a warning" do
+    expect(described_class.call(sheet([comment("DE-001")], stage: "AWI"))).to eq([])
+  end
+
+  it "reports missing ids, missing text, and duplicate ids as errors" do
+    problems = described_class.call(sheet([
+                                            comment("", text: "No id"),
+                                            comment("DE-001"),
+                                            comment("DE-001"),
+                                            comment("DE-002", text: "")
+                                          ]))
+
+    errors = problems.select { |problem| problem[:severity] == :error }
+    expect(errors.map { |problem| problem[:message] }).to include("missing id", "missing comment text",
+                                                                  "duplicate comment id DE-001")
+  end
+
+  it "warns about unrecognized comment types without failing" do
+    problems = described_class.call(sheet([comment("DE-001", type: "ge/te")]))
+
+    expect(problems).to contain_exactly(severity: :warning, comment: "DE-001",
+                                        message: a_string_including("unrecognized comment type"))
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: commenter
 version: !ruby/object:Gem::Version
-  version: 0.6.1
+  version: 0.7.1
 platform: ruby
 authors:
 - Ribose
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-09-08 00:00:00.000000000 Z
+date: 2026-11-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: base64
@@ -166,9 +166,12 @@ files:
 - lib/commenter/github_issue_retriever.rb
 - lib/commenter/github_session.rb
 - lib/commenter/github_sync.rb
+- lib/commenter/issue_marker.rb
+- lib/commenter/milestone_resolver.rb
 - lib/commenter/parser.rb
 - lib/commenter/parser/osd_xlsx_parser.rb
 - lib/commenter/parser/track_change_docx_parser.rb
+- lib/commenter/sheet_validator.rb
 - lib/commenter/version.rb
 - schema/iso_comment_2012-03.yaml
 - schema/iso_comment_osd.yaml
@@ -185,6 +188,7 @@ files:
 - spec/commenter/github_integration_spec.rb
 - spec/commenter/github_sync_spec.rb
 - spec/commenter/osd_xlsx_parser_spec.rb
+- spec/commenter/sheet_validator_spec.rb
 - spec/commenter/track_change_docx_parser_spec.rb
 - spec/commenter_spec.rb
 - spec/spec_helper.rb