nitfr 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 397b4e4f6de3e985e4d8bfbfe18ad663ab6bdd350536365a826728f87704c571
+  data.tar.gz: 775aab503543f0f678abbca1bb55e8420ec1ad829f7e2470a0f609dd2924798f
+SHA512:
+  metadata.gz: 482f2856dd1c1e1854f2d6f463dcdcf46ed644b417934b46847bf96597c3b1cf6012a2a9001de5bdcdd0573e53c2cf34ab0177a74d3a645fd2d8a6d3a074aa35
+  data.tar.gz: 59ac46fea40da9ace71f9ce9663ac6c6275d5c268b589d4f3f583261e34d78fe74dc027105fb3a27d4389650a34b9585fd6df7369ebdbd012fb11c8c60a85b70

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2008-2025 Mark Turner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,330 @@
+# NITFr
+
+[![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.0-red.svg)]()
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+A Ruby gem for parsing NITF (News Industry Text Format) XML files.
+
+NITF is a standard XML format developed by the IPTC (International Press Telecommunications Council) for marking up news articles. NITFr makes it easy for Ruby applications to parse and extract content from NITF documents.
+
+## Requirements
+
+- Ruby 3.0 or higher
+- No native extensions or external dependencies (pure Ruby using REXML)
+
+## Security
+
+NITFr is designed with security in mind:
+
+- **XXE Protection**: REXML does not expand external entities by default, protecting against XML External Entity (XXE) attacks
+- **Entity Expansion Limits**: Configured to prevent "Billion Laughs" and similar entity expansion attacks
+- **No Code Execution**: The parser never evaluates or executes content from XML documents
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'nitfr'
+```
+
+And then execute:
+
+```bash
+bundle install
+```
+
+Or install it yourself:
+
+```bash
+gem install nitfr
+```
+
+## Usage
+
+### Basic Parsing
+
+```ruby
+require 'nitfr'
+
+# Parse from a string
+xml = File.read('article.xml')
+doc = NITFr.parse(xml)
+
+# Or parse directly from a file
+doc = NITFr.parse_file('article.xml')
+
+# With explicit encoding
+doc = NITFr.parse_file('article.xml', encoding: 'ISO-8859-1')
+```
+
+### Accessing Content
+
+```ruby
+# Get the headline
+doc.headline          # => "Revolutionary Technology Changes Industry"
+doc.headlines.primary # => "Revolutionary Technology Changes Industry"
+doc.headlines.secondary # => "Experts predict widespread adoption"
+
+# Get byline information
+doc.byline.text       # => "By Jane Smith, Senior Reporter"
+doc.byline.person     # => "Jane Smith"
+doc.byline.title      # => "Senior Reporter"
+
+# Get the article text
+doc.paragraphs.each do |para|
+  puts para.text
+end
+
+# Or get all text at once
+puts doc.text
+```
+
+### Working with Metadata
+
+```ruby
+# Document metadata
+doc.title           # => "Sample News Article Title"
+doc.doc_id          # => "article-2024-001"
+doc.issue_date      # => #<Date: 2024-12-15>
+
+# Copyright info
+doc.docdata.copyright_holder  # => "Example News Corp"
+doc.docdata.copyright_year    # => "2024"
+
+# Urgency (1-8, 1 being most urgent)
+doc.docdata.urgency           # => 4
+
+# Identified content
+doc.docdata.subjects      # => ["Technology", "Business"]
+doc.docdata.organizations # => ["TechCorp Inc"]
+doc.docdata.people        # => ["John Doe"]
+doc.docdata.locations     # => ["San Francisco"]
+```
+
+### Working with Body Content
+
+```ruby
+# Access the body section
+body = doc.body
+
+# Dateline and abstract
+body.dateline   # => "SAN FRANCISCO, Dec 15"
+body.abstract   # => "A new technology platform..."
+
+# Block quotes
+body.block_quotes  # => ["Innovation distinguishes..."]
+
+# Tagline from body.end
+body.tagline    # => "Contact: press@example.com"
+```
+
+### Working with Paragraphs
+
+```ruby
+doc.paragraphs.each do |para|
+  # Check if it's the lead paragraph
+  puts "LEAD: " if para.lead?
+
+  # Get plain text
+  puts para.text
+
+  # Get entities mentioned in this paragraph
+  puts "People: #{para.people.join(', ')}"
+  puts "Organizations: #{para.organizations.join(', ')}"
+  puts "Locations: #{para.locations.join(', ')}"
+
+  # Get emphasized text
+  puts "Emphasized: #{para.emphasis.join(', ')}"
+
+  # Get links
+  para.links.each do |link|
+    puts "Link: #{link[:text]} -> #{link[:href]}"
+  end
+
+  # Word count
+  puts "Words: #{para.word_count}"
+end
+```
+
+### Working with Media
+
+```ruby
+doc.media.each do |media|
+  puts "Caption: #{media.caption}"
+  puts "Credit: #{media.credit}"
+  puts "MIME type: #{media.mime_type}"
+
+  if media.image?
+    puts "Image: #{media.source}"
+    puts "Size: #{media.width}x#{media.height}"
+    puts "Alt text: #{media.alt_text}"
+  elsif media.video?
+    puts "Video: #{media.source}"
+  elsif media.audio?
+    puts "Audio: #{media.source}"
+  end
+
+  # Access all references (different sizes/formats)
+  media.references.each do |ref|
+    puts "  #{ref[:source]} (#{ref[:mime_type]})"
+  end
+end
+```
+
+### Error Handling
+
+```ruby
+begin
+  doc = NITFr.parse(xml)
+rescue NITFr::ParseError => e
+  puts "Invalid XML: #{e.message}"
+rescue NITFr::InvalidDocumentError => e
+  puts "Not a valid NITF document: #{e.message}"
+end
+```
+
+### Document Attributes
+
+```ruby
+# NITF version and change information
+doc.version      # => "-//IPTC//DTD NITF 3.5//EN"
+doc.change_date  # => "October 18, 2007"
+doc.change_time  # => "19:30"
+
+# Check validity
+doc.valid?       # => true
+
+# Get raw XML
+doc.to_xml       # => "<?xml version..."
+```
+
+## Advanced Usage
+
+### Head Section Details
+
+```ruby
+head = doc.head
+
+# Meta tags as a hash
+head.meta        # => {"keywords" => "tech, news", "author" => "Jane"}
+head.keywords    # => ["tech, news"]
+
+# Publication data
+head.pubdata[:type]        # => "print"
+head.pubdata[:name]        # => "Example Times"
+head.pubdata[:edition]     # => "Morning"
+head.pubdata[:volume]      # => "42"
+
+# Revision history
+head.revision_history.each do |rev|
+  puts "#{rev[:name]} (#{rev[:function]}): #{rev[:comment]}"
+end
+```
+
+### Extended Docdata
+
+```ruby
+docdata = doc.docdata
+
+# Additional dates
+docdata.release_date  # => #<Date: 2024-12-15>
+docdata.expire_date   # => #<Date: 2024-12-31>
+
+# Document scope and fixture
+docdata.doc_scope     # => "national"
+docdata.fixture       # => "fixture-123"
+
+# Series information
+docdata.series[:name]   # => "Investigation"
+docdata.series[:part]   # => 2
+docdata.series[:total]  # => 5
+
+# Editorial status
+docdata.management_status[:info]         # => "Approved"
+docdata.management_status[:message_type] # => "advisory"
+```
+
+### Body Section Extras
+
+```ruby
+body = doc.body
+
+# Distributor and series
+body.distributor  # => "Wire Service"
+body.series[:name]      # => "Special Report"
+body.series[:part]      # => "1"
+body.series[:totalpart] # => "3"
+
+# Lists in the content
+body.lists.each do |list|
+  puts "#{list[:type]}: #{list[:items].join(', ')}"
+end
+
+# Tables (returns raw REXML elements)
+body.tables.each do |table|
+  # Process table XML as needed
+end
+
+# Notes from body.end
+body.notes  # => ["Editor's note: ...", "Correction: ..."]
+
+# Bibliography
+body.body_end_content[:bibliography]  # => ["Source 1", "Source 2"]
+```
+
+## NITF Structure
+
+A typical NITF document has this structure:
+
+```xml
+<nitf>
+  <head>
+    <title>...</title>
+    <docdata>
+      <doc-id id-string="..."/>
+      <date.issue norm="YYYYMMDD"/>
+      ...
+    </docdata>
+  </head>
+  <body>
+    <body.head>
+      <headline>
+        <hl1>Primary Headline</hl1>
+        <hl2>Secondary Headline</hl2>
+      </headline>
+      <byline>By Author Name</byline>
+      <dateline>CITY, Date</dateline>
+    </body.head>
+    <body.content>
+      <p>Paragraph content...</p>
+      <media media-type="image">...</media>
+    </body.content>
+    <body.end>
+      <tagline>...</tagline>
+    </body.end>
+  </body>
+</nitf>
+```
+
+## Development
+
+After checking out the repo, install dependencies and run the tests:
+
+```bash
+bundle install
+bundle exec rake test
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/amerine/nitfr.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## References
+
+- [IPTC NITF Specification](https://iptc.org/standards/nitf/)
+- [NITF 3.5 DTD](http://www.nitf.org/IPTC/NITF/3.5/specification/nitf-3-5.dtd)

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+task default: :test

data/lib/nitfr/body.rb

@@ -0,0 +1,191 @@
+# frozen_string_literal: true
+
+module NITFr
+  # Represents the body section of an NITF document
+  #
+  # The body contains the main content of the news article including
+  # headline, byline, and the actual article text in body.content.
+  class Body
+    attr_reader :node
+
+    def initialize(node)
+      @node = node
+    end
+
+    # Get the headline object
+    #
+    # @return [Headline, nil] the headline with all levels
+    def headline
+      @headline ||= begin
+        headline_node = body_head && xpath_first(body_head, "headline")
+        Headline.new(headline_node) if headline_node
+      end
+    end
+
+    # Get the byline object
+    #
+    # @return [Byline, nil] the byline information
+    def byline
+      @byline ||= begin
+        byline_node = body_head && xpath_first(body_head, "byline")
+        Byline.new(byline_node) if byline_node
+      end
+    end
+
+    # Get the dateline text
+    #
+    # @return [String, nil] the dateline
+    def dateline
+      @dateline ||= (body_head && xpath_first(body_head, "dateline"))&.text&.strip
+    end
+
+    # Get the abstract/summary
+    #
+    # @return [String, nil] the abstract text
+    def abstract
+      @abstract ||= (body_head && xpath_first(body_head, "abstract"))&.text&.strip
+    end
+
+    # Get distributor information
+    #
+    # @return [String, nil] the distributor
+    def distributor
+      @distributor ||= (body_head && xpath_first(body_head, "distributor"))&.text&.strip
+    end
+
+    # Get series information
+    #
+    # @return [Hash, nil] series metadata
+    def series
+      @series ||= parse_series
+    end
+
+    # Get all paragraphs from body.content
+    #
+    # @return [Array<Paragraph>] array of paragraph objects
+    def paragraphs
+      @paragraphs ||= begin
+        return [] unless body_content
+
+        xpath_match(body_content, ".//p").map { |p| Paragraph.new(p) }
+      end
+    end
+
+    # Get all media objects from body.content
+    #
+    # @return [Array<Media>] array of media objects
+    def media
+      @media ||= begin
+        return [] unless body_content
+
+        xpath_match(body_content, ".//media").map { |m| Media.new(m) }
+      end
+    end
+
+    # Get all block quotes
+    #
+    # @return [Array<String>] array of block quote texts
+    def block_quotes
+      @block_quotes ||= begin
+        return [] unless body_content
+
+        xpath_match(body_content, ".//bq/block").map { |b| b.text&.strip }.compact
+      end
+    end
+
+    # Get all lists from the content
+    #
+    # @return [Array<Hash>] array of list structures
+    def lists
+      @lists ||= parse_lists
+    end
+
+    # Get all tables from the content
+    #
+    # @return [Array<REXML::Element>] raw table nodes
+    def tables
+      @tables ||= begin
+        return [] unless body_content
+
+        xpath_match(body_content, ".//table")
+      end
+    end
+
+    # Get the body.end content (tagline, bibliography)
+    #
+    # @return [Hash] body end content
+    def body_end_content
+      @body_end_content ||= parse_body_end
+    end
+
+    # Get the tagline
+    #
+    # @return [String, nil] the tagline text
+    def tagline
+      body_end_content[:tagline]
+    end
+
+    # Get notes from body.end
+    #
+    # @return [Array<String>] array of notes
+    def notes
+      body_end_content[:notes] || []
+    end
+
+    private
+
+    def xpath_first(context, path)
+      REXML::XPath.first(context, path)
+    end
+
+    def xpath_match(context, path)
+      REXML::XPath.match(context, path)
+    end
+
+    def body_head
+      @body_head ||= xpath_first(node, "body.head")
+    end
+
+    def body_content
+      @body_content ||= xpath_first(node, "body.content")
+    end
+
+    def body_end
+      @body_end ||= xpath_first(node, "body.end")
+    end
+
+    def parse_series
+      return nil unless body_head
+
+      series_node = xpath_first(body_head, "series")
+      return nil unless series_node
+
+      {
+        name: series_node.attributes["series.name"],
+        part: series_node.attributes["series.part"],
+        totalpart: series_node.attributes["series.totalpart"]
+      }.compact
+    end
+
+    def parse_lists
+      return [] unless body_content
+
+      xpath_match(body_content, ".//ul | .//ol | .//dl").map do |list|
+        {
+          type: list.name,
+          items: xpath_match(list, ".//li | .//dt | .//dd").map { |item| item.text&.strip }.compact
+        }
+      end
+    end
+
+    def parse_body_end
+      return {} unless body_end
+
+      {
+        tagline: xpath_first(body_end, "tagline")&.text&.strip,
+        notes: xpath_match(body_end, ".//note").map { |n| n.text&.strip }.compact,
+        bibliography: xpath_match(body_end, ".//biblio").map { |b| b.text&.strip }.compact
+      }.compact
+    end
+  end
+end

data/lib/nitfr/byline.rb

@@ -0,0 +1,66 @@
+# frozen_string_literal: true
+
+module NITFr
+  # Represents byline information from an NITF document
+  #
+  # Bylines can contain the author name, their title/role,
+  # and additional location information.
+  class Byline
+    include TextExtractor
+
+    attr_reader :node
+
+    def initialize(node)
+      @node = node
+    end
+
+    # Get the full byline text
+    #
+    # @return [String, nil] the complete byline text
+    def text
+      @text ||= extract_all_text(node).strip
+    end
+    alias to_s text
+
+    # Get the person/author name from byttl element
+    #
+    # @return [String, nil] the author name
+    def person
+      @person ||= xpath_first("person")&.text&.strip
+    end
+
+    # Get the byline title/role
+    #
+    # @return [String, nil] the title or role
+    def title
+      @title ||= xpath_first("byttl")&.text&.strip
+    end
+
+    # Get the location if specified
+    #
+    # @return [String, nil] the location
+    def location
+      @location ||= xpath_first("location")&.text&.strip
+    end
+
+    # Get organization/affiliation
+    #
+    # @return [String, nil] the organization name
+    def org
+      @org ||= xpath_first("org")&.text&.strip
+    end
+
+    # Check if byline has content
+    #
+    # @return [Boolean] true if byline has text
+    def present?
+      !text.empty?
+    end
+
+    private
+
+    def xpath_first(path)
+      REXML::XPath.first(node, path)
+    end
+  end
+end