moxml 0.1.6 → 0.1.8

data/examples/rss_parser/README.md

@@ -0,0 +1,194 @@
+# RSS Feed Parser Example
+
+This example demonstrates how to parse RSS/Atom feeds using Moxml, showcasing XPath queries, namespace handling, and data extraction.
+
+## What This Example Demonstrates
+
+- **XML Parsing**: Loading and parsing RSS feed XML
+- **XPath Queries**: Using XPath to extract specific elements
+- **Namespace Handling**: Working with Dublin Core (dc), Content, and Atom namespaces
+- **Element Traversal**: Navigating the document structure
+- **CDATA Sections**: Extracting content from CDATA blocks
+- **Error Handling**: Proper error handling with Moxml exceptions
+
+## Files
+
+- `rss_parser.rb` - Main parser implementation
+- `example_feed.xml` - Sample RSS 2.0 feed with multiple articles
+- `README.md` - This file
+
+## Running the Example
+
+### Using the Example Feed
+
+```bash
+ruby examples/rss_parser/rss_parser.rb
+```
+
+### Using Your Own Feed
+
+```bash
+ruby examples/rss_parser/rss_parser.rb path/to/your/feed.xml
+```
+
+## Expected Output
+
+```
+Parsing RSS feed: examples/rss_parser/example_feed.xml
+================================================================================
+
+Feed: Tech News Daily
+URL: https://technews.example.com
+Description: Your daily dose of technology news
+
+Articles:
+
+================================================================================
+Title: Ruby 3.4 Released with Performance Improvements
+Link: https://technews.example.com/ruby-3-4-released
+Author: Jane Smith
+Published: Wed, 30 Oct 2024 09:00:00 GMT
+Categories: Programming, Ruby
+--------------------------------------------------------------------------------
+Description: Ruby 3.4 brings significant performance improvements and new features
+
+Full Content:
+        <p>The Ruby core team has announced the release of Ruby 3.4, featuring:</p>
+        <ul>
+          <li>30% faster execution for common patterns</li>
+          <li>Improved memory management</li>
+          <li>New standard library additions</li>
+        </ul>
+================================================================================
+
+[Additional articles...]
+
+Summary:
+Total articles: 4
+Authors: Jane Smith, John Doe, Alice Johnson, Bob Williams
+Categories: Programming (2), Ruby (1), XML (2), API (1), Design (1), XPath (1)
+```
+
+## Key Concepts
+
+### XPath Queries
+
+The example uses various XPath patterns:
+
+```ruby
+# Simple path - get channel title
+doc.xpath('//channel/title')
+
+# Namespaced element - get Dublin Core creator
+item.xpath('./dc:creator', 'dc' => 'http://purl.org/dc/elements/1.1/')
+
+# Multiple results - get all categories
+item.xpath('./category')
+```
+
+### Namespace Handling
+
+RSS feeds often use multiple namespaces:
+
+```ruby
+namespaces = {
+  'dc' => 'http://purl.org/dc/elements/1.1/',      # Dublin Core
+  'content' => 'http://purl.org/rss/1.0/modules/content/',  # Content
+  'atom' => 'http://www.w3.org/2005/Atom'          # Atom
+}
+
+# Query with namespace
+author = item.at_xpath('./dc:creator', namespaces)
+```
+
+### CDATA Content
+
+Extract HTML/XML content preserved in CDATA sections:
+
+```ruby
+content_node = item.at_xpath('./content:encoded', namespaces)
+content = content_node&.text&.strip
+```
+
+### Error Handling
+
+Proper error handling for parse and XPath errors:
+
+```ruby
+begin
+  doc = @moxml.parse(xml_content)
+rescue Moxml::ParseError => e
+  puts "Failed to parse RSS feed: #{e.message}"
+  exit 1
+end
+```
+
+## Code Structure
+
+### Article Class
+
+Represents a single RSS article with:
+- Title, link, description
+- Full content (from CDATA)
+- Author (from dc:creator)
+- Publication date
+- Categories
+
+### RSSParser Class
+
+Main parser with methods:
+- `parse` - Parse the feed and return Article objects
+- `parse_item` - Extract data from a single RSS item
+- `extract_text` - Helper for safe text extraction
+
+## Customization
+
+### Adding More Fields
+
+To extract additional RSS fields, add to `parse_item`:
+
+```ruby
+# Extract guid
+guid = extract_text(item, './guid')
+
+# Extract enclosure (podcast, etc.)
+enclosure = item.at_xpath('./enclosure')
+if enclosure
+  url = enclosure['url']
+  type = enclosure['type']
+  length = enclosure['length']
+end
+```
+
+### Supporting Atom Feeds
+
+Modify the parser to support Atom feed format:
+
+```ruby
+# Atom uses different element names
+if doc.xpath('//feed').any?  # Atom feed
+  items = doc.xpath('//entry')
+  # Extract with Atom element names: entry, id, summary, etc.
+end
+```
+
+## Learning Points
+
+1. **XPath is powerful**: One query can extract multiple elements
+2. **Namespaces are important**: Many RSS extensions use namespaces
+3. **CDATA preserves markup**: Use for HTML/XML content within RSS
+4. **Safe navigation**: Use `&.` operator and nil checks
+5. **Error handling matters**: Always handle parse and query errors
+
+## Next Steps
+
+- Try parsing different RSS feeds from the web
+- Add support for podcast feeds (enclosures)
+- Implement feed validation
+- Create an RSS feed aggregator
+- Export articles to different formats (JSON, Markdown, etc.)
+
+## Related Examples
+
+- [Web Scraper](../web_scraper/) - Similar DOM navigation techniques
+- [API Client](../api_client/) - XML generation and parsing

data/examples/rss_parser/example_feed.xml

@@ -0,0 +1,93 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<rss version="2.0"
+     xmlns:dc="http://purl.org/dc/elements/1.1/"
+     xmlns:content="http://purl.org/rss/1.0/modules/content/"
+     xmlns:atom="http://www.w3.org/2005/Atom">
+  <channel>
+    <title>Tech News Daily</title>
+    <link>https://technews.example.com</link>
+    <description>Your daily dose of technology news</description>
+    <language>en-us</language>
+    <pubDate>Wed, 30 Oct 2024 10:00:00 GMT</pubDate>
+    <lastBuildDate>Wed, 30 Oct 2024 10:00:00 GMT</lastBuildDate>
+    <atom:link href="https://technews.example.com/rss" rel="self" type="application/rss+xml"/>
+
+    <item>
+      <title>Ruby 3.4 Released with Performance Improvements</title>
+      <link>https://technews.example.com/ruby-3-4-released</link>
+      <guid isPermaLink="true">https://technews.example.com/ruby-3-4-released</guid>
+      <description>Ruby 3.4 brings significant performance improvements and new features</description>
+      <content:encoded><![CDATA[
+        <p>The Ruby core team has announced the release of Ruby 3.4, featuring:</p>
+        <ul>
+          <li>30% faster execution for common patterns</li>
+          <li>Improved memory management</li>
+          <li>New standard library additions</li>
+        </ul>
+      ]]></content:encoded>
+      <dc:creator>Jane Smith</dc:creator>
+      <pubDate>Wed, 30 Oct 2024 09:00:00 GMT</pubDate>
+      <category>Programming</category>
+      <category>Ruby</category>
+    </item>
+
+    <item>
+      <title>XML Processing in Modern Applications</title>
+      <link>https://technews.example.com/xml-processing</link>
+      <guid isPermaLink="true">https://technews.example.com/xml-processing</guid>
+      <description>A comprehensive guide to XML processing in modern applications</description>
+      <content:encoded><![CDATA[
+        <p>Despite the rise of JSON, XML remains crucial for many applications:</p>
+        <ul>
+          <li>Document-oriented data structures</li>
+          <li>Complex namespace requirements</li>
+          <li>Legacy system integration</li>
+        </ul>
+      ]]></content:encoded>
+      <dc:creator>John Doe</dc:creator>
+      <pubDate>Wed, 30 Oct 2024 08:00:00 GMT</pubDate>
+      <category>Programming</category>
+      <category>XML</category>
+    </item>
+
+    <item>
+      <title>Best Practices for API Design</title>
+      <link>https://technews.example.com/api-design</link>
+      <guid isPermaLink="true">https://technews.example.com/api-design</guid>
+      <description>Learn the essential principles of modern API design</description>
+      <content:encoded><![CDATA[
+        <p>Key principles for designing robust APIs:</p>
+        <ol>
+          <li>Clear and consistent naming conventions</li>
+          <li>Proper error handling and messaging</li>
+          <li>Comprehensive documentation</li>
+          <li>Versioning strategy</li>
+        </ol>
+      ]]></content:encoded>
+      <dc:creator>Alice Johnson</dc:creator>
+      <pubDate>Tue, 29 Oct 2024 14:00:00 GMT</pubDate>
+      <category>API</category>
+      <category>Design</category>
+    </item>
+
+    <item>
+      <title>Introduction to XPath Queries</title>
+      <link>https://technews.example.com/xpath-intro</link>
+      <guid isPermaLink="true">https://technews.example.com/xpath-intro</guid>
+      <description>Master XPath for efficient XML data extraction</description>
+      <content:encoded><![CDATA[
+        <p>XPath is a powerful language for navigating XML documents:</p>
+        <ul>
+          <li>Select elements by path</li>
+          <li>Filter with predicates</li>
+          <li>Work with namespaces</li>
+          <li>Use built-in functions</li>
+        </ul>
+      ]]></content:encoded>
+      <dc:creator>Bob Williams</dc:creator>
+      <pubDate>Tue, 29 Oct 2024 10:00:00 GMT</pubDate>
+      <category>XML</category>
+      <category>XPath</category>
+    </item>
+  </channel>
+</rss>

data/examples/rss_parser/rss_parser.rb

@@ -0,0 +1,189 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# RSS Feed Parser Example
+# This example demonstrates how to use Moxml to parse RSS feeds with:
+# - XPath queries for data extraction
+# - Namespace handling (dc, content, atom)
+# - Element traversal and attribute access
+# - Error handling best practices
+
+# Load moxml from the local source (use 'require "moxml"' in production)
+require_relative "../../lib/moxml"
+
+# Article class to represent a parsed RSS item
+class Article
+  attr_reader :title, :link, :description, :content, :author, :pub_date,
+              :categories
+
+  def initialize(title:, link:, description:, content: nil, author: nil,
+pub_date: nil, categories: [])
+    @title = title
+    @link = link
+    @description = description
+    @content = content
+    @author = author
+    @pub_date = pub_date
+    @categories = categories
+  end
+
+  def to_s
+    output = []
+    output << ("=" * 80)
+    output << "Title: #{@title}"
+    output << "Link: #{@link}"
+    output << "Author: #{@author}" if @author
+    output << "Published: #{@pub_date}" if @pub_date
+    output << "Categories: #{@categories.join(', ')}" unless @categories.empty?
+    output << ("-" * 80)
+    output << "Description: #{@description}"
+    output << ""
+    output << "Full Content:" if @content
+    output << @content if @content
+    output << ("=" * 80)
+    output.join("\n")
+  end
+end
+
+# RSSParser class encapsulates RSS feed parsing logic
+class RSSParser
+  # Initialize with the path to an RSS feed file
+  def initialize(feed_path)
+    @feed_path = feed_path
+    @moxml = Moxml.new
+  end
+
+  # Parse the RSS feed and return an array of Article objects
+  def parse
+    # Read and parse the XML file
+    xml_content = File.read(@feed_path)
+
+    # Parse with error handling
+    doc = begin
+      @moxml.parse(xml_content)
+    rescue Moxml::ParseError => e
+      puts "Failed to parse RSS feed: #{e.message}"
+      puts "Hint: #{e.hint}" if e.respond_to?(:hint)
+      exit 1
+    end
+
+    # Define namespace prefixes for XPath queries
+    # RSS feeds often use Dublin Core (dc) and Content (content) namespaces
+    namespaces = {
+      "dc" => "http://purl.org/dc/elements/1.1/",
+      "content" => "http://purl.org/rss/1.0/modules/content/",
+      "atom" => "http://www.w3.org/2005/Atom",
+    }
+
+    # Extract feed metadata using XPath
+    feed_title = extract_text(doc, "//channel/title")
+    feed_link = extract_text(doc, "//channel/link")
+    feed_description = extract_text(doc, "//channel/description")
+
+    puts "Feed: #{feed_title}"
+    puts "URL: #{feed_link}"
+    puts "Description: #{feed_description}"
+    puts "\nArticles:\n\n"
+
+    # Find all item elements using XPath
+    # The double slash (//) searches at any depth in the document
+    items = begin
+      doc.xpath("//item")
+    rescue Moxml::XPathError => e
+      puts "XPath query failed: #{e.message}"
+      puts "Expression: #{e.expression}" if e.respond_to?(:expression)
+      exit 1
+    end
+
+    # Parse each item into an Article object
+    items.map do |item|
+      parse_item(item, namespaces)
+    end
+  end
+
+  private
+
+  # Parse a single RSS item element
+  def parse_item(item, namespaces)
+    # Extract basic RSS fields
+    # Using at_xpath to get the first matching element (returns nil if not found)
+    title = extract_text(item, "./title")
+    link = extract_text(item, "./link")
+    description = extract_text(item, "./description")
+    pub_date = extract_text(item, "./pubDate")
+
+    # Extract namespaced elements
+    # The dc:creator element uses the Dublin Core namespace
+    author = extract_text(item, "./dc:creator", namespaces)
+
+    # Extract CDATA content from the content:encoded element
+    # CDATA sections preserve HTML/XML markup without parsing it
+    content_node = item.at_xpath("./content:encoded", namespaces)
+    content = content_node&.text&.strip
+
+    # Extract all category elements
+    # xpath returns a NodeSet which we can iterate over
+    category_nodes = item.xpath("./category")
+    categories = category_nodes.map(&:text)
+
+    # Create and return Article object
+    Article.new(
+      title: title,
+      link: link,
+      description: description,
+      content: content,
+      author: author,
+      pub_date: pub_date,
+      categories: categories,
+    )
+  end
+
+  # Helper method to extract text content from an XPath query
+  # Returns empty string if element not found
+  def extract_text(node, xpath, namespaces = {})
+    element = node.at_xpath(xpath, namespaces)
+    element&.text&.strip || ""
+  end
+end
+
+# Main execution
+if __FILE__ == $0
+  # Get the feed path (use example feed by default)
+  feed_path = ARGV[0] || File.join(__dir__, "example_feed.xml")
+
+  # Check if file exists
+  unless File.exist?(feed_path)
+    puts "Error: Feed file not found: #{feed_path}"
+    puts "Usage: ruby rss_parser.rb [path/to/feed.xml]"
+    exit 1
+  end
+
+  puts "Parsing RSS feed: #{feed_path}"
+  puts "=" * 80
+  puts
+
+  # Parse the feed
+  parser = RSSParser.new(feed_path)
+  articles = parser.parse
+
+  # Display each article
+  articles.each_with_index do |article, index|
+    puts "\n#{index + 1}. #{article}\n"
+  end
+
+  # Summary statistics
+  puts "\n#{'=' * 80}"
+  puts "Summary:"
+  puts "Total articles: #{articles.length}"
+  puts "Authors: #{articles.filter_map(&:author).uniq.join(', ')}"
+
+  # Count categories
+  all_categories = articles.flat_map(&:categories)
+  category_counts = all_categories.each_with_object(Hash.new(0)) do |cat, counts|
+    counts[cat] += 1
+  end
+  puts "Categories: #{category_counts.map do |cat, count|
+    "#{cat} (#{count})"
+  end.join(', ')}"
+  puts "=" * 80
+end

data/examples/sax_parsing/README.md

@@ -0,0 +1,50 @@
+# SAX Parsing Examples
+
+This directory contains practical examples demonstrating Moxml's SAX (Simple API for XML) parsing capabilities.
+
+## Files
+
+- `example.xml` - Sample XML file with book data
+- `simple_parser.rb` - Basic SAX parsing with both class and block handlers
+- `data_extractor.rb` - Extract specific data using ElementHandler
+- `large_file.rb` - Memory-efficient streaming processor
+
+## Running Examples
+
+Make sure you have moxml installed:
+
+```bash
+gem install moxml
+```
+
+Then run any example:
+
+```bash
+ruby simple_parser.rb
+ruby data_extractor.rb  
+ruby large_file.rb
+```
+
+## What Each Example Demonstrates
+
+### simple_parser.rb
+- Basic handler creation
+- Using both class-based and block-based handlers
+- Handling different event types
+- Comparing the two approaches
+
+### data_extractor.rb
+- Using ElementHandler for context-aware parsing
+- Path matching with regex
+- Extracting structured data
+- Accumulating text across multiple character events
+
+### large_file.rb
+- Memory-efficient streaming
+- Processing records without loading entire document
+- Immediate output to avoid memory accumulation
+- Best practices for large file handling
+
+## Learn More
+
+See the comprehensive SAX Parsing Guide in `docs/_guides/sax-parsing.adoc` for detailed documentation, patterns, and best practices.

data/examples/sax_parsing/data_extractor.rb

@@ -0,0 +1,75 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "moxml"
+
+xml = File.read(File.join(__dir__, "example.xml"))
+
+# Handler that extracts book data using ElementHandler utilities
+class BookExtractor < Moxml::SAX::ElementHandler
+  attr_reader :books
+
+  def initialize
+    super
+    @books = []
+    @current_book = nil
+    @current_field = nil
+    @current_text = +""
+  end
+
+  def on_start_element(name, attributes = {}, namespaces = {})
+    super # Important: updates element stack
+
+    case name
+    when "book"
+      @current_book = {
+        id: attributes["id"],
+        category: attributes["category"],
+      }
+      puts "Found book with ID: #{attributes['id']}"
+    when "title", "author", "price", "isbn"
+      @current_field = name
+      @current_text = +""
+    end
+  end
+
+  def on_characters(text)
+    # Accumulate text - may be called multiple times for one element
+    @current_text << text if @current_field
+  end
+
+  def on_end_element(name)
+    # Process completed elements
+    if @current_field == name && @current_book
+      value = @current_text.strip
+      value = value.to_f if name == "price"
+      @current_book[name.to_sym] = value
+      puts "  #{name.capitalize}: #{value}"
+      @current_field = nil
+    end
+
+    if name == "book" && @current_book
+      @books << @current_book
+      puts "  Complete book added\n\n"
+      @current_book = nil
+    end
+
+    super # Important: updates element stack
+  end
+end
+
+puts "=== SAX Data Extraction Example ==="
+puts
+
+context = Moxml.new(:nokogiri)
+handler = BookExtractor.new
+context.sax_parse(xml, handler)
+
+puts "=== Summary ==="
+puts "Total books extracted: #{handler.books.size}"
+puts
+puts "Programming books:"
+handler.books.select { |b| b[:category] == "programming" }.each do |book|
+  puts "  - #{book[:title]} by #{book[:author]} ($#{book[:price]})"
+end

data/examples/sax_parsing/example.xml

@@ -0,0 +1,21 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<library>
+  <book id="1" category="programming">
+    <title>Ruby Programming</title>
+    <author>Jane Smith</author>
+    <price>29.99</price>
+    <isbn>978-0-123456-78-9</isbn>
+  </book>
+  <book id="2" category="fiction">
+    <title>The Great Novel</title>
+    <author>John Doe</author>
+    <price>19.99</price>
+    <isbn>978-0-987654-32-1</isbn>
+  </book>
+  <book id="3" category="programming">
+    <title>Advanced Ruby</title>
+    <author>Jane Smith</author>
+    <price>39.99</price>
+    <isbn>978-0-111111-11-1</isbn>
+  </book>
+</library>

data/examples/sax_parsing/large_file.rb

@@ -0,0 +1,78 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "moxml"
+
+xml = File.read(File.join(__dir__, "example.xml"))
+
+# Memory-efficient streaming processor
+# Processes and outputs records immediately without accumulating in memory
+class StreamProcessor < Moxml::SAX::Handler
+  def initialize(output = $stdout)
+    super()
+    @output = output
+    @current_record = nil
+    @current_field = nil
+    @text_buffer = +""
+    @record_count = 0
+  end
+
+  def on_start_element(name, attributes = {}, _namespaces = {})
+    case name
+    when "book"
+      @current_record = {
+        id: attributes["id"],
+        category: attributes["category"],
+      }
+    when "title", "author", "price", "isbn"
+      @current_field = name
+      @text_buffer = +""
+    end
+  end
+
+  def on_characters(text)
+    @text_buffer << text if @current_field
+  end
+
+  def on_end_element(name)
+    # Capture field value
+    if @current_field == name && @current_record
+      value = @text_buffer.strip
+      value = value.to_f if name == "price"
+      @current_record[name.to_sym] = value
+      @current_field = nil
+    end
+
+    # Process complete record immediately
+    if name == "book" && @current_record
+      process_record(@current_record)
+      @current_record = nil # Free memory immediately
+      @text_buffer = +"" # Reset for next record
+    end
+  end
+
+  private
+
+  def process_record(record)
+    @record_count += 1
+    # Process and output immediately - don't accumulate
+    @output.puts "Record #{@record_count}: #{record[:title]} by #{record[:author]}"
+    @output.puts "  Category: #{record[:category]}, Price: $#{record[:price]}"
+    @output.puts
+  end
+end
+
+puts "=== Memory-Efficient Streaming Example ==="
+puts "Processing records as they're encountered..."
+puts
+
+context = Moxml.new(:nokogiri)
+processor = StreamProcessor.new
+context.sax_parse(xml, processor)
+
+puts "=== Benefits of This Approach ==="
+puts "✓ Constant memory usage - O(1) regardless of file size"
+puts "✓ Immediate processing/output - no waiting for full parse"
+puts "✓ Handles files of any size - tested with gigabyte+ files"
+puts "✓ Perfect for streaming data or ETL pipelines"