moxml 0.1.7 → 0.1.8

data/examples/sax_parsing/simple_parser.rb

@@ -0,0 +1,55 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "moxml"
+
+xml = File.read(File.join(__dir__, "example.xml"))
+
+puts "=== Example 1: Class-Based Handler ==="
+puts
+
+# Define a simple handler class
+class SimpleHandler < Moxml::SAX::Handler
+  def on_start_document
+    puts "Document started"
+  end
+
+  def on_start_element(name, attributes = {}, _namespaces = {})
+    attrs_str = attributes.map { |k, v| "#{k}=#{v}" }.join(", ")
+    puts "  Start element: #{name}" + (attrs_str.empty? ? "" : " [#{attrs_str}]")
+  end
+
+  def on_characters(text)
+    text = text.strip
+    puts "    Text: #{text}" unless text.empty?
+  end
+
+  def on_end_element(name)
+    puts "  End element: #{name}"
+  end
+
+  def on_end_document
+    puts "Document ended"
+  end
+end
+
+context = Moxml.new(:nokogiri)
+handler = SimpleHandler.new
+context.sax_parse(xml, handler)
+
+puts
+puts "=== Example 2: Block-Based Handler ==="
+puts
+
+element_count = 0
+context.sax_parse(xml) do
+  start_document { puts "Document started" }
+
+  start_element do |name, _attrs|
+    element_count += 1
+    puts "  Element #{element_count}: #{name}"
+  end
+
+  end_document { puts "Document ended - processed #{element_count} elements" }
+end

data/examples/web_scraper/README.md

@@ -0,0 +1,352 @@
+# Web Scraper Example
+
+This example demonstrates how to scrape data from HTML/XML documents using Moxml, showcasing table extraction, DOM navigation, and attribute access.
+
+## What This Example Demonstrates
+
+- **HTML Parsing**: Parsing HTML as XML for data extraction
+- **Table Scraping**: Extracting structured data from HTML tables
+- **DOM Navigation**: Traversing the document structure
+- **Attribute Access**: Reading element attributes and data attributes
+- **XPath Patterns**: Various XPath selectors for element selection
+- **Data Structuring**: Converting scraped data into Ruby objects
+
+## Files
+
+- `web_scraper.rb` - Main scraper implementation
+- `example_page.html` - Sample HTML page with programming language statistics
+- `README.md` - This file
+
+## Running the Example
+
+### Using the Example Page
+
+```bash
+ruby examples/web_scraper/web_scraper.rb
+```
+
+### Using Your Own HTML
+
+```bash
+ruby examples/web_scraper/web_scraper.rb path/to/your/page.html
+```
+
+## Expected Output
+
+```
+Scraping HTML page: examples/web_scraper/example_page.html
+================================================================================
+Programming Language Statistics Scraper
+================================================================================
+
+Page Title: Programming Language Statistics - 2024
+
+Summary:
+  10 total languages tracked
+  Last updated: October 30, 2024
+
+Languages Extracted: 10
+--------------------------------------------------------------------------------
+1. Python (Interpreted) - 95.5% | Created: 1991 | Uses: Data Science, Web, AI
+2. JavaScript (Interpreted) - 94.2% | Created: 1995 | Uses: Web Development
+3. Java (Compiled) - 89.7% | Created: 1995 | Uses: Enterprise, Android
+[...]
+
+Category Statistics:
+--------------------------------------------------------------------------------
+Interpreted: 3 languages, avg 85.0%, top: Python
+Compiled: 7 languages, avg 70.1%, top: Java
+
+Detailed Information:
+--------------------------------------------------------------------------------
+python:
+  Paradigm: Multi-paradigm: object-oriented, procedural, functional
+  Typing: Dynamic, strong
+  Community: Very large and active
+  Learning Curve: Beginner-friendly
+[...]
+
+XPath Pattern Demonstrations
+================================================================================
+1. All table headers (//th):
+   Found 12 headers: Rank, Language, Category, ...
+[...]
+```
+
+## Key Concepts
+
+### Table Scraping
+
+Extract data from HTML tables systematically:
+
+```ruby
+# Find table by ID
+table = doc.at_xpath("//table[@id='popularity-table']")
+
+# Get all rows
+rows = table.xpath('.//tbody/tr')
+
+# Extract cells from each row
+rows.each do |row|
+  cells = row.xpath('./td')
+  rank = cells[0].text.strip
+  name = cells[1].text.strip
+  # ...
+end
+```
+
+### Attribute Access
+
+Read element attributes using the `[]` operator:
+
+```ruby
+# Get data attribute
+score = cell['data-score']
+
+# Get class attribute
+class_name = element['class']
+
+# Check if attribute exists
+if row['data-language']
+  lang = row['data-language']
+end
+```
+
+### XPath Patterns
+
+The example demonstrates various XPath patterns:
+
+```ruby
+# By ID
+doc.at_xpath("//div[@id='summary']")
+
+# By class (contains for multi-class support)
+doc.xpath("//*[contains(@class, 'language-name')]")
+
+# By attribute existence
+doc.xpath("//tr[@data-language]")
+
+# Combining conditions
+doc.xpath("//div[contains(@class, 'stats-card') and @data-language]")
+
+# Direct descendants only
+element.xpath('./td')  # Not './/td'
+```
+
+### DOM Navigation
+
+Navigate the document tree:
+
+```ruby
+# Get parent
+parent = element.parent
+
+# Get children
+children = element.children
+
+# Get siblings
+next_elem = element.next_sibling
+prev_elem = element.previous_sibling
+```
+
+### Error Handling
+
+Handle parsing errors gracefully:
+
+```ruby
+begin
+  doc = @moxml.parse(html_content)
+rescue Moxml::ParseError => e
+  puts "Failed to parse HTML: #{e.message}"
+  exit 1
+end
+```
+
+## Code Structure
+
+### Language Class
+
+Represents a programming language with:
+- Rank, name, category
+- Popularity score
+- Year created
+- Primary use cases
+
+### CategoryStats Class
+
+Represents category statistics:
+- Category name
+- Language count
+- Average score
+- Top language
+
+### WebScraper Class
+
+Main scraper with methods:
+- `scrape` - Main scraping entry point
+- `extract_page_title` - Get page title
+- `extract_summary` - Extract summary statistics
+- `extract_languages_table` - Parse language table
+- `extract_category_stats` - Parse category table
+- `extract_detailed_info` - Parse detail cards
+
+## XPath Pattern Reference
+
+### Basic Selectors
+
+```ruby
+# All elements of a type
+doc.xpath('//div')
+
+# Element by ID
+doc.at_xpath("//div[@id='content']")
+
+# Element by class (single class)
+doc.xpath("//div[@class='card']")
+
+# Element by class (multiple classes)
+doc.xpath("//*[contains(@class, 'card')]")
+```
+
+### Attribute Selectors
+
+```ruby
+# Has attribute
+doc.xpath("//tr[@data-language]")
+
+# Attribute equals value
+doc.xpath("//input[@type='text']")
+
+# Attribute contains value
+doc.xpath("//div[contains(@class, 'active')]")
+```
+
+### Hierarchical Selectors
+
+```ruby
+# Direct child
+div.xpath('./p')  # Only direct <p> children
+
+# Any descendant
+div.xpath('.//p')  # All <p> descendants
+
+# Parent
+element.parent
+
+# Sibling
+element.next_sibling
+```
+
+### Combining Conditions
+
+```ruby
+# AND condition
+doc.xpath("//div[@class='card' and @id='main']")
+
+# Multiple conditions
+doc.xpath("//tr[contains(@class, 'row') and @data-id]")
+```
+
+## Customization
+
+### Scraping Different Tables
+
+Modify XPath selectors for your table structure:
+
+```ruby
+# Different table structure
+table = doc.at_xpath("//table[@class='data-table']")
+headers = table.xpath('.//thead/tr/th').map(&:text)
+rows = table.xpath('.//tbody/tr')
+```
+
+### Handling Complex HTML
+
+For nested structures:
+
+```ruby
+# Extract nested data
+card.xpath('.//div[@class="section"]').each do |section|
+  title = section.at_xpath('./h3').text
+  items = section.xpath('.//li').map(&:text)
+end
+```
+
+### Data Cleaning
+
+Clean extracted text:
+
+```ruby
+# Strip whitespace
+text = element.text.strip
+
+# Remove special characters
+text = text.gsub(/[^\w\s]/, '')
+
+# Parse numbers
+score = text.delete('%').to_f
+```
+
+## Learning Points
+
+1. **HTML as XML**: Well-formed HTML can be parsed as XML
+2. **XPath is powerful**: One query can find many elements
+3. **Attributes are key**: Use data attributes for reliable scraping
+4. **Structure matters**: Understand the DOM structure before scraping
+5. **Clean data**: Always clean and validate scraped data
+6. **Error handling**: Handle missing elements gracefully
+
+## Best Practices
+
+1. **Use specific selectors**: Prefer IDs over classes when available
+2. **Validate data**: Check for nil/empty values
+3. **Handle errors**: Wrap parsing in begin/rescue blocks
+4. **Clean text**: Strip whitespace and normalize data
+5. **Document structure**: Understand the HTML before writing XPath
+6. **Test thoroughly**: Test with different HTML structures
+
+## Common Issues
+
+### Issue: Element not found
+
+```ruby
+# Bad - will raise error if not found
+title = doc.xpath('//title').first.text
+
+# Good - safe navigation
+title = doc.at_xpath('//title')&.text || 'Unknown'
+```
+
+### Issue: Incorrect XPath
+
+```ruby
+# Wrong - searches entire document
+row.xpath('//td')
+
+# Correct - searches within row only
+row.xpath('./td')
+```
+
+### Issue: Class matching
+
+```ruby
+# Won't work with multiple classes
+div.xpath("//div[@class='card active']")
+
+# Works with multiple classes
+div.xpath("//div[contains(@class, 'card')]")
+```
+
+## Next Steps
+
+- Scrape real websites (check robots.txt and terms of service)
+- Add data export (CSV, JSON)
+- Implement pagination handling
+- Add retry logic for failed requests
+- Create scrapers for different domains
+- Implement data validation
+
+## Related Examples
+
+- [RSS Parser](../rss_parser/) - Similar XPath techniques for RSS
+- [API Client](../api_client/) - XML generation and parsing

data/examples/web_scraper/example_page.html

@@ -0,0 +1,201 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8"/>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
+    <title>Programming Language Statistics - 2024</title>
+    <style>
+        body {
+            font-family: Arial, sans-serif;
+            max-width: 1200px;
+            margin: 0 auto;
+            padding: 20px;
+        }
+        table {
+            width: 100%;
+            border-collapse: collapse;
+            margin: 20px 0;
+        }
+        th, td {
+            border: 1px solid #ddd;
+            padding: 12px;
+            text-align: left;
+        }
+        th {
+            background-color: #4CAF50;
+            color: white;
+        }
+        tr:nth-child(even) {
+            background-color: #f2f2f2;
+        }
+        .stats-card {
+            background-color: #f9f9f9;
+            border: 1px solid #ddd;
+            border-radius: 5px;
+            padding: 15px;
+            margin: 10px 0;
+        }
+        .language-name {
+            font-weight: bold;
+            color: #333;
+        }
+    </style>
+</head>
+<body>
+    <h1>Programming Language Statistics - 2024</h1>
+
+    <div class="stats-card" id="summary">
+        <h2>Summary</h2>
+        <p>Total languages tracked: <span class="stat-value">10</span></p>
+        <p>Data last updated: <span class="stat-value">October 30, 2024</span></p>
+    </div>
+
+    <h2>Most Popular Programming Languages</h2>
+    <table id="popularity-table">
+        <thead>
+            <tr>
+                <th>Rank</th>
+                <th>Language</th>
+                <th>Category</th>
+                <th>Popularity Score</th>
+                <th>Year Created</th>
+                <th>Primary Use</th>
+            </tr>
+        </thead>
+        <tbody>
+            <tr data-language="python">
+                <td>1</td>
+                <td class="language-name">Python</td>
+                <td>Interpreted</td>
+                <td data-score="95.5">95.5%</td>
+                <td>1991</td>
+                <td>Data Science, Web, AI</td>
+            </tr>
+            <tr data-language="javascript">
+                <td>2</td>
+                <td class="language-name">JavaScript</td>
+                <td>Interpreted</td>
+                <td data-score="94.2">94.2%</td>
+                <td>1995</td>
+                <td>Web Development</td>
+            </tr>
+            <tr data-language="java">
+                <td>3</td>
+                <td class="language-name">Java</td>
+                <td>Compiled</td>
+                <td data-score="89.7">89.7%</td>
+                <td>1995</td>
+                <td>Enterprise, Android</td>
+            </tr>
+            <tr data-language="csharp">
+                <td>4</td>
+                <td class="language-name">C#</td>
+                <td>Compiled</td>
+                <td data-score="82.3">82.3%</td>
+                <td>2000</td>
+                <td>.NET, Games, Enterprise</td>
+            </tr>
+            <tr data-language="cpp">
+                <td>5</td>
+                <td class="language-name">C++</td>
+                <td>Compiled</td>
+                <td data-score="78.9">78.9%</td>
+                <td>1985</td>
+                <td>Systems, Games, Performance</td>
+            </tr>
+            <tr data-language="ruby">
+                <td>6</td>
+                <td class="language-name">Ruby</td>
+                <td>Interpreted</td>
+                <td data-score="65.4">65.4%</td>
+                <td>1995</td>
+                <td>Web Development, Scripting</td>
+            </tr>
+            <tr data-language="go">
+                <td>7</td>
+                <td class="language-name">Go</td>
+                <td>Compiled</td>
+                <td data-score="63.8">63.8%</td>
+                <td>2009</td>
+                <td>Cloud, Microservices</td>
+            </tr>
+            <tr data-language="rust">
+                <td>8</td>
+                <td class="language-name">Rust</td>
+                <td>Compiled</td>
+                <td data-score="61.2">61.2%</td>
+                <td>2010</td>
+                <td>Systems, WebAssembly</td>
+            </tr>
+            <tr data-language="swift">
+                <td>9</td>
+                <td class="language-name">Swift</td>
+                <td>Compiled</td>
+                <td data-score="58.7">58.7%</td>
+                <td>2014</td>
+                <td>iOS, macOS Development</td>
+            </tr>
+            <tr data-language="kotlin">
+                <td>10</td>
+                <td class="language-name">Kotlin</td>
+                <td>Compiled</td>
+                <td data-score="56.3">56.3%</td>
+                <td>2011</td>
+                <td>Android, JVM</td>
+            </tr>
+        </tbody>
+    </table>
+
+    <h2>Language Statistics by Category</h2>
+    <table id="category-table">
+        <thead>
+            <tr>
+                <th>Category</th>
+                <th>Count</th>
+                <th>Average Score</th>
+                <th>Top Language</th>
+            </tr>
+        </thead>
+        <tbody>
+            <tr>
+                <td>Interpreted</td>
+                <td>3</td>
+                <td>85.0%</td>
+                <td>Python</td>
+            </tr>
+            <tr>
+                <td>Compiled</td>
+                <td>7</td>
+                <td>70.1%</td>
+                <td>Java</td>
+            </tr>
+        </tbody>
+    </table>
+
+    <h2>Detailed Language Information</h2>
+    <div class="stats-card" data-language="python">
+        <h3 class="language-name">Python</h3>
+        <ul>
+            <li><strong>Paradigm:</strong> Multi-paradigm: object-oriented, procedural, functional</li>
+            <li><strong>Typing:</strong> Dynamic, strong</li>
+            <li><strong>Community:</strong> Very large and active</li>
+            <li><strong>Learning Curve:</strong> Beginner-friendly</li>
+        </ul>
+    </div>
+
+    <div class="stats-card" data-language="ruby">
+        <h3 class="language-name">Ruby</h3>
+        <ul>
+            <li><strong>Paradigm:</strong> Multi-paradigm: object-oriented, functional</li>
+            <li><strong>Typing:</strong> Dynamic, strong</li>
+            <li><strong>Community:</strong> Active, focused on web development</li>
+            <li><strong>Learning Curve:</strong> Beginner to intermediate</li>
+        </ul>
+    </div>
+
+    <footer>
+        <p><em>Data compiled from various programming language popularity indices</em></p>
+        <p>Contact: <a href="mailto:info@example.com">info@example.com</a></p>
+    </footer>
+</body>
+</html>