rss_feed_plus 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: ec579256585d8f4a9ddf84497d214779d61c605cbf443bd1e55e4f094204cdc5
+  data.tar.gz: 0bf2827e8bfab1aa2806578303ff04a570991c97a61673dc0edd8c9bb82c57e1
+SHA512:
+  metadata.gz: 566a7978173b0d7527a6413d80f6f0a2245633c51b7dc2950346e0b1b81ba10d465f906887f5e0b1fa5b06870c657e7426f47a3ac7e1bd989aa739daf3dfac7f
+  data.tar.gz: 1f0c46d97d326cbb2aa65dced535beee715b58b9570e7ced3f2d4cb36d5d876ac25c8d16cd17884b383171cdcd0dae34927c90d20e40cc1f1952b17c4bba3f23

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.rubocop.yml

@@ -0,0 +1,11 @@
+require:
+  - rubocop-performance
+  - rubocop-rspec
+
+AllCops:
+  TargetRubyVersion: 2.6
+  NewCops: enable
+Style/FrozenStringLiteralComment:
+  EnforcedStyle: never
+Style/Documentation:
+  Enabled: false

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+## [Unreleased]
+
+## [0.1.0] - 2024-03-16
+
+- Initial release

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2024 talaatmagdyx
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,82 @@
+# rss_feed_plus
+
+## Introduction
+
+**rss_feed_plus** is your go-to Ruby gem for effortlessly fetching and parsing RSS feeds. Whether you're building a news aggregator, content management system, or simply want to integrate RSS feeds into your application, **rss_feed_plus** simplifies the process, allowing you to easily retrieve and process RSS feed data from various sources.
+
+## Features
+
+- **Effortless Parsing**: Fetch and parse RSS feeds with ease.
+- **Customization: Tailor** parsing to fit your needs with customizable XML and URI parsers and timeout duration.
+- **Seamless Integration**: Integrate with Ruby applications smoothly.
+
+## Installation
+
+Getting started with **rss_feed_plus** is quick and easy. Simply add the gem to your application's Gemfile:
+
+```ruby
+gem 'rss_feed_plus'
+```
+
+Then, install the gem by running:
+
+```bash
+bundle install
+```
+
+Alternatively, you can install the gem directly using RubyGems:
+
+```bash
+gem install rss_feed_plus
+```
+
+## Usage
+
+Here's a basic example of how to use **rss_feed_plus** to fetch and parse RSS feeds:
+
+```ruby
+require 'rss_feed'
+require 'nokogiri'
+
+# Define your custom options
+feed_urls = 'https://www.ruby-lang.org/en/feeds/news.rss'
+xml_parser = Nokogiri
+uri_parser = URI
+timeout = 10
+
+# Initialize the Parser class with custom options
+parser = RssFeed::Parser.new(feed_urls, xml_parser: xml_parser, uri_parser: uri_parser, timeout: timeout)
+# or 
+parser = RssFeed::Parser.new(feed_urls)
+# Parse the RSS feeds
+parsed_data = parser.parse_as_object
+
+# Process the parsed data
+puts parsed_data.inspect
+```
+
+## Customization
+
+**rss_feed_plus** allows you to tailor the parsing process to fit your needs. Customize the XML parser, URI parser, and timeout duration according to your requirements.
+
+## Contributing
+
+Contributions to **rss_feed_plus** are welcome! If you encounter any issues, have feature requests, or would like to contribute enhancements, please feel free to open an issue or submit a pull request on [GitHub](https://github.com/talaatmagdyx/rss_feed_plus).
+
+Before contributing, please review the [Contributing Guidelines](https://github.com/talaatmagdyx/rss_feed_plus/blob/master/.github/CONTRIBUTING.md) and adhere to the [Code of Conduct](https://github.com/talaatmagdyx/rss_feed_plus/blob/master/.github/CODE_OF_CONDUCT.md).
+
+## Reporting Bugs / Feature Requests
+
+If you encounter any bugs or have suggestions for new features, please [open an issue on GitHub](https://github.com/talaatmagdyx/rss_feed_plus/issues). Your feedback is valuable and helps improve the quality of the gem.
+
+## License
+
+**rss_feed_plus** is released under the [MIT License](https://opensource.org/licenses/MIT). You are free to use, modify, and distribute the gem according to the terms of the license.
+
+## Code of Conduct
+
+Please review and adhere to the [Code of Conduct](https://github.com/talaatmagdyx/rss_feed_plus/blob/master/.github/CODE_OF_CONDUCT.md) when interacting with the **rss_feed_plus** project. We strive to maintain a welcoming and inclusive community for all contributors and users.
+
+---
+
+Experience the simplicity of RSS feed integration with **rss_feed_plus**. Happy coding!

data/Rakefile

@@ -0,0 +1,6 @@
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec

data/lib/rss_feed/dynamic_object.rb

@@ -0,0 +1,58 @@
+class DynamicObject
+  ## Example usage:
+  # data = {
+  #   name: "John",
+  #   age: 30,
+  #   address: {
+  #     city: "New York",
+  #     state: "NY"
+  #   },
+  #   hobbies: ["reading", "hiking"]
+  # }
+  #
+  # dynamic_object = DynamicObject.new(data)
+  #
+  # puts dynamic_object.name   # Output: John
+  # puts dynamic_object.age    # Output: 30
+  # puts dynamic_object.address.city   # Output: New York
+  # puts dynamic_object.address.state  # Output: NY
+  # puts dynamic_object.hobbies   # Output: ["reading", "hiking"]
+
+  # A class for initializing objects dynamically based on given data.
+  # Initializes a new instance of DynamicInitializer.
+  #
+  # @param data [Hash] The data used to initialize the object.
+  # @return [DynamicInitializer] A new instance of DynamicInitializer.
+  def initialize(data)
+    data.each do |key, value|
+      key = key.to_s
+      set_instance_variable(key, value)
+      define_singleton_method(key.tr(':', '_')) { instance_variable_get("@#{key.tr(':', '_')}") }
+    end
+  end
+
+  private
+
+  # Sets an instance variable based on the provided key and value.
+  #
+  # @param key [String] The key for the instance variable.
+  # @param value [Object] The value to be assigned to the instance variable.
+  # @return [void]
+  def set_instance_variable(key, value)
+    if value.is_a?(Hash)
+      instance_variable_set("@#{key.tr(':', '_')}", DynamicObject.new(value))
+    elsif value.is_a?(Array)
+      instance_variable_set("@#{key}", process_array(value))
+    else
+      instance_variable_set("@#{key}", value)
+    end
+  end
+
+  # Processes an array, creating DynamicObject instances if elements are hashes.
+  #
+  # @param array [Array] The array to be processed.
+  # @return [Array] The processed array.
+  def process_array(array)
+    array.map { |v| v.is_a?(Hash) ? DynamicObject.new(v) : v }
+  end
+end

data/lib/rss_feed/feed/base.rb

@@ -0,0 +1,41 @@
+module RssFeed
+  module Feed
+    # The Base class serves as the base class for all feed parsers.
+    class Base
+      attr_reader :document
+
+      # Initializes a new Base instance.
+      #
+      # @param document [Nokogiri::XML::Document] The parsed XML document.
+      def initialize(document)
+        @document = document
+      end
+
+      # This method should be implemented by subclasses to define the specific parsing logic.
+      #
+      # @abstract
+      def parser
+        raise NotImplementedError
+      end
+
+      private
+
+      # Detects the type of the feed based on the root element of the XML document.
+      #
+      # @return [String] The name of the root element.
+      def detect_feed_type
+        document.root.name
+      end
+
+      # Executes the appropriate parsing method based on the detected feed type.
+      #
+      # @raise [NotImplementedError] If the parsing method for the detected feed type is not implemented.
+      def execute_method
+        method_name = detect_feed_type
+        raise NotImplementedError unless respond_to?(method_name)
+
+        send(method_name)
+      end
+    end
+  end
+end

data/lib/rss_feed/feed/channel.rb

@@ -0,0 +1,47 @@
+require_relative 'base'
+require_relative '../object'
+
+module RssFeed
+  module Feed
+    # The Channel class represents a channel in an RSS feed.
+    class Channel < Base
+      # List of commonly used tags in an RSS channel.
+      TAGS = %w[
+        id
+        title subtitle link
+        description
+        author webMaster managingEditor contributor
+        pubDate lastBuildDate updated dc:date
+        generator language docs cloud
+        ttl skipHours skipDays
+        image logo icon rating
+        rights copyright
+        textInput feedburner:browserFriendly
+        itunes:author itunes:category
+      ].freeze
+
+      # XPath expression for selecting the RSS channel.
+      def rss
+        return nil if document.blank?
+
+        '//channel'
+      end
+
+      # XPath expression for selecting the Atom feed.
+      def atom
+        return nil if document.blank?
+
+        '//feed'
+      end
+
+      alias feed atom
+
+      # Parses the RSS channel or Atom feed based on the detected feed type.
+      #
+      # @return [Nokogiri::XML::NodeSet, nil] The parsed channel or feed, or nil if not found.
+      def parse
+        document.xpath(execute_method)
+      end
+    end
+  end
+end

data/lib/rss_feed/feed/item.rb

@@ -0,0 +1,42 @@
+require_relative 'base'
+
+module RssFeed
+  module Feed
+    # The Item class represents an item in an RSS feed.
+    class Item < Base
+      # List of commonly used tags in an RSS item.
+      TAGS = %w[
+        id
+        title link link+alternate link+self link+edit link+replies
+        author contributor
+        description summary content content:encoded comments
+        pubDate published updated expirationDate modified dc:date
+        category guid
+        trackback:ping trackback:about
+        dc:creator dc:title dc:subject dc:rights dc:publisher
+        feedburner:origLink media:content media:thumbnail
+        media:title
+        media:credit
+        media:category
+      ].freeze
+
+      # XPath expression for selecting the RSS item.
+      def rss
+        '//item'
+      end
+
+      # XPath expression for selecting the Atom entry.
+      def atom
+        '//entry'
+      end
+
+      alias feed atom
+      # Parses the RSS item or Atom entry based on the detected feed type.
+      #
+      # @return [Nokogiri::XML::NodeSet] The parsed item or entry.
+      def parse
+        document.xpath(execute_method)
+      end
+    end
+  end
+end

data/lib/rss_feed/feed/namespace.rb

@@ -0,0 +1,75 @@
+require 'cgi'
+require_relative '../object'
+
+module RssFeed
+  module Feed
+    # The Namespace module provides utility methods for accessing and manipulating XML namespaces in RSS feeds.
+    class Namespace
+      # Mapping of namespace prefixes to their corresponding URIs.
+      NAMESPACES = {
+        'itunes' => 'http://www.itunes.com/dtds/podcast-1.0.dtd',
+        'dc' => 'http://purl.org/dc/elements/1.1/',
+        'feedburner' => 'http://rssnamespace.org/feedburner/ext/1.0',
+        'content' => 'http://purl.org/rss/1.0/modules/content/',
+        'trackback' => 'http://example.com/trackback',
+        'media' => 'http://search.yahoo.com/mrss/'
+      }.freeze
+
+      class << self
+        # Accesses the specified XML tag within the document with proper namespace handling.
+        #
+        # @param tag [String] The XML tag to access.
+        # @param doc [Nokogiri::XML::Document] The XML document.
+        # @return [Hash] The tag data including text, nested elements flag, nested attributes flag, and the document.
+        def access_tag(tag, doc)
+          doc = doc.xpath(tag, namespace(tag))
+          nested_elements = nested_elements?(doc)
+          { text: doc.to_s, nested_elements: nested_elements, nested_attributes: nested_attributes?(doc), docs: doc }
+        end
+
+        # Resolves the namespace for the given XML tag.
+        #
+        # @param tag [String] The XML tag.
+        # @return [Hash] The namespace declaration.
+        def namespace(tag)
+          namespace_key = tag.split(':').first
+          { namespace_key.to_s => NAMESPACES[namespace_key] }.compact
+        end
+
+        # Removes HTML tags from the given content.
+        #
+        # @param content [String] The content containing HTML tags.
+        # @return [String] The content without HTML tags.
+        def remove_html_tags(content)
+          if %r{([^-_.!~*'()a-zA-Z\d;/?:@&=+$,\[\]]%)}.match?(content)
+            CGI.unescape(content)
+          else
+            content
+          end.gsub(/(<!\[CDATA\[|\]\]>)/, '').strip.gsub(/<[^>]+>/, '')
+        end
+
+        # Checks if the XML node has nested elements.
+        #
+        # @param node [Nokogiri::XML::NodeSet] The XML node.
+        # @return [Boolean] Whether the node has nested elements.
+        def nested_elements?(node)
+          return false if node.blank? || node.to_s == 'NaN'
+
+          return true if node.children.any?(&:element?)
+
+          false
+        end
+
+        # Checks if the XML node has nested attributes.
+        #
+        # @param node [Nokogiri::XML::NodeSet] The XML node.
+        # @return [Boolean] Whether the node has nested attributes.
+        def nested_attributes?(node)
+          return false if node.blank? || node.to_s == 'NaN'
+
+          node.any? { |thumbnail| !thumbnail.attributes.empty? }
+        end
+      end
+    end
+  end
+end

data/lib/rss_feed/object.rb

@@ -0,0 +1,47 @@
+# Monkey-patching the Object class to add convenience methods for checking presence or absence of content.
+class Object
+  # An object is present if it's not blank.
+  #
+  # @return [true, false].
+  def present?
+    !blank?
+  end
+
+  # An object is blank if it's false, empty, or a whitespace string.
+  # For example, +nil+, '', '   ', [], {}, and +false+ are all blank.
+  #
+  # This simplifies
+  #
+  #   !address || address.empty?
+  #
+  # to
+  #
+  #   address.blank?
+  #
+  # @return [true, false]
+  def blank?
+    # If the object responds to `empty?`, checks if it's empty or equals 'NaN'.
+    # Otherwise, checks if the object is nil.
+    respond_to?(:empty?) ? (empty? || self == 'NaN') : !self
+  end
+
+  # Returns the receiver if it's present otherwise returns +nil+.
+  # <tt>object.presence</tt> is equivalent to
+  #
+  #    object.present? ? object : nil
+  #
+  # For example, something like
+  #
+  #   state   = params[:state]   if params[:state].present?
+  #   country = params[:country] if params[:country].present?
+  #   region  = state || country || 'US'
+  #
+  # becomes
+  #
+  #   region = params[:state].presence || params[:country].presence || 'US'
+  #
+  # @return [Object]
+  def presence
+    self if present?
+  end
+end

data/lib/rss_feed/parser.rb

@@ -0,0 +1,209 @@
+require 'nokogiri'
+require 'open-uri'
+require 'socket'
+
+require_relative 'feed/channel'
+require_relative 'feed/item'
+require_relative 'feed/namespace'
+require_relative '../rss_feed/object'
+require_relative '../rss_feed/dynamic_object'
+
+module RssFeed
+  # The Parser class is responsible for parsing RSS feeds.
+  class Parser
+    attr_reader :feed_urls, :xml_parser, :uri_parser
+
+    # Initialize the Parser with a list of feed URLs.
+    #
+    # @param feed_urls String The URLs of the RSS feeds to parse.
+    def initialize(feed_urls, options = {})
+      @feed_urls = feed_urls
+      @xml_parser = options.fetch(:xml_parser, Nokogiri)
+      @uri_parser = options.fetch(:uri_parser, URI)
+      @timeout = options.fetch(:timeout, 10) # Default timeout: 10 seconds
+      @logger = options[:logger]
+    end
+
+    # Parse the RSS feeds and extract channel and item information.
+    #
+    # @return [Hash] The parsed channel and item information.
+    def parse
+      document = fetch_and_parse_xml(feed_urls)
+      channel = RssFeed::Feed::Channel.new(document)
+      channel_info = extract_channel_info(channel)
+
+      items = RssFeed::Feed::Item.new(document)
+      item_info = extract_item_info(items)
+
+      { 'channel' => channel_info, 'items' => item_info }
+    end
+
+    def parse_as_object
+      DynamicObject.new(parse)
+    end
+
+    private
+
+    # Fetch and parse XML data from the given URL.
+    #
+    # @param url [String] The URL of the XML data.
+    # @return [Nokogiri::XML::Document] The parsed XML document.
+    # def fetch_and_parse_xml(url)
+    #   rss_data = uri_parser.parse(url).open
+    #   @xml_parser::XML(rss_data)
+    # rescue StandardError => e
+    #   handle_error(e)
+    #   raise RssFetchError, "Failed to fetch or parse XML: #{e.message}"
+    # end
+    def fetch_and_parse_xml(url)
+      rss_data = URI.parse(url).open(**uri_options)
+      @xml_parser::XML(rss_data)
+    rescue SocketError, URI::InvalidURIError => e
+      raise RssFetchError, "Failed to fetch or parse XML: #{e.message}"
+    rescue Timeout::Error => e
+      raise RssFetchError, "HTTP request timed out: #{e.message}"
+    end
+
+    def uri_options
+      { open_timeout: @timeout, read_timeout: @timeout }.compact
+    end
+
+    # Extract channel information from the parsed XML document.
+    #
+    # @param channel [RssFeed::Feed::Channel] The channel object.
+    # @return [Hash] The extracted channel information.
+    def extract_channel_info(channel)
+      extract_info(channel, channel.parse)
+    end
+
+    # Extract item information from the parsed XML document.
+    #
+    # @param items [RssFeed::Feed::Item] The items object.
+    # @return [Array<Hash>] The extracted item information.
+    def extract_item_info(items)
+      items.parse.map { |item| extract_info(items, item) }
+    end
+
+    # Extract information from the XML document based on specified tags.
+    #
+    # @param feed [RssFeed::Feed::Channel/RssFeed::Feed::Item] The feed object.
+    # @param feed_parse [Hash] The parsed XML data.
+    # @return [Hash] The extracted information.
+    def extract_info(feed, feed_parse)
+      item_data = {}
+
+      feed.class::TAGS.each do |tag|
+        tag_data = extract_tag_data(tag, feed_parse)
+        next if skip_extraction?(tag_data)
+
+        items = extract_items(tag_data)
+        next if skip_items?(items, tag_data[:nested_attributes])
+
+        item_data[tag] = create_item_info(items, tag_data)
+      end
+
+      item_data
+    end
+
+    # Check if extraction of tag data should be skipped.
+    #
+    # @param tag_data [Hash] The tag data.
+    # @return [Boolean] True if extraction should be skipped, otherwise false.
+    def skip_extraction?(tag_data)
+      tag_data.values_at(:text, :nested_elements, :nested_attributes).all?(&:blank?)
+    end
+
+    # Check if extraction of items should be skipped.
+    #
+    # @param items [Object] The items to check.
+    # @param nested_attributes [Boolean] Whether the items have nested attributes.
+    # @return [Boolean] True if extraction should be skipped, otherwise false.
+    def skip_items?(items, nested_attributes)
+      items.blank? && nested_attributes.blank?
+    end
+
+    # Create item information hash.
+    #
+    # @param items [Object] The items data.
+    # @param tag_data [Hash] The tag data.
+    # @return [Hash] The item information.
+    def create_item_info(items, tag_data)
+      { 'values' => items, 'attributes' => tag_data[:attributes] }.compact
+    end
+
+    # Extract tag data from the XML document.
+    #
+    # @param tag [String] The tag to extract.
+    # @param feed_parse [Hash] The parsed XML data.
+    # @return [Hash] The extracted tag data.
+    def extract_tag_data(tag, feed_parse)
+      value = RssFeed::Feed::Namespace.access_tag(tag, feed_parse)
+      value[:attributes] = extract_attributes(value[:docs]) if value[:nested_attributes]
+      value
+    end
+
+    # Extract items from the XML document.
+    #
+    # @param tag_data [Hash] The tag data.
+    # @return [Object] The extracted items.
+    def extract_items(tag_data)
+      tag_data[:nested_elements] ? extract_nested_data(tag_data[:docs]) : extract_clean_value(tag_data[:text])
+    end
+
+    # Add attributes to the item information hash.
+    #
+    # @param tag_item [Hash] The item information hash.
+    # @param tag_data [Hash] The tag data.
+    def add_attributes(tag_item, tag_data)
+      tag_item['attributes'] = tag_data[:attributes] if tag_data[:attributes].present?
+    end
+
+    # Extract clean value from the XML document.
+    #
+    # @param docs [Object] The XML document.
+    # @return [String] The extracted clean value.
+    def extract_clean_value(docs)
+      RssFeed::Feed::Namespace.remove_html_tags(docs).presence
+    end
+
+    # Extract nested data from the XML document.
+    #
+    # @param nodes [Object] The XML nodes.
+    # @return [Hash] The extracted nested data.
+    def extract_nested_data(nodes)
+      nodes.each_with_object({}) do |node, nested_data|
+        node.children.each do |child|
+          child_value = RssFeed::Feed::Namespace.remove_html_tags(child.text)
+          nested_data[child.name.to_sym] = child_value if child_value.present?
+        end
+      end
+    end
+
+    # Extract attributes from the XML document.
+    #
+    # @param node [Object] The XML node.
+    # @return [Array<Hash>] The extracted attributes.
+    def extract_attributes(node)
+      node.map do |thumbnail|
+        attributes_hash = {}
+        thumbnail.attributes.each do |name, value|
+          attributes_hash[name.to_s] = value.to_s
+        end
+        attributes_hash
+      end
+    end
+
+    def handle_error(error)
+      error_message = "Error occurred: #{error.message}"
+      # Fallback to puts if logger is not configured
+      @logger.present? ? @logger.error(error_message) : puts(error_message)
+    end
+
+    def configure_logger
+      @logger ||= Logger.new($stdout)
+      @logger.level = Logger::INFO
+    end
+  end
+
+  class RssFetchError < StandardError; end
+end

data/lib/rss_feed/version.rb

@@ -0,0 +1,3 @@
+module RssFeed
+  VERSION = '0.1.0'.freeze
+end

data/lib/rss_feed.rb

@@ -0,0 +1,5 @@
+require_relative 'rss_feed/version'
+
+module RssFeed
+  autoload :Parser, 'rss_feed/parser'
+end