web-author 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 7b0531fe79190a66e3119213938b8cf6236bf104b2d2f94efb7f955c149befcf
+  data.tar.gz: 57ea9eafe75368e6d28d8ea631690c44c682d677cf9b3ea49c1780d1c811619b
+SHA512:
+  metadata.gz: c15ab0e076f4a1956dbddf3be9f2fe918fccb01966650427fe32e292d65783cffa857e88295d28878f878cd36a215f1c3a1469848d608503fc72270cbf2f8a91
+  data.tar.gz: cac9b4fb83ecbf69c176a2d4e365876c9290fcf429141a0923afa49e6140999226a8c06d4058cdde1eaeb6d266b5bd681a896001e232f6448ea4b184e77aab59

data/.ruby-version

@@ -0,0 +1 @@
+3.4.1

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+# [Releases]
+
+## [0.1.0] - 2025-04-14
+
+- Initial release with support for meta tags and JSON-LD schema

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 Lucian Ghinda
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,234 @@
+# WebAuthor
+
+WebAuthor is a Ruby gem that extracts author information from web pages using multiple strategies. It can detect authors from both meta tags and JSON-LD schema, providing a reliable way to identify content creators.
+
+## Features
+
+- Extract author information from HTML meta tags
+- Extract author information from JSON-LD schema (schema.org)
+- Support for multiple authors in a single page
+- Fallback strategy - tries different methods until an author is found
+- Clean, type-safe code with Sorbet
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'web-author'
+```
+
+And then execute:
+
+```bash
+$ bundle install
+```
+
+Or install it yourself as:
+
+```bash
+$ gem install web-author
+```
+
+## Usage
+
+### Basic Usage
+
+```ruby
+require 'web_author'
+
+# Create a new Page object with a URL
+page = WebAuthor::Page.new(url: 'https://example.com/article')
+
+# Get the author of the page
+author = page.author
+# => "John Doe"
+```
+
+WebAuthor will first try to find author information in JSON-LD schema data, then fall back to meta tags if needed.
+
+### Handling Multiple Authors
+
+If a page has multiple authors in the JSON-LD schema, WebAuthor returns them as a comma-separated string:
+
+```ruby
+page = WebAuthor::Page.new(url: 'https://example.com/collaboration-article')
+authors = page.author
+# => "Jane Smith, Bob Johnson"
+```
+
+### Error Handling
+
+WebAuthor raises `WebAuthor::Error` when it encounters problems fetching the page:
+
+```ruby
+begin
+  page = WebAuthor::Page.new(url: 'https://example.com/article')
+  author = page.author
+rescue WebAuthor::Error => e
+  puts "Failed to get author: #{e.message}"
+end
+```
+
+## How It Works
+
+WebAuthor uses a strategy to extract author information:
+
+1. First, it tries to find author information in JSON-LD schema (often found in `<script type="application/ld+json">` tags)
+2. If no author is found in JSON-LD, it looks for a meta tag with the name "author" (`<meta name="author" content="Author Name">`)
+3. If no author is found using any strategy, it returns `nil`
+
+## Supported Author Formats
+
+### Meta Tags
+
+```html
+<meta name="author" content="Author Name" />
+```
+
+### JSON-LD Schema
+
+Single author:
+
+```html
+<script type="application/ld+json">
+  {
+    "@context": "https://schema.org",
+    "@type": "Article",
+    "author": {
+      "@type": "Person",
+      "name": "Author Name"
+    }
+  }
+</script>
+```
+
+Multiple authors:
+
+```html
+<script type="application/ld+json">
+  {
+    "@context": "https://schema.org",
+    "@type": "Article",
+    "author": [
+      {
+        "@type": "Person",
+        "name": "First Author"
+      },
+      {
+        "@type": "Person",
+        "name": "Second Author"
+      }
+    ]
+  }
+</script>
+```
+
+## Requirements
+
+- Ruby 3.4 or higher
+- Nokogiri
+- Sorbet Runtime
+- Zeitwerk
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`.
+
+### Development Workflow
+
+#### Type Checking with Sorbet
+
+This project uses [Sorbet](https://sorbet.org/) for static type checking. To run the type checker:
+
+```bash
+$ bin/type-check
+```
+
+or directly:
+
+```bash
+$ bundle exec srb tc
+```
+
+#### Running Tests
+
+Run all tests using:
+
+```bash
+$ bundle exec rake test
+```
+
+Run a specific test file:
+
+```bash
+$ bundle exec ruby -Ilib:test test/web_author/page_test.rb
+```
+
+#### Code Style and Linting
+
+This project follows Ruby style guidelines enforced by RuboCop. Run the linter with:
+
+```bash
+$ bundle exec rubocop
+```
+
+Auto-fix issues when possible:
+
+```bash
+$ bundle exec rubocop -A
+```
+
+#### Running All Checks
+
+The default Rake task runs both tests and RuboCop:
+
+```bash
+$ bundle exec rake
+```
+
+### Working with Sorbet
+
+WebAuthor uses Sorbet for static type checking. When adding new code:
+
+1. Add comment on top of the file: `# typed: strict`
+2. Add type signatures to methods using `sig` blocks
+3. Run `bin/type-check` to verify type safety
+
+Example of typed code:
+
+```ruby
+extend T::Sig
+
+sig { params(url: String).void }
+def initialize(url:)
+  @url = T.let(url, String)
+  @content = T.let(nil, T.nilable(String))
+end
+
+sig { returns(T.nilable(String)) }
+def author
+  # method implementation
+end
+```
+
+### Adding a new strategy
+
+You should create a new class that inherits from `WebAuthor::Strategy` and implement the `author` method.
+You will notice that you will get the `document` from the initializer as every strategy receives it. This is a `Nokogiri::XML::Document` object.
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/lucianghinda/web_author.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require 'bundler/gem_tasks'
+require 'minitest/test_task'
+
+Minitest::TestTask.create
+
+require 'rubocop/rake_task'
+
+RuboCop::RakeTask.new
+
+task default: %i(test rubocop)

data/lib/web_author/author/strategies/author_from_ld_schema.rb

@@ -0,0 +1,54 @@
+# typed: strict
+# frozen_string_literal: true
+
+module WebAuthor
+  module Author
+    module Strategies
+      class AuthorFromLdSchema < Strategy
+        extend T::Sig
+
+        sig { override.params(document: Nokogiri::XML::Document).void }
+        def initialize(document)
+          @_schemas = T.let(nil, T.nilable(T::Array[LdSchema]))
+          super
+        end
+
+        sig { override.returns(T.nilable(String)) }
+        def author
+          return nil if schemas.empty?
+
+          all_author_names = author_names
+          return nil if all_author_names.empty?
+
+          all_author_names.uniq.join(', ')
+        end
+
+        private
+
+          sig { returns(T::Array[LdSchema]) }
+          def schemas
+            @_schemas ||= JsonLdSchemaProcessor.new(document:).schemas
+          end
+
+          sig { returns(T::Array[String]) }
+          def author_names
+            names = []
+            schemas.each do |schema|
+              author = schema.parsed_author
+              next if author.nil?
+
+              current_names = if author.is_a?(Array)
+                                author.filter_map(&:name)
+                              else
+                                [author.name].compact
+                              end
+
+              names.concat(current_names) unless current_names.empty?
+            end
+
+            names
+          end
+      end
+    end
+  end
+end

data/lib/web_author/author/strategies/author_from_meta.rb

@@ -0,0 +1,16 @@
+# typed: strict
+# frozen_string_literal: true
+
+module WebAuthor
+  module Author
+    module Strategies
+      class AuthorFromMeta < Strategy
+        sig { override.returns(T.nilable(String)) }
+        def author
+          meta_author = document.at_css('meta[name="author"]')
+          meta_author&.attribute('content')&.value
+        end
+      end
+    end
+  end
+end

data/lib/web_author/author/strategy.rb

@@ -0,0 +1,25 @@
+# typed: strict
+# frozen_string_literal: true
+
+module WebAuthor
+  module Author
+    class Strategy
+      extend T::Sig
+      extend T::Helpers
+      abstract!
+
+      sig { overridable.params(document: Nokogiri::XML::Document).void }
+      def initialize(document)
+        @document = document
+      end
+
+      sig { abstract.returns(T.nilable(String)) }
+      def author; end
+
+      private
+
+        sig { overridable.returns(Nokogiri::XML::Document) }
+        attr_reader :document
+    end
+  end
+end

data/lib/web_author/json_ld_schema_processor.rb

@@ -0,0 +1,46 @@
+# typed: strict
+# frozen_string_literal: true
+
+require 'json'
+
+module WebAuthor
+  class JsonLdSchemaProcessor
+    extend T::Sig
+
+    JSON_LD_SCHEMA_XPATH = '//script[@type="application/ld+json"]'
+
+    sig { params(document: Nokogiri::XML::Document).void }
+    def initialize(document:)
+      @document = T.let(document, Nokogiri::XML::Document)
+      @_schemas = T.let(nil, T.nilable(T::Array[LdSchema]))
+    end
+
+    sig { returns(T::Array[LdSchema]) }
+    def schemas
+      @_schemas ||= extract_schemas
+    end
+
+    private
+
+      sig { returns(Nokogiri::XML::Document) }
+      attr_reader :document
+
+      sig { returns(T::Array[LdSchema]) }
+      def extract_schemas
+        json_ld_script_tags.filter_map do |script_tag|
+          content = script_tag.content.strip
+          next if content.empty?
+
+          json_data = JSON.parse(content)
+          LdSchema.from_hash(json_data)
+        rescue JSON::ParserError
+          # Skip invalid JSON as we don't need to process it but we want to
+          # let the processing move on to the next script tag that is JSON-LD
+          nil
+        end
+      end
+
+      sig { returns(Nokogiri::XML::NodeSet) }
+      def json_ld_script_tags = document.xpath(JSON_LD_SCHEMA_XPATH)
+  end
+end

data/lib/web_author/ld_author.rb

@@ -0,0 +1,30 @@
+# typed: true
+# frozen_string_literal: true
+
+module WebAuthor
+  class LdAuthor < T::Struct
+    extend T::Sig
+
+    const :type, T.nilable(String), default: nil
+    const :name, T.nilable(String), default: nil
+    const :url, T.nilable(String), default: nil
+    const :additional_properties, T::Hash[String, T.untyped], default: {}
+
+    ATTRIBUTES = T.let(['@type', 'name', 'url'].freeze, T::Array[String])
+
+    sig { params(hash: T.any(String, T::Hash[String, T.untyped])).returns(T.attached_class) }
+    def self.from_hash(hash)
+      return new(name: hash) if hash.is_a?(String)
+
+      main_properties = hash.dup.select { |key, _| ATTRIBUTES.include?(key) }
+      main_properties['type'] = main_properties.delete('@type')
+      main_properties.transform_keys!(&:to_sym)
+
+      additional_properties = hash.dup
+      ATTRIBUTES.each { |it| additional_properties.delete(it) }
+      additional_properties.transform_keys!(&:to_s)
+
+      new(**main_properties, additional_properties:)
+    end
+  end
+end

data/lib/web_author/ld_schema.rb

@@ -0,0 +1,74 @@
+# typed: strict
+# frozen_string_literal: true
+
+require 'json'
+
+module WebAuthor
+  class LdSchema < T::Struct
+    extend T::Sig
+
+    AuthorBaseType = T.type_alias { T.any(String, T::Hash[String, T.untyped]) }
+    ArrayofAuthors = T.type_alias { T::Array[AuthorBaseType] }
+    AuthorType = T.type_alias { T.any(AuthorBaseType, ArrayofAuthors) }
+
+    const :context, T.nilable(T.any(String, T::Hash[String, T.untyped])), default: nil
+    const :type, T.nilable(T.any(String, T::Array[String])), default: nil
+    const :id, T.nilable(String), default: nil
+    const :name, T.untyped, default: nil
+    const :description, T.untyped, default: nil
+    const :headline, T.untyped, default: nil
+    const :url, T.untyped, default: nil
+    const :image, T.untyped, default: nil
+    const :author, T.nilable(AuthorType), default: nil
+    const :publisher, T.nilable(T.any(String, T::Hash[String, T.untyped])), default: nil
+    const :blog_post, T.nilable(T::Array[T::Hash[String, T.untyped]]), default: nil
+    const :additional_properties, T::Hash[String, T.untyped], default: {}
+
+    ATTRIBUTES = T.let([
+      '@context', '@type', '@id', 'name', 'description', 'url', 'image',
+      'author', 'headline', 'publisher', 'blogPost'
+    ].freeze, T::Array[String])
+
+    sig { params(script_html_tag: String).returns(T.attached_class) }
+    def self.from_script_tag(script_html_tag)
+      hash = JSON.parse(script_html_tag)
+      from_hash(hash)
+    end
+
+    sig { params(hash: T::Hash[String, T.untyped]).returns(T.attached_class) }
+    def self.from_hash(hash)
+      main_properties = hash.dup.select { |key, _| ATTRIBUTES.include?(key) }
+      main_properties['blog_post'] = main_properties.delete('blogPost')
+      main_properties.transform_keys! { |it| it.start_with?('@') ? it.sub('@', '').to_sym : it.to_sym }
+
+      additional_properties = hash.dup
+      ATTRIBUTES.each { |it| additional_properties.delete(it) }
+      additional_properties.transform_keys!(&:to_s)
+
+      new(**main_properties, additional_properties:)
+    end
+
+    sig { returns(T.nilable(T.any(LdAuthor, T::Array[LdAuthor]))) }
+    def parsed_author
+      return @_parsed_author if defined?(@_parsed_author)
+
+      @_parsed_author = T.let(parse_author, T.nilable(T.any(LdAuthor, T::Array[LdAuthor])))
+    end
+
+    private
+
+      sig { returns(T.nilable(T.any(LdAuthor, T::Array[LdAuthor]))) }
+      def parse_author
+        return nil if author.nil?
+
+        case author
+        when String, Hash
+          topical_author = T.cast(author, AuthorBaseType)
+          LdAuthor.from_hash(topical_author)
+        when Array
+          topical_author = T.cast(author, ArrayofAuthors)
+          topical_author.map { |it| LdAuthor.from_hash(it) }
+        end
+      end
+  end
+end

data/lib/web_author/page.rb

@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+# typed: strict
+
+module WebAuthor
+  class Page
+    extend T::Sig
+
+    sig { returns(String) }
+    attr_reader :url
+
+    sig { returns(T.nilable(Nokogiri::XML::Document)) }
+    attr_reader :page_content
+
+    sig { params(url: String).void }
+    def initialize(url:)
+      @url = T.let(url, String)
+      @page_content = T.let(nil, T.nilable(Nokogiri::XML::Document))
+    end
+
+    sig { returns(T.nilable(String)) }
+    def author
+      fetch_page_content unless page_content
+
+      STRATEGIES.each do |strategy_class|
+        author = strategy_class.new(T.must(page_content)).author
+        return author if author
+      end
+
+      nil
+    end
+
+    private
+
+      # The order of these strategies is important as it determines in case of conflicting data
+      # which strategy should be used first.
+      STRATEGIES = T.let([
+        WebAuthor::Author::Strategies::AuthorFromLdSchema,
+        WebAuthor::Author::Strategies::AuthorFromMeta
+      ].freeze, T::Array[T.class_of(WebAuthor::Author::Strategy)])
+      private_constant :STRATEGIES
+
+      sig { returns(T.nilable(Nokogiri::XML::Document)) }
+      def fetch_page_content
+        uri = URI.parse(url)
+        response = Net::HTTP.get_response(uri)
+
+        if response.is_a?(Net::HTTPSuccess)
+          @page_content = Nokogiri::HTML(response.body)
+        else
+          raise Error, "Failed to fetch page: #{response.code} #{response.message}"
+        end
+      end
+  end
+end

data/lib/web_author/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module WebAuthor
+  VERSION = '0.1.0'
+end

data/lib/web_author.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+require 'zeitwerk'
+require 'nokogiri'
+require 'sorbet-runtime'
+require 'net/http'
+require 'uri'
+
+loader = Zeitwerk::Loader.for_gem
+loader.setup
+
+module WebAuthor
+  class Error < StandardError; end
+end

data/sig/webauthor.rbs

@@ -0,0 +1,4 @@
+module Webauthor
+  VERSION: String
+  # See the writing guide of rbs: https://github.com/ruby/rbs#guides
+end

data/sorbet/config

@@ -0,0 +1,4 @@
+--dir
+.
+--ignore=/tmp/
+--ignore=/vendor/bundle

data/sorbet/rbi/annotations/.gitattributes

@@ -0,0 +1 @@
+**/*.rbi linguist-vendored=true