wikidatum 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d5aa9e8eb9fccc24e9c423dcba185ef7550ad465426f90c57128968e06e99d51
-  data.tar.gz: 19fc905557859d97d1e604155d08d56d416eb332d6f285246c5354bdc1f024a9
+  metadata.gz: c8f1ec02cd83bf3ea3e432868a8283474f810bfd55740b5759c820d880f95e31
+  data.tar.gz: a5af62f917570a29ceceb6e420377154775d9820dec8abb02eb84e037f0bab89
 SHA512:
-  metadata.gz: 6ee9f49ee6689d953f308a9e2f4deaad71226080f70bf6fe3d7f375ac86bbd887e3494d50eed06b96308e6965165c86596fc7dae6abbdf2367e5f4302e49d73a
-  data.tar.gz: 4147886d03d2f17e2743d0b9309a22e278cbe2e0febb95acc77dbdebeb30319058b1cb33f2b142d0f1f0c84858fa3d59b40039492ff393b89fa19f04e516f54b
+  metadata.gz: a6c46eacdb7d7455ee5e96d3cf7c0970bd85ffc3abd1dc09c856b0f92062beaede5d451c9587ab72afc14d3b3c3a4e800c9685b435fbb4074b7d5e0b77cda463
+  data.tar.gz: bf4810fc7384b5b5cecf2db956758a1125a1664b62b8f7275d77347c7e0b78d64fbd76d011375914162c7a3a687bb4b679fd2ace7e8addcda2ad84775b88503f

data/CHANGELOG.md

@@ -6,6 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## Unreleased
 
+## 0.2.0 - 2022-08-13
+### Added
+
+- Add ability to serialize most of an item response from the Beta REST API into a usable `Wikidatum::Item` instance.
+  - This enables basic functionality like `client.item(id: 'Q123')` and `client.item(id: 'Q123').statements(properties: ['P123'])`.
+- Add `Client#add_statement` method for creating new statements on an item.
+- Add `Client#delete_statement` method for deleting statements from an item.
+- Add support for reading and writing all statement types: `novalue`, `somevalue`, `string`, `time`, `quantity`, `globecoordinate`, `monolingualtext`, and `wikibase-entityid`.
+
+### Internal
+
+- Add YARD docs and auto-deploy them with GitHub Pages.
+- Add comprehensive unit tests for the gem.
+
 ## 0.1.0 - 2022-06-20
 ### Added
 

data/Gemfile

@@ -7,5 +7,7 @@ gemspec
 
 gem 'rake', '~> 13.0'
 gem 'minitest', '~> 5.16'
+gem 'simplecov', '~> 0.21', require: false
 gem 'rubocop', '~> 1.30'
 gem 'yard', '~> 0.9'
+gem 'webmock', '~> 3.17', require: false

data/Gemfile.lock

@@ -1,16 +1,28 @@
 PATH
   remote: .
   specs:
-    wikidatum (0.1.0)
+    wikidatum (0.2.0)
+      faraday (~> 2.4)
 
 GEM
   remote: https://rubygems.org/
   specs:
+    addressable (2.8.0)
+      public_suffix (>= 2.0.2, < 5.0)
     ast (2.4.2)
+    crack (0.4.5)
+      rexml
+    docile (1.4.0)
+    faraday (2.4.0)
+      faraday-net_http (~> 2.0)
+      ruby2_keywords (>= 0.0.4)
+    faraday-net_http (2.1.0)
+    hashdiff (1.0.1)
     minitest (5.16.0)
     parallel (1.22.1)
     parser (3.1.2.0)
       ast (~> 2.4.1)
+    public_suffix (4.0.7)
     rainbow (3.1.1)
     rake (13.0.6)
     regexp_parser (2.5.0)
@@ -27,18 +39,33 @@ GEM
     rubocop-ast (1.18.0)
       parser (>= 3.1.1.0)
     ruby-progressbar (1.11.0)
+    ruby2_keywords (0.0.5)
+    simplecov (0.21.2)
+      docile (~> 1.1)
+      simplecov-html (~> 0.11)
+      simplecov_json_formatter (~> 0.1)
+    simplecov-html (0.12.3)
+    simplecov_json_formatter (0.1.4)
     unicode-display_width (2.1.0)
+    webmock (3.17.1)
+      addressable (>= 2.8.0)
+      crack (>= 0.3.2)
+      hashdiff (>= 0.4.0, < 2.0.0)
     webrick (1.7.0)
     yard (0.9.28)
       webrick (~> 1.7.0)
 
 PLATFORMS
+  arm64-darwin-21
   x86_64-darwin-21
+  x86_64-linux
 
 DEPENDENCIES
   minitest (~> 5.16)
   rake (~> 13.0)
   rubocop (~> 1.30)
+  simplecov (~> 0.21)
+  webmock (~> 3.17)
   wikidatum!
   yard (~> 0.9)
 

data/README.md

@@ -1,8 +1,8 @@
 # Wikidatum
 
-This gem will support making requests with the [new Wikidata/Wikibase REST API](https://doc.wikimedia.org/Wikibase/master/js/rest-api/).
+This gem supports making requests to the [new Wikidata/Wikibase REST API](https://doc.wikimedia.org/Wikibase/master/js/rest-api/).
 
-**Currently it is in very early development and is not ready for actual usage**.
+**The gem is currently in very early development and is not ready for production usage**.
 
 ## Installation
 
@@ -22,7 +22,42 @@ Or install it yourself as:
 
 ## Usage
 
-TODO: Write usage instructions here
+You can view the YARD docs on GitHub Pages [here](https://connorshea.github.io/wikidatum/index.html).
+
+Currently, the gem is able to hit a few GET endpoints, and currently has no way to provide authentication and perform POST/PUT/DELETE requests. The additional features will be added later.
+
+```ruby
+require 'wikidatum'
+
+wikidatum_client = Wikidatum::Client.new(
+  user_agent: 'REPLACE ME WITH THE NAME OF YOUR BOT!',
+  # Currently only the beta site has the API available, you'll
+  # likely want to use wikidata.org once it's stable.
+  wikibase_url: 'https://wikidata.beta.wmflabs.org',
+  bot: true
+)
+
+# Get an item from the Wikibase instance.
+item = wikidatum_client.item(id: 'Q2') #=> Wikidatum::Item
+
+# Get the statements from the item.
+item.statements #=> Array<Wikidatum::Statement>
+
+# Get the statments for property P123 on the item.
+item.statements(properties: ['P123']) #=> Array<Wikidatum::Statement>
+
+# Get all the labels for the item.
+item.labels #=> Array<Wikidatum::Term>
+
+# Get the English label for the item.
+item.label(lang: :en) #=> Wikidatum::Term
+
+# Get the actual value for the label.
+item.label(lang: :en).value #=> "Earth"
+
+# Get the values for all English aliases on this item.
+item.aliases(langs: [:en]).map(&:value) #=> ["Planet Earth", "Pale Blue Dot"]
+```
 
 ## Development
 

data/Rakefile

@@ -6,7 +6,7 @@ require "rake/testtask"
 Rake::TestTask.new(:test) do |t|
   t.libs << "test"
   t.libs << "lib"
-  t.test_files = FileList["test/**/test_*.rb"]
+  t.test_files = FileList["test/**/*_spec.rb"]
 end
 
 require "rubocop/rake_task"

data/lib/wikidatum/client.rb

@@ -1,22 +1,370 @@
 # frozen_string_literal: true
 
+require 'faraday'
+require 'faraday/net_http'
+
 class Wikidatum::Client
-  # @return [String] The root URL of the Wikibase instance we want to interact
-  #                  with. If not provided, will default to Wikidata.
+  ITEM_REGEX = /^Q?\d+$/.freeze
+  STATEMENT_REGEX = /^Q?\d+\$[\w-]+$/.freeze
+
+  # @return [String] the root URL of the Wikibase instance we want to interact
+  #   with. If not provided, will default to Wikidata.
   attr_reader :wikibase_url
 
-  # @return [Boolean] Whether this client instance should identify itself
-  #                   as a bot when making requests.
+  # @return [Boolean] whether this client instance should identify itself
+  #   as a bot when making requests.
   attr_reader :bot
 
+  # @return [String] the UserAgent header to send with all requests to the
+  #   Wikibase API.
+  attr_reader :user_agent
+
+  # Create a new Wikidatum::Client to interact with the Wikibase REST API.
+  #
+  # @example
+  #  wikidatum_client = Wikidatum::Client.new(
+  #    user_agent: 'REPLACE ME WITH THE NAME OF YOUR BOT!',
+  #    wikibase_url: 'https://www.wikidata.org',
+  #    bot: true
+  #  )
+  #
+  # @param user_agent [String] The UserAgent header to send with all requests
+  #   to the Wikibase API.
   # @param wikibase_url [String] The root URL of the Wikibase instance we want
-  #                              to interact with. If not provided, will
-  #                              default to Wikidata.
+  #   to interact with. If not provided, will default to
+  #   `https://www.wikidata.org`. Do not include a `/` at the end of the URL.
   # @param bot [Boolean] Whether requests sent by this client instance should
-  #                      be registered as bot requests.
+  #   be registered as bot requests. Defaults to `true`.
   # @return [Wikidatum::Client]
-  def initialize(wikibase_url: 'https://www.wikidata.org', bot: true)
+  def initialize(user_agent:, wikibase_url: 'https://www.wikidata.org', bot: true)
+    raise ArgumentError, "Wikibase URL must not end with a `/`, got #{wikibase_url.inspect}." if wikibase_url.end_with?('/')
+
+    # TODO: Add the Ruby gem version to the UserAgent automatically, and
+    # restrict the ability for end-users to actually set the UserAgent?
+    @user_agent = user_agent
     @wikibase_url = wikibase_url
     @bot = bot
+
+    Faraday.default_adapter = :net_http
+  end
+
+  # Get an item from the Wikibase API based on its QID.
+  #
+  # @example
+  #   wikidatum_client.item(id: 'Q123')
+  #   wikidatum_client.item(id: 123)
+  #   wikidatum_client.item(id: '123')
+  #
+  # @param id [String, Integer] Either a string or integer representation of
+  #   the item's QID, e.g. `"Q123"`, `"123"`, or `123`.
+  # @return [Wikidatum::Item]
+  def item(id:)
+    raise ArgumentError, "#{id.inspect} is an invalid Wikibase QID. Must be an integer, a string representation of an integer, or in the format 'Q123'." unless id.is_a?(Integer) || id.match?(ITEM_REGEX)
+
+    id = coerce_item_id(id)
+
+    response = get_request("/entities/items/#{id}")
+
+    puts JSON.pretty_generate(response) if ENV['DEBUG']
+
+    Wikidatum::Item.marshal_load(response)
+  end
+
+  # Get a statement from the Wikibase API based on its ID.
+  #
+  # @example
+  #   wikidatum_client.statement(id: 'Q123$f004ec2b-4857-3b69-b370-e8124f5bd3ac')
+  #
+  # @param id [String] A string representation of the statement's ID.
+  # @return [Wikidatum::Statement]
+  def statement(id:)
+    raise ArgumentError, "#{id.inspect} is an invalid Wikibase Statement ID. Must be a string in the format 'Q123$f004ec2b-4857-3b69-b370-e8124f5bd3ac'." unless id.match?(STATEMENT_REGEX)
+
+    response = get_request("/statements/#{id}")
+
+    puts JSON.pretty_generate(response) if ENV['DEBUG']
+
+    Wikidatum::Statement.marshal_load(response)
+  end
+
+  # Add a statement to an item.
+  #
+  # NOTE: Adding references/qualifiers with `add_statement` is untested and
+  # effectively unsupported for now.
+  #
+  # @example Add a string statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P23',
+  #     datavalue: Wikidatum::DataValueType::WikibaseString.new(string: 'Foo'),
+  #     comment: 'Adding something or another.'
+  #   )
+  #
+  # @example Add a 'no value' statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P124',
+  #     datavalue: Wikidatum::DataValueType::NoValue.new(
+  #       type: :no_value,
+  #       value: nil
+  #     )
+  #   )
+  #
+  # @example Add an 'unknown value' statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P124',
+  #     datavalue: Wikidatum::DataValueType::SomeValue.new(
+  #       type: :some_value,
+  #       value: nil
+  #     )
+  #   )
+  #
+  # @example Add a globe coordinate statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P124',
+  #     datavalue: Wikidatum::DataValueType::GlobeCoordinate.new(
+  #       latitude: 52.51666,
+  #       longitude: 13.3833,
+  #       precision: 0.01666,
+  #       globe: 'https://wikidata.org/entity/Q2'
+  #     )
+  #   )
+  #
+  # @example Add a monolingual text statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P124',
+  #     datavalue: Wikidatum::DataValueType::MonolingualText.new(
+  #       language: 'en',
+  #       text: 'Foobar'
+  #     )
+  #   )
+  #
+  # @example Add a quantity statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P124',
+  #     datavalue: Wikidatum::DataValueType::Quantity.new(
+  #       amount: '+12',
+  #       upper_bound: nil,
+  #       lower_bound: nil,
+  #       unit: 'https://wikidata.org/entity/Q1234'
+  #     )
+  #   )
+  #
+  # @example Add a time statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P124',
+  #     datavalue: Wikidatum::DataValueType::Time.new(
+  #       time: '+2022-08-12T00:00:00Z',
+  #       time_zone: 0,
+  #       precision: 11,
+  #       calendar_model: 'https://wikidata.org/entity/Q1234'
+  #     )
+  #   )
+  #
+  # @example Add a Wikibase item statement.
+  #   wikidatum_client.add_statement(
+  #     id: 'Q123',
+  #     property: 'P124',
+  #     datavalue: Wikidatum::DataValueType::WikibaseEntityId.new(
+  #       entity_type: 'item',
+  #       numeric_id: 1234,
+  #       id: 'Q1234'
+  #     )
+  #   )
+  #
+  # @param id [String] the ID of the item on which the statement will be added.
+  # @param property [String] property ID in the format 'P123'.
+  # @param datavalue [Wikidatum::DataValueType::GlobeCoordinate, Wikidatum::DataValueType::MonolingualText, Wikidatum::DataValueType::Quantity, Wikidatum::DataValueType::WikibaseString, Wikidatum::DataValueType::Time, Wikidatum::DataValueType::WikibaseEntityId, Wikidatum::DataValueType::NoValue, Wikidatum::DataValueType::SomeValue] the datavalue of the statement being created.
+  # @param datatype [String, nil] if nil, it'll determine the type based on what was passed for the statement argument. This may differ from the type of the Statement's datavalue (for example with the 'url' type).
+  # @param qualifiers [Hash<String, Array<Wikidatum::Snak>>]
+  # @param references [Array<Wikidatum::Reference>]
+  # @param rank [String]
+  # @param tags [Array<String>]
+  # @param comment [String, nil]
+  # @return [Boolean] True if the request succeeded.
+  def add_statement(id:, property:, datavalue:, datatype: nil, qualifiers: {}, references: [], rank: 'normal', tags: [], comment: nil)
+    raise ArgumentError, "#{id.inspect} is an invalid Wikibase QID. Must be an integer, a string representation of an integer, or in the format 'Q123'." unless id.is_a?(Integer) || id.match?(ITEM_REGEX)
+
+    id = coerce_item_id(id)
+
+    # Unless datatype is set explicitly by the caller, just assume we can pull the
+    # default from the datavalue class.
+    datatype ||= datavalue.wikibase_datatype
+
+    case datavalue.class.to_s
+    when 'Wikidatum::DataValueType::NoValue'
+      statement_hash = {
+        mainsnak: {
+          snaktype: 'novalue',
+          property: property,
+          datatype: datatype
+        }
+      }
+    when 'Wikidatum::DataValueType::SomeValue'
+      statement_hash = {
+        mainsnak: {
+          snaktype: 'somevalue',
+          property: property,
+          datatype: datatype
+        }
+      }
+    when 'Wikidatum::DataValueType::GlobeCoordinate', 'Wikidatum::DataValueType::MonolingualText', 'Wikidatum::DataValueType::Quantity', 'Wikidatum::DataValueType::WikibaseString', 'Wikidatum::DataValueType::Time', 'Wikidatum::DataValueType::WikibaseEntityId'
+      statement_hash = {
+        mainsnak: {
+          snaktype: 'value',
+          property: property,
+          datatype: datatype,
+          datavalue: {
+            type: datavalue.wikibase_type,
+            value: datavalue.marshal_dump
+          }
+        }
+      }
+    else
+      raise ArgumentError, "Expected an instance of one of Wikidatum::DataValueType's subclasses for datavalue, but got #{datavalue.inspect}."
+    end
+
+    body = { statement: statement_hash.merge({ qualifiers: qualifiers, references: references, rank: rank, type: "statement" }) }
+
+    response = post_request("/entities/items/#{id}/statements", body, tags: tags, comment: comment)
+
+    puts JSON.pretty_generate(response) if ENV['DEBUG']
+
+    response.success?
+  end
+
+  # Delete a statement from an item.
+  #
+  # @example
+  #   wikidatum_client.delete_statement(
+  #     id: 'Q123$4543523c-1d1d-1111-1e1e-11b11111b1f1',
+  #     comment: "Deleting this statement because it's bad."
+  #   )
+  #
+  # @param id [String] the ID of the statemnt being deleted.
+  # @param tags [Array<String>]
+  # @param comment [String, nil]
+  # @return [Boolean] True if the request succeeded.
+  def delete_statement(id:, tags: [], comment: nil)
+    raise ArgumentError, "#{id.inspect} is an invalid Wikibase Statement ID. Must be a string in the format 'Q123$f004ec2b-4857-3b69-b370-e8124f5bd3ac'." unless id.match?(STATEMENT_REGEX)
+
+    response = delete_request("/statements/#{id}", tags: tags, comment: comment)
+
+    puts JSON.pretty_generate(response) if ENV['DEBUG']
+
+    response.success?
+  end
+
+  private
+
+  # For now this just returns the `@wikibase_url`, but in the future the API
+  # routes will presumably be nested further, so this is just future-proofing
+  # to allow that to be easily changed later.
+  #
+  # @return [String] URL for the Wikibase API endpoint.
+  def api_url
+    @api_url ||= "#{@wikibase_url}/w/rest.php/wikibase/v0"
+  end
+
+  # Default headers to be sent with every request.
+  #
+  # @return [Hash] A hash of some headers that should be used when sending a request.
+  def universal_headers
+    @universal_headers ||= {
+      'User-Agent' => @user_agent,
+      'Content-Type' => 'application/json'
+    }
+  end
+
+  # Make a GET request to a given Wikibase endpoint.
+  #
+  # @param path [String] The relative path for the API endpoint.
+  # @param params [Hash] Query parameters to send with the request, if any.
+  # @return [Hash] JSON response, parsed into a hash.
+  def get_request(path, params = nil)
+    url = "#{api_url}#{path}"
+
+    response = Faraday.get(url, params, universal_headers)
+
+    # Error handling if it doesn't return a 200
+    unless response.success?
+      puts 'Something went wrong with this request!'
+      puts "Status Code: #{response.status}"
+      puts response.body.inspect
+    end
+
+    JSON.parse(response.body)
+  end
+
+  # Make a POST request to a given Wikibase endpoint.
+  #
+  # @param path [String] The relative path for the API endpoint.
+  # @param body [Hash] The body to post to the endpoint.
+  # @param tags [Array<String>] The tags to apply to the edit being made by this request, for PUT/POST/DELETE requests.
+  # @param comment [String] The edit description, for PUT/POST/DELETE requests.
+  # @return [Hash] JSON response, parsed into a hash.
+  def post_request(path, body = {}, tags: nil, comment: nil)
+    url = "#{api_url}#{path}"
+
+    body[:bot] = @bot
+    body[:tags] = tags unless tags.empty?
+    body[:comment] = comment unless comment.nil?
+
+    response = Faraday.post(url) do |req|
+      req.body = JSON.generate(body)
+      req.headers = universal_headers
+    end
+
+    puts response.body.inspect if ENV['DEBUG']
+
+    response
+  end
+
+  # Make a DELETE request to a given Wikibase endpoint.
+  #
+  # @param path [String] The relative path for the API endpoint.
+  # @param tags [Array<String>] The tags to apply to the edit being made by this request, for PUT/POST/DELETE requests.
+  # @param comment [String] The edit description, for PUT/POST/DELETE requests.
+  # @return [Hash] JSON response, parsed into a hash.
+  def delete_request(path, tags: [], comment: nil)
+    url = "#{api_url}#{path}"
+
+    body = {}
+    body[:bot] = @bot
+    body[:tags] = tags unless tags.empty?
+    body[:comment] = comment unless comment.nil?
+
+    response = Faraday.delete(url) do |req|
+      req.body = JSON.generate(body)
+      req.headers = universal_headers
+    end
+
+    puts response.body.inspect if ENV['DEBUG']
+
+    # Error handling if it doesn't return a 200
+    unless response.success?
+      puts 'Something went wrong with this request!'
+      puts "Status Code: #{response.status}"
+      puts response.body.inspect
+    end
+
+    response
+  end
+
+  # Coerce an Item ID in the formats 'Q123', '123' or 123 into a consistent
+  # 'Q123' format. We need to have the ID in the format 'Q123' for the API
+  # request, which is why coercion is necessary.
+  #
+  # @param id [String, Integer]
+  # @return [String]
+  def coerce_item_id(id)
+    return id if id.to_s.start_with?('Q')
+
+    "Q#{id}"
   end
 end

data/lib/wikidatum/data_value_type/base.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+# For more information on the possible types that can be returned by
+# datavalues, see the official documentation:
+# https://doc.wikimedia.org/Wikibase/master/php/md_docs_topics_json.html#json_datavalues
+module Wikidatum::DataValueType
+  class Base
+    # Represents the type for this datavalue instance.
+    #
+    # Possible values for the `type` attribute are:
+    #
+    # - `:no_value`: No value
+    # - `:some_value`: Unknown value
+    # - `:globe_coordinate`: {DataValueType::GlobeCoordinate}
+    # - `:monolingual_text`: {DataValueType::MonolingualText}
+    # - `:quantity`: {DataValueType::Quantity}
+    # - `:string`: {DataValueType::WikibaseString}
+    # - `:time`: {DataValueType::Time}
+    # - `:wikibase_entity_id`: {DataValueType::WikibaseEntityId}
+    #
+    # @return [Symbol]
+    attr_reader :type
+
+    # The value of the datavalue object in the response.
+    #
+    # If the `type` is `novalue` or `somevalue`, this returns `nil`.
+    #
+    # @return [DataValueType::GlobeCoordinate, DataValueType::MonolingualText, DataValueType::Quantity, DataValueType::WikibaseString, DataValueType::Time, DataValueType::WikibaseEntityId, nil]
+    attr_reader :value
+
+    # @param type [Symbol]
+    # @param value [DataValueType::GlobeCoordinate, DataValueType::MonolingualText, DataValueType::Quantity, DataValueType::WikibaseString, DataValueType::Time, DataValueType::WikibaseEntityId, nil] nil if type is no_value or some_value
+    # @return [void]
+    def initialize(type:, value:)
+      @type = type
+      @value = value
+    end
+
+    # @return [Hash]
+    def to_h
+      {
+        type: @type,
+        value: @value&.to_h
+      }
+    end
+
+    # @!visibility private
+    #
+    # @param data_value_type [String] The value of `type` for the given Snak's datavalue.
+    # @param data_value_json [Hash] The `value` part of datavalue.
+    # @return [Wikidatum::DataValueType::Base] An instance of Base.
+    def self.marshal_load(data_value_type, data_value_json)
+      unless Wikidatum::DataValueType::DATA_VALUE_TYPES.keys.include?(data_value_type.to_sym)
+        puts 'WARNING: Unsupported datavalue type.'
+        return nil
+      end
+
+      Object.const_get(Wikidatum::DataValueType::DATA_VALUE_TYPES[data_value_type.to_sym]).marshal_load(data_value_json)
+    end
+  end
+end