mediawiktory 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 3dbe24768ed1be0926f57bb4cd72e07e098c59b6
+  data.tar.gz: 230b1128fb7ae5b854d714dc564d7d60aaa6be08
+SHA512:
+  metadata.gz: e2a9446d6a9a39c21d2b8f7fa3569a60c33e79607356929af29164201485ef892f84f60c7954713a7e27bf6f29d271f5454909234f69b39e2ada503a09d54142
+  data.tar.gz: 7ec9c4d535a690d2906a64cf81d401ab3bd1198a014a316180d11b7fced7270c9f4fb4eb1d5b3b80d9706d6b42bb29c1ae23ee9960235a955c288035ebc220db

data/Gemfile.lock

@@ -0,0 +1,63 @@
+PATH
+  remote: .
+  specs:
+    mediawiktory (0.0.1)
+      addressable
+      faraday
+      faraday_middleware
+      hashie
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    addressable (2.3.8)
+    crack (0.4.2)
+      safe_yaml (~> 1.0.0)
+    diff-lcs (1.2.5)
+    faker (1.5.0)
+      i18n (~> 0.5)
+    faraday (0.9.1)
+      multipart-post (>= 1.2, < 3)
+    faraday_middleware (0.10.0)
+      faraday (>= 0.7.4, < 0.10)
+    hashie (3.4.2)
+    i18n (0.7.0)
+    mini_portile (0.6.2)
+    multipart-post (2.0.0)
+    naught (1.1.0)
+    nokogiri (1.6.6.2)
+      mini_portile (~> 0.6.0)
+    rspec (3.3.0)
+      rspec-core (~> 3.3.0)
+      rspec-expectations (~> 3.3.0)
+      rspec-mocks (~> 3.3.0)
+    rspec-core (3.3.2)
+      rspec-support (~> 3.3.0)
+    rspec-expectations (3.3.1)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.3.0)
+    rspec-its (1.2.0)
+      rspec-core (>= 3.0.0)
+      rspec-expectations (>= 3.0.0)
+    rspec-mocks (3.3.2)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.3.0)
+    rspec-support (3.3.0)
+    rubygems-tasks (0.2.4)
+    safe_yaml (1.0.4)
+    webmock (1.21.0)
+      addressable (>= 2.3.6)
+      crack (>= 0.3.2)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  faker
+  mediawiktory!
+  naught
+  nokogiri
+  rspec
+  rspec-its
+  rubygems-tasks
+  webmock

data/README.md

@@ -0,0 +1,103 @@
+**Warning! Even 0.0.1 unrealeased yet! Possible first release date
+is Dec 3, 2015**
+
+**MediaWiktory** is MediaWiki API client that doesn't suck. No, seriously.
+
+[MediaWiki API](https://www.mediawiki.org/wiki/API:Main_page)
+currently is very powerful and full-featured (thought not very easy to use).
+Things like "fetch first 50 pages from that category alongside with their
+revision history and interwiki links and mediafile stats" are typically done
+with one carefully constructed request and return lots of useful information.
+
+Yes, there already are [several](https://www.mediawiki.org/wiki/API:Client_code#Ruby)
+API clients for Ruby, including ["official" one](https://github.com/wikimedia/mediawiki-ruby-api).
+Typical approach for all of them is thick wrapper around some functionality
+(like "login and edit pages" or "search and analyze pages"), and leave
+all the other cool things for generic `query` method (at best), or
+without any coverage at all.
+
+MediaWiktory, on the contrary, does this:
+* wraps all the functionality of all the modules of modern MediaWiki,
+  as per [this documentation](https://en.wikipedia.org/w/api.php?action=help&modules=main);
+  * and, for the record, provides sparse documentation for all modules
+    and possibilities;
+* makes the wrapper reasonably thin, so you can always inspect raw responses
+  and raw request MediaWiktory will provide to the API;
+* makes the wrapper reasonably smart, so things like "fetch next page for
+  this request" or "login with a token" are done with ease (but without
+  unnecessary "auto-magic", like fetching 500 pages of data without any
+  chance to control this);
+* validates requests and simplifies parameters passing, while still
+  preserving abilitity to use any existing parameter and feature;
+* it manages cookies and tokens for you, but can expose underlying
+  Faraday web client so the middleware can be added (like caching).
+
+**Warning: Above is IDEAL state of the library. Current state, at
+version 0.0.1 is far less impressive (yet already useful).**
+
+## Structure and usage
+
+MediaWiktory tries to resemble original API structure, but make its calls
+more Ruby-ish. So, [Arel](https://github.com/rails/arel)-alike "chainable
+query" approach is implemented.
+
+Fast example:
+
+**Goal**: fetch content for first 30 pages from category "Countries in
+South America".
+
+**Raw request** (split in lines for readability):
+
+```
+https://en.wikipedia.org/w/api.php?
+  action=query&
+  generator=categorymembers&
+  gcmtitle=Category:Countries_in_South_America&
+  gcmlimit=30&
+  prop=revisions&
+  rvprop=content
+```
+
+**MediaWiktory request**—almost the same, but structured and validated
+on-the-fly, and without `gcm` and `rv` prefixes black magic:
+
+```ruby
+client = MediaWiktory::Client.new('https://en.wikipedia.org/w/api.php')
+
+response = client.
+  query.
+  generator(categorymembers: {title: 'Category:Countries_in_South_America', limit: 30}).
+  prop(revisions: {prop: :content}).
+  perform
+
+# MediaWiktory handles "next page fetching" for you, if you want
+response.continue! while response.can_continue?
+
+# MediaWiktory parses response and provides smart shortcuts
+p response.pages.map(&:title)
+```
+
+The same approach works for ANY `action` described in [docs](https://en.wikipedia.org/w/api.php),
+and all of its "modules" and "submodules".
+
+**NB**: its 0.0.1, ok? Domain is complex and is still investigated. For
+example, above statement definitely **wants** to look like
+
+```ruby
+response = client.
+  query.
+  generator(categorymembers: 'Category:Countries_in_South_America').
+  limit(30).
+  prop(revisions: :content).
+  perform
+```
+
+...but it should be done without flexibility loss.
+
+## Author
+
+[Victor Shepelev](http://zverok.github.io)
+
+## License
+
+MIT.

data/bin/mediawiktory

@@ -0,0 +1,13 @@
+#!/usr/bin/env ruby
+
+require 'rubygems'
+require 'bundler/setup'
+
+$:.unshift File.join(File.dirname(__FILE__), '..', 'lib')
+require_relative '../lib/mediawiktory'
+
+include MediaWiktory
+
+require 'irb'
+ARGV.shift until ARGV.empty?
+IRB.start

data/doc/mediawiki_api_types.md

@@ -0,0 +1,14 @@
+
+* single value:
+  * boolean ("if the parameter is specified, regardless of value, it is considered true");
+  * string;
+  * integer;
+    * integer or "max";
+  * timestamp ([list of allowed formats](https://www.mediawiki.org/w/api.php?action=help&modules=main#main.2Fdatatypes));
+  * enum value (string of limited list);
+  * module (see below);
+* list:
+  * of integers;
+  * of strings;
+  * of enum values;
+  * of modules.

data/examples/query.rb

@@ -0,0 +1,15 @@
+$:.unshift 'lib'
+require 'mediawiktory'
+include MediaWiktory
+
+require 'pp'
+
+c = Client.new('https://en.wikipedia.org/w/api.php')
+res = c.query.
+  titles('Argentina', 'Chile').
+  prop(:categories).
+  perform
+
+res.continue! while res.continue?
+
+pp res.pages.map(&:to_h)

data/examples/query_generator.rb

@@ -0,0 +1,15 @@
+$:.unshift 'lib'
+require 'mediawiktory'
+include MediaWiktory
+
+require 'pp'
+
+c = Client.new('https://en.wikipedia.org/w/api.php')
+res = c.query.
+  generator(categorymembers: {title: 'Category:Countries_in_South_America', limit: 50}).
+  prop(revisions: {prop: ['content']}).
+  perform
+
+res.continue! while res.continue?
+
+pp res.pages.map(&:title)

data/lib/faraday/naive_cache.rb

@@ -0,0 +1,40 @@
+require 'addressable/uri'
+require 'json'
+
+module Faraday
+  class NaiveCache < Middleware
+    BASE_PATH = 'tmp/cache' # FIXME, obviously
+    
+    def call(env)
+      if response = from_cache(env[:url])
+        env[:response] = response
+      else
+        @app.call(env).on_complete do |env|
+          to_cache(env[:url], env)
+        end
+      end
+    end
+
+    private
+
+    def from_cache(url)
+      path = cached_path(url)
+      File.exists?(path) ? Response.new(JSON.parse(File.read(path))) : nil
+    end
+
+    def to_cache(url, env)
+      env.body = env.body.force_encoding('UTF-8')
+      
+      path = cached_path(url)
+      FileUtils.mkdir_p File.dirname(path)
+      File.write path, env.to_hash.to_json
+    end
+
+    def cached_path(url)
+      uri = Addressable::URI.parse(url)
+      filename = [uri.path, uri.query].compact.join('?').gsub(/[?\/&]/, '-')
+      filename = '_root_' if filename.empty?
+      File.join(BASE_PATH, uri.host, filename)
+    end
+  end
+end

data/lib/media_wiktory.rb

@@ -0,0 +1 @@
+require_relative './mediawiktory'

data/lib/mediawiktory/action.rb

@@ -0,0 +1,48 @@
+require 'json'
+require 'hashie'
+
+module MediaWiktory
+  class Action < MWModule
+    attr_reader :client
+    
+    def initialize(client, **values)
+      @client = client
+      super(**values)
+    end
+
+    def dup
+      self.class.new(client, to_h)
+    end
+
+    def perform(params = {})
+      self.class::Response. # child classes can enrich and specialize their Response
+        from_json(self, @client.send(self.class.request_method, to_param.merge(params)))
+    end
+
+    def to_param
+      super.merge('action' => self.class.symbol.to_s, 'format' => 'json')
+    end
+
+    class Response
+      def self.from_json(action, data)
+        new(action, JSON.parse(data))
+      end
+
+      attr_reader :action, :raw
+
+      def initialize(action, raw)
+        @action = action
+        @raw = Hashie::Mash.new(raw)
+      end
+
+      def initialize_copy(other)
+        @action = other.action
+        @raw = other.raw.dup
+      end
+
+      def to_h
+        raw.to_h
+      end
+    end
+  end
+end

data/lib/mediawiktory/array_with_keys.rb

@@ -0,0 +1,41 @@
+module MediaWiktory
+  # FIXME: maybe, its a wrongdoing
+  # But for now, I cant invent wiser structure for MediaWiki's
+  # {'numericpageid' => {page}} hash
+  #
+  # Having it as a hash complicates code (most time you just need a list
+  # of pages), dropping keys completely and using only values seem to
+  # have its own drawbacks.
+  #
+  class ArrayWithKeys < Array
+    def self.from_hash(hash)
+      new(hash.values, hash.keys)
+    end
+
+    attr_reader :keys
+    
+    def initialize(values, keys)
+      super(values)
+      @keys = keys
+    end
+
+    def [](key)
+      case key
+      when Fixnum
+        super
+      else
+        idx = @keys.index(key)
+        idx && super(idx)
+      end
+    end
+
+    def push(value, key)
+      super(value)
+      @keys.push(key)
+    end
+
+    def to_h
+      keys.zip(self).to_h
+    end
+  end
+end

data/lib/mediawiktory/client.rb

@@ -0,0 +1,22 @@
+require_relative 'generated/actions_mixin'
+
+module MediaWiktory
+  class Client
+    attr_reader :url
+    
+    def initialize(url)
+      @url = Addressable::URI.parse(url)
+      @faraday = Faraday.new(url)
+    end
+
+    def get(params)
+      @faraday.get('', params).body
+    end
+
+    def post(params)
+      @faraday.post('', params).body
+    end
+
+    include ActionsMixin
+  end
+end

data/lib/mediawiktory/generated/abusefiltercheckmatch.rb

@@ -0,0 +1,9 @@
+module MediaWiktory
+  class Abusefiltercheckmatch < Action
+    symbol :abusefiltercheckmatch
+    param :filter, Params::String
+    param :vars, Params::String
+    param :rcid, Params::Integer
+    param :logid, Params::Integer
+  end
+end

data/lib/mediawiktory/generated/abusefilterchecksyntax.rb

@@ -0,0 +1,6 @@
+module MediaWiktory
+  class Abusefilterchecksyntax < Action
+    symbol :abusefilterchecksyntax
+    param :filter, Params::String
+  end
+end

data/lib/mediawiktory/generated/abusefilterevalexpression.rb

@@ -0,0 +1,6 @@
+module MediaWiktory
+  class Abusefilterevalexpression < Action
+    symbol :abusefilterevalexpression
+    param :expression, Params::String
+  end
+end

data/lib/mediawiktory/generated/abusefilters.rb

@@ -0,0 +1,12 @@
+module MediaWiktory
+  class Abusefilters < MWModule
+    symbol :abusefilters
+    prefix "abf"
+    param :startid, Params::Integer
+    param :endid, Params::Integer
+    param :dir, Params::Enum["newer", "older"]
+    param :show, Params::List[Params::Enum["enabled", "!enabled", "deleted", "!deleted", "private", "!private"]]
+    param :limit, Params::IntegerOrMax
+    param :prop, Params::List[Params::Enum["id", "description", "pattern", "actions", "hits", "comments", "lasteditor", "lastedittime", "status", "private"]]
+  end
+end

data/lib/mediawiktory/generated/abusefilterunblockautopromote.rb

@@ -0,0 +1,8 @@
+module MediaWiktory
+  class Abusefilterunblockautopromote < Action
+    symbol :abusefilterunblockautopromote
+    post!
+    param :user, Params::String
+    param :token, Params::String
+  end
+end

data/lib/mediawiktory/generated/abuselog.rb

@@ -0,0 +1,14 @@
+module MediaWiktory
+  class Abuselog < MWModule
+    symbol :abuselog
+    prefix "afl"
+    param :start, Params::Timestamp
+    param :end, Params::Timestamp
+    param :dir, Params::Enum["newer", "older"]
+    param :user, Params::String
+    param :title, Params::String
+    param :filter, Params::List[Params::String]
+    param :limit, Params::IntegerOrMax
+    param :prop, Params::List[Params::Enum["ids", "filter", "user", "ip", "title", "action", "details", "result", "timestamp", "hidden", "revid"]]
+  end
+end