whatthegem 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: c9f1d103cc46e0222855e20a653a1f0d6035d706
+  data.tar.gz: 482bb2927a0e82b9125bd5d0805da02073536025
+SHA512:
+  metadata.gz: c26011bb1bc4043411507d0faf444942de248b92af28c842e035aa316e443d2ef5ba2c142569034a41ebffdb9d8f3a9a132785c7add87bf7b8bacae49a05960b
+  data.tar.gz: ce913168e10295374f15a3ae09cd6f7d0a6423a6c42917af36c4ac9581c9b3d540f80f441aa093fd7487e9cfd3b25b687c85d72279d1367069198304438d5d62

data/README.md

@@ -0,0 +1,56 @@
+[![Gem Version](https://badge.fury.io/rb/whatthegem.svg)](http://badge.fury.io/rb/whatthegem)
+
+`whatthegem` is a small utility to answer some questions about Ruby gems you work with or planning to work with.
+
+It tries to answer—**right in your terminal**—questions like the following:
+
+* Colleague added `gemname` to our `Gemfile`, what is it?
+* How outdated is my favorite `gemname` I am using locally, what's changed since then?
+* What's that benchmarking `gemname`'s synopsis was, again?
+* There is `gemname` advised on the internetz for my problem, is it still maintained? Is it widely used?
+
+There are a lot of ways to answer those questions through Google and various sites, but if you are in a terminal currently, `whatthegem` is fastest, most focused and convenient.
+
+Showcase:
+
+`gem install whatthegem`
+
+## `whatthegem <gem> info`
+
+![](https://github.com/zverok/whatthegem/blob/master/screenshots/info.png?raw=true)
+
+Just extracts gem's description and version from RubyGems.org and presents it to you alongside your local versions.
+
+## `whatthegem <gem> usage`
+
+![](https://github.com/zverok/whatthegem/blob/master/screenshots/usage.png?raw=true)
+
+Tries to parse gem's GitHub README (or local README, if the gem is installed and includes it), and extract Ruby code blocks from there (except trivial, like "Add to your `Gemfile`: `gem 'gemname'`) and prints them. Typically, it is the main/most basic usage examples.
+
+## `whatthegem <gem> changes`
+
+![](https://github.com/zverok/whatthegem/blob/master/screenshots/changes.png?raw=true)
+
+Parses gem's GitHub `Changelog.md`/`NEWS`/`History`, or GitHub releases description, and lists versions and their changes (up to your local version, if it is older than recent one).
+
+## `whatthegem <gem> stats`
+
+![](https://github.com/zverok/whatthegem/blob/master/screenshots/stats.png?raw=true)
+
+Some statistics about gem's maintenance, freshness, and popularity (again, from RubyGems.org and GitHub). It doesn't _judge_, just provides helpful quick insights.
+
+> Note that, for example, "not having a new version in last year or two" doesn't necessary means the gem is abandoned, it could be just "complete".
+
+_This functionality is ported from my earlier gem [any_good](https://github.com/zverok/any_good)—after its creation, I eventually understood there are several things I'd like to be able to know about the gems, and so whatthegem was born._
+
+## Limitations
+
+* As significant amount of information is taken from GitHub, `whatthegem` is less useful for gems that aren't on GitHub, or doesn't specify their repo URL in gemspec (available via RubyGems.org API); integrating of other source hosting is possible, but currently, in my experience, it seems like \~99% of gems are there;
+* Also, GitHub usage/changes extraction for gems that aren't in the root of the repo (like [ActiveRecord](https://github.com/rails/rails/tree/v5.2.3/activerecord)) aren't supported yet;
+* As `usage` and `changes` are extracted by heuristics, it doesn't always work well (though it **does** in a surprisingly large number of cases);
+  * For example, `usage` is typically informative for stand-alone libraries, but hardly so for Rails plugins (which have instructions like "run this generator, add that to config, insert this line in the model" in their README)
+  * For other, not all gems have their Changelog findable, or parseable.
+
+## Author
+
+[@zverok](https://zverok.github.io)

data/exe/whatthegem

@@ -0,0 +1,20 @@
+#!/usr/bin/env ruby
+$LOAD_PATH.unshift File.expand_path '../lib', __dir__
+
+require 'whatthegem'
+
+name, cmd, *args = ARGV
+
+# FIXME: Currently, there is no gem named `help`, but what if it would be?
+# Maybe it is a reserved name?
+if %w[help h --help -h].include?(name) || name.nil?
+  WhatTheGem::Help.call
+  exit
+end
+
+cmd ||= 'info'
+gem = WhatTheGem::Gem.fetch(name)
+
+WhatTheGem::Command.get(cmd)
+  .then { |command| command || WhatTheGem::Help }
+  .call(gem, *args)

data/lib/whatthegem/changes/markdown_parser.rb

@@ -0,0 +1,41 @@
+module WhatTheGem
+  class Changes
+    class MarkdownParser < Parser
+      def versions
+        nodes = I::Kramdowns.elements(content)
+        level = detect_version_level(nodes) # find level of headers which contain version
+
+        sections = sections(nodes, level)
+          .select { |title,| title.match?(VERSION_LINE_REGEXP) }
+          .map(&method(:make_version))
+          .sort_by(&:number) # it is internally converted to Gem::Version, so "1.12" is correctly > "1.2"
+      end
+
+      private
+
+      def sections(nodes, level)
+        nodes
+          .chunk { |n| n.type == :header && n.options[:level] == level } # chunk into sections by header
+          .drop_while { |header,| !header }                              # drop before first header
+          .map(&:last)                                                   # drop `true`, `false` flags after chunk
+          .each_slice(2)                                                 # join header with subsequent nodes
+          .map { |(h, *), nodes| [h.options[:raw_text], nodes] }
+      end
+
+      def detect_version_level(nodes)
+        nodes
+          .select { |n| n.type == :header && n.options[:raw_text].match?(VERSION_LINE_REGEXP) }
+          .map { |n| n.options[:level] }.min
+      end
+
+      def make_version((title, nodes))
+        # TODO: date, if known
+        Version.new(
+          number: title[VERSION_LINE_REGEXP, :version],
+          header: title,
+          body: nodes.map(&I::Kramdowns.method(:el2md)).join
+        )
+      end
+    end
+  end
+end

data/lib/whatthegem/changes/parser.rb

@@ -0,0 +1,42 @@
+module WhatTheGem
+  class Changes
+    class Parser
+      extend I::Callable
+
+      class << self
+        def call(file)
+          parser_for(file).versions
+        end
+
+        def parser_for(file)
+          case file.basename
+          when /\.(md|markdown)$/i
+            MarkdownParser.new(file)
+          else
+            # Most of the time in Ruby-land, when no extension it is RDoc or RDoc-alike
+            RDocParser.new(file)
+          end
+        end
+      end
+
+      attr_reader :file
+
+      def initialize(file)
+        @file = file
+      end
+
+      memoize def versions
+        fail NotImplementedError
+      end
+
+      private
+
+      memoize def content
+        file.read
+      end
+    end
+  end
+end
+
+require_relative 'markdown_parser'
+require_relative 'rdoc_parser'

data/lib/whatthegem/changes/rdoc_parser.rb

@@ -0,0 +1,42 @@
+module WhatTheGem
+  class Changes
+    # FIXME: It kinda duplicates MarkdownParser for the _logic_, so should be unified somehow?
+    class RDocParser < Parser
+      def versions
+        nodes = I::RDocs.parts(content)
+        level = detect_version_level(nodes) # find level of headers which contain version
+
+        sections = sections(nodes, level)
+          .select { |title,| title.match?(VERSION_LINE_REGEXP) }
+          .map(&method(:make_version))
+          .sort_by(&:number) # it is internally converted to Gem::Version, so "1.12" is correctly > "1.2"
+      end
+
+      private
+
+      def sections(nodes, level)
+        nodes
+          .chunk { |n| n.is_a?(RDoc::Markup::Heading) && n.level == level } # chunk into sections by header
+          .drop_while { |header,| !header }                                 # drop before first header
+          .map(&:last)                                                      # drop `true`, `false` flags after chunk
+          .each_slice(2)                                                    # join header with subsequent nodes
+          .map { |(h, *), nodes| [h.text, nodes] }
+      end
+
+      def detect_version_level(nodes)
+        nodes.grep(RDoc::Markup::Heading)
+          .select { |n| n.text.match?(VERSION_LINE_REGEXP) }
+          .map(&:level).min
+      end
+
+      def make_version((title, nodes))
+        # TODO: date, if known
+        Version.new(
+          number: title[VERSION_LINE_REGEXP, :version],
+          header: title,
+          body: nodes.map(&I::RDocs.method(:part2md)).join
+        )
+      end
+    end
+  end
+end

data/lib/whatthegem/changes/releases_parser.rb

@@ -0,0 +1,27 @@
+module WhatTheGem
+  class Changes
+    class ReleasesParser
+      extend I::Callable
+
+      def self.call(releases)
+        new(releases).versions
+      end
+
+      attr_reader :releases
+
+      def initialize(releases)
+        @releases = releases
+      end
+
+      def versions
+        releases.map { |tag_name:, name:, body:, **|
+          Version.new(
+            number: tag_name[VERSION_LINE_REGEXP, :version],
+            header: name.then.reject(&:empty?).first || tag_name,
+            body: body
+          )
+        }.sort_by(&:number)
+      end
+    end
+  end
+end

data/lib/whatthegem/changes.rb

@@ -0,0 +1,120 @@
+module WhatTheGem
+  class Changes < Command
+    register description: 'Latest gem changes'
+
+    VERSION_REGEXP = '(v(er(sion)?)? ?)?(?<version>\d+\.\d+(\.\d+(\.\w+)?)?)'
+    VERSION_LINE_REGEXP = /(^#{VERSION_REGEXP}(\s|:|$)|\s#{VERSION_REGEXP}$)/i
+
+    CHANGELOG_NOT_FOUND = I::Pastel.red.bold("Can't find changelog to extract versions.")
+
+    class Version < Struct.new(:number, :header, :body, keyword_init: true)
+      def initialize(number:, header:, body:)
+        super(number: ::Gem::Version.new(number), header: header, body: body)
+      end
+
+      def to_h
+        super.merge(number: number.to_s)
+      end
+    end
+
+    BundledSince = Struct.new(:version) do
+      def to_h
+        {description: "your bundled version: #{version}"}
+      end
+    end
+
+    GlobalSince = Struct.new(:version) do
+      def to_h
+        {description: "your latest global version: #{version}"}
+      end
+    end
+
+    SinceFirst = Struct.new(:version) do
+      def to_h
+        {description: "the first version"}
+      end
+    end
+
+    TEMPLATE = Template.parse(<<~CHANGES)
+        _(since {{ since.description }})_
+
+      {% for version in versions %}
+      ### {{ version.header }}
+
+      {{ version.body | md_header_shift:4 }}
+
+      {% endfor %}
+
+    CHANGES
+
+    def locals
+      {
+        since: since.to_h,
+        versions: select_versions.reverse.map(&:to_h)
+      }
+    end
+
+    private
+
+    def output
+      return CHANGELOG_NOT_FOUND unless versions
+      markdown super
+    end
+
+    # TODO: allow to pass `since` as a parameter
+    def since
+      bundled_since || global_since || SinceFirst.new(versions.first.number)
+    end
+
+    def select_versions
+      return [] unless versions&.any?
+      res = versions.select { |v| v.number > since.version }
+      res = versions if res.empty? # if we are at the last version, show all historical changes
+
+      # TODO: If there are a lot of versions, we can do "intellectual selection", like
+      # two last minor + last major or something
+
+      res
+    end
+
+    def bundled_since
+      return unless gem.bundled.present?
+      gem.bundled.spec.version.then(&BundledSince.method(:new))
+    end
+
+    def global_since
+      gem.specs.first&.version&.then(&GlobalSince.method(:new))
+    end
+
+    memoize def versions
+      best_changelog(versions_from_releases, versions_from_file)
+    end
+
+    def versions_from_releases
+      gem.github&.releases&.then(&ReleasesParser)
+    end
+
+    def versions_from_file
+      # always take the freshest from GitHub, even when installed locally
+      gem.github&.changelog&.then(&Parser)
+    end
+
+    def best_changelog(*changelogs)
+      # in case they have both (releases & CHANGELOG.md)...
+      list = changelogs.compact.reject(&:empty?)
+      return list.first if list.size < 2
+
+      # Some gems have "old" changelog in file, and new in releases, or vice versa
+      if list.map { |*, last| last.number }.uniq.one?
+        # If both have the same latest version, return one having more text
+        list.max_by { |*, last| last.body.size }
+      else
+        # Return newer, if one of them are
+        list.max_by { |*, last| last.number }
+      end
+    end
+  end
+end
+
+require_relative 'changes/parser'
+require_relative 'changes/releases_parser'

data/lib/whatthegem/commands.rb

@@ -0,0 +1,101 @@
+require 'tty/markdown'
+require_relative 'tty-markdown_patch'
+
+module WhatTheGem
+  class Command
+    Meta = Struct.new(:handle, :title, :description, keyword_init: true)
+
+    class << self
+      memoize def registry
+        {}
+      end
+
+      attr_reader :meta
+
+      def register(title: name.split('::').last, handle: title.downcase, description:)
+        Command.registry[handle] = self
+        @meta = Meta.new(
+          handle: handle,
+          title: title,
+          description: description
+        )
+      end
+
+      def get(handle)
+        Command.registry[handle]
+      end
+
+      def call(*args)
+        new(*args).call
+      end
+    end
+
+    # About info shortening: It is because minitest pastes their whole README
+    # there, including a quotations of how they are better than RSpec.
+    # (At the same time, RSpec's info reads "BDD for Ruby".)
+
+    # FIXME: It was > **{{ info.info | paragraphs:1 }}** but it looks weird due to tty-markdown
+    # bug: https://github.com/piotrmurach/tty-markdown/issues/11
+    HEADER_TEMPLATE = Template.parse(<<~HEADER)
+      # {{info.name}}
+      > {{info.info | paragraphs:1 | reflow }}
+      > ({{uris | join:", "}})
+
+      ## {{title}}
+
+
+    HEADER
+
+    attr_reader :gem
+
+    def initialize(gem)
+      @gem = gem
+    end
+
+    def meta
+      self.class.meta
+    end
+
+    def call
+      puts full_output
+    end
+
+    private
+
+    def full_output
+      gm = gem # otherwise next line breaks Sublime highlighting...
+      return %{Gem "#{gm.name}" is not registered at rubygems.org.} unless gem.exists?
+      header_locals.then(&HEADER_TEMPLATE).then(&method(:markdown)) + output
+    end
+
+    def output
+      locals.then(&template)
+    end
+
+    def header_locals
+      {
+        title: meta.title,
+        info: gem.rubygems.info,
+        uris: guess_uris(gem.rubygems.info),
+      }
+    end
+
+    def markdown(text)
+      TTY::Markdown.parse(text)
+    end
+
+    def template
+      self.class::TEMPLATE
+    end
+
+    def guess_uris(info)
+      [
+        info[:source_code_uri],
+        info.values_at(:homepage_uri, :documentation_uri, :project_uri).compact.reject(&:empty?).first
+      ]
+      .compact.reject(&:empty?).uniq { |u| u.chomp('/').sub('http:', 'https:') }
+    end
+  end
+end
+
+%w[info usage changes stats help].each { |f| require_relative(f) }

data/lib/whatthegem/core_ext.rb

@@ -0,0 +1,30 @@
+require 'backports/latest'
+require 'memoist'
+require 'pp'
+require 'hm'
+require 'pathname'
+require 'forwardable'
+
+class Object
+  extend Memoist
+  alias then yield_self # will be so in Ruby 2.6
+  alias :_class :class
+end
+
+Module.include Forwardable
+
+class Pathname
+  def glob(pattern) # exists in Ruby 2.5, but not in backports
+    Dir.glob(self./(pattern).to_s).map(&Pathname.method(:new))
+  end
+end
+
+class Hm
+  class << self
+    alias call new
+  end
+
+  def self.to_proc
+    proc { |val| new(val) }
+  end
+end

data/lib/whatthegem/gem/bundled.rb

@@ -0,0 +1,55 @@
+require 'bundler'
+
+module WhatTheGem
+  class Gem
+    class Bundled
+      NoBundle = Struct.new(:name) do
+        def to_h
+          {type: 'nobundle'}
+        end
+
+        def present?
+          false
+        end
+      end
+
+      NotBundled = Struct.new(:name) do
+        def to_h
+          {type: 'notbundled', name: name}
+        end
+
+        def present?
+          false
+        end
+      end
+
+      def self.fetch(name)
+        return NoBundle.new(name) unless File.exists?('Gemfile') && File.exists?('Gemfile.lock')
+
+        definition = Bundler::Definition.build('Gemfile', 'Gemfile.lock', nil)
+        spec = definition.locked_gems.specs.detect { |s| s.name == name } or return NotBundled.new(name)
+        spec.send(:__materialize__)
+        new(spec)
+      end
+
+      attr_reader :spec
+
+      def initialize(spec)
+        @spec = spec
+      end
+
+      def present?
+        true
+      end
+
+      def to_h
+        {
+          type: 'bundled',
+          name: spec.name,
+          version: spec.version.to_s,
+          dir: spec.gem_dir
+        }
+      end
+    end
+  end
+end

data/lib/whatthegem/gem/github.rb

@@ -0,0 +1,94 @@
+module WhatTheGem
+  class Gem
+    class GitHub
+      CHANGELOG_PATTERN = /^(history|changelog|changes)(\.\w+)?$/i
+      Path = Struct.new(:basename, :read)
+
+      class << self
+        memoize def octokit
+          if (token = ENV['GITHUB_ACCESS_TOKEN'])
+            Octokit::Client.new(access_token: token).tap { |client| client.user.login }
+          else
+            Octokit::Client.new
+          end
+        end
+      end
+
+      attr_reader :repo_id
+
+      def initialize(repo_id)
+        @repo_id = repo_id
+      end
+
+      memoize def repository
+        req(:repository)
+      end
+
+      alias repo repository
+
+      memoize def last_commit
+        req(:commits, per_page: 1).first
+      end
+
+      memoize def open_issues
+        req(:issues, state: 'open', per_page: 50)
+      end
+
+      memoize def closed_issues
+        req(:issues, state: 'closed', per_page: 50)
+      end
+
+      memoize def releases
+        req(:releases)
+      end
+
+      def contents(path = '.')
+        req(:contents, path: path)
+      end
+
+      alias files contents
+
+      memoize def changelog
+        locate_file(CHANGELOG_PATTERN)
+      end
+
+      memoize def readme
+        locate_file(/^readme(\.\w+)?$/i)
+      end
+
+      private
+
+      def locate_file(pattern)
+        files.detect { |f| f.fetch(:name).match?(pattern) }
+          &.fetch(:path)
+          &.then { |path|
+            Path.new(path, contents(path).then(&method(:decode_content)))
+          }
+      end
+
+      def req(method, *args)
+        octokit.public_send(method, repo_id, *args).then(&method(:sawyer_to_hashes))
+      end
+
+      def sawyer_to_hashes(val)
+        case
+        when val.respond_to?(:to_hash)
+          val.to_hash.transform_values(&method(:sawyer_to_hashes))
+        when val.respond_to?(:to_ary)
+          val.to_ary.map(&method(:sawyer_to_hashes))
+        else
+          val
+        end
+      end
+
+      def decode_content(obj)
+        # TODO: encoding is specified as a part of the answer, could it be other than base64?
+        Base64.decode64(obj.fetch(:content)).force_encoding('UTF-8')
+      end
+
+      def octokit
+        self.class.octokit
+      end
+    end
+  end
+end

data/lib/whatthegem/gem/rubygems.rb

@@ -0,0 +1,29 @@
+module WhatTheGem
+  class Gem
+    class RubyGems
+      attr_reader :name
+
+      def initialize(name)
+        @name = name
+      end
+
+      memoize def info
+        req(:info)
+      end
+
+      memoize def versions
+        req(:versions)
+      end
+
+      memoize def reverse_dependencies
+        req(:reverse_dependencies)
+      end
+
+      private
+
+      def req(method, *args)
+        ::Gems.public_send(method, name, *args).then(&Hm).transform_keys(&:to_sym).to_h
+      end
+    end
+  end
+end