holepicker 0.1

data/Gemfile

@@ -0,0 +1,2 @@
+source "http://rubygems.org"
+gemspec

data/Gemfile.lock

@@ -0,0 +1,18 @@
+PATH
+  remote: .
+  specs:
+    holepicker (0.1)
+      json (>= 1.7.7)
+      rainbow (>= 1.1.4)
+
+GEM
+  remote: http://rubygems.org/
+  specs:
+    json (1.7.7)
+    rainbow (1.1.4)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  holepicker!

data/MIT-LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2013 Jakub Suder
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.markdown

@@ -0,0 +1,94 @@
+# HolePicker
+
+HolePicker is a Ruby gem for quickly checking all your `Gemfile.lock` files for gem versions with known vulnerabilities.
+
+
+## The story
+
+The beginning of 2013 was a [really bad time](http://www.kalzumeus.com/2013/01/31/what-the-rails-security-issue-means-for-your-startup/) for the Ruby community. In the first few weeks of the year at least 7 serious security issues were found, and Rails had to be updated 4 times so far because of this. It's probably not the end. It's hard to keep track of all the issues and remember which gem versions are OK and which aren't, especially if you have several older and newer Ruby or Rails projects to maintain. So I wrote this tool in order to help with identifying which gems in your projects' gemfiles need to be updated.
+
+
+## Details
+
+The idea is that there is a [JSON file](https://github.com/psionides/holepicker/blob/master/lib/holepicker/data/data.json)\* stored in this repository that lists all the recent security-related updates to popular gems: date of the release, URL of the announcement, and a list of affected gems and updated versions. HolePicker provides a command line tool that **downloads the latest data file from GitHub every time**, scans your `Gemfile.lock` files and checks if they contain vulnerable gem versions.
+
+The reason I've done it this way is to make it easier to run the checks against the very latest version of the vulnerability list. It's kind of important to be sure that you haven't missed any last minute updates, and it would be annoying to have to check for new gem versions every time you want to run the tool (and you might not even remember to do that).
+
+If for some reason you don't want to download the JSON file every time, you can use the [`-o` option](#full-option-list). Also, the JSON file specifies the minimum compatible gem version that it can work with, so if new kind of information is added to the file that requires the gem to be updated in order to parse it, the gem will let you know.
+
+Of course the whole system still relies on me manually adding entries to the JSON file and pushing it to GitHub. I'll try to do that quickly, my trusty [@rails_bot](https://github.com/psionides/rails-retweeter-bot) notifies me pretty quickly when something really bad is happening. If for some reason I don't update the list in time, by all means please send me a pull request.
+
+(\*) YAML obviously wouldn't be appropriate, if you know what I mean.
+
+
+## Running the tool
+
+To install the tool, just run:
+
+    gem install holepicker
+
+There are two main modes of operation:
+
+### Scanning projects directly
+
+This can be used to scan project directories on your development machine:
+
+    holepicker ~/Projects
+
+You can also scan all apps deployed to a production or demo server; in this case, it's recommended to use the `-c` (`--current`) option in order to skip the old releases in `releases` directories and only scan the `current` directories (I'm assuming you use Capistrano for deployment, because who doesn't?).
+
+    holepicker -c /var/www
+
+### Scanning Nginx/Apache config directory
+
+You might have a lot of random apps deployed in the `/var/www` directory, but only some of them currently enabled in the Nginx config files. In this case, you might want to only check the apps that are actually running. To do that, use the `-f` (`--follow-roots`) option and point HolePicker to your HTTP server's config directory. It will find all the `root` or `DocumentRoot` directives and follow the paths to find the gemfiles of enabled apps.
+
+    holepicker -f /etc/nginx/sites-enabled
+
+
+## Results
+
+This is more or less what you will get if you run HolePicker in a directory with some old Rails projects:
+
+![screenshot](http://f.cl.ly/items/1l3C2c2s0r1k3v033B34/Screen%20Shot%202013-02-16%20at%2001.43.39.png)
+
+
+## Full option list
+
+`-a`, `--all`
+
+By default, HolePicker will skip directories like `.git`, `tmp`, `cached-copy` etc. when searching for gemfiles. This option turns this feature off.
+
+`-c`, `--current`
+
+Look only for gemfiles that are located directly in a `current` directory.
+
+`-f`, `--follow-roots`
+
+Look for `root`/`DocumentRoot` directives in config files at given locations instead of gemfiles directly.
+
+`-i`, `--ignore gem1,gem2,gem3`
+
+Ignore the gems passed in the parameter.
+
+`-o`, `--offline`
+
+Use an offline copy of the data file - useful if you really need to run the tool, but the network or GitHub is down.
+
+
+## Similar projects
+
+The [bundler-audit](https://github.com/postmodern/bundler-audit) project that was also created this week has a similar purpose, but it only uses an offline issue list and it only scans the current project.
+
+The [gemcanary](https://gemcanary.com/) project might be something similar, but it hasn't been released yet (as of 16.02).
+
+It might make sense to agree on a shared list of vulnerabilities in the future that these and other projects could share - no point having the same information in a few different places maintained by a few people in parallel.
+
+
+## Credits & contributing
+
+Created by [Jakub Suder](http://psionides.eu), licensed under MIT License.
+
+Any feedback and help is welcome, if you have an idea how to improve this tool, let me know or send me an issue or a pull request.
+
+And BTW, big thanks to all the smart people that find and fix all these issues - I hope you won't find much more, but please keep looking.

data/bin/holepicker

@@ -0,0 +1,53 @@
+#!/usr/bin/env ruby
+
+lib = File.expand_path('../../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+require 'holepicker'
+require 'optparse'
+
+options = {}
+
+OptionParser.new do |opts|
+  opts.banner = "Usage: #{File.basename($0)} [options] paths..."
+
+  opts.on("-a", "--all", "Don't skip directories like .git or tmp while looking for gemfiles") do
+    options[:dont_skip] = true
+  end
+
+  opts.on("-c", "--current", "Look for gemfiles only in 'current' directories") do
+    options[:current] = true
+  end
+
+  opts.on("-f", "--follow-roots", "Follow root/DocumentRoot paths in Nginx/Apache configs to find gemfiles") do
+    options[:follow_roots] = true
+  end
+
+  opts.on("-i", "--ignore gem1,gem2,gem3", Array, "Ignore given gems") do |names|
+    options[:ignored_gems] = names
+  end
+
+  opts.on("-o", "--offline", "Use an offline copy of the data.json file") do
+    options[:offline] = true
+  end
+
+  opts.on("-h", "--help", "Display this help") do
+    puts opts
+    exit
+  end
+
+  opts.on("-v", "--version", "Print gem version") do
+    puts HolePicker::VERSION
+    exit
+  end
+
+  opts.parse!
+end
+
+if ARGV.empty?
+  abort "Please choose at least one directory to scan for gemfiles."
+end
+
+success = HolePicker::Scanner.new(ARGV, options).scan
+
+exit(success ? 0 : 1)

data/lib/holepicker/data/data.json

@@ -0,0 +1,54 @@
+{
+  "min_version": "0.1",
+  "vulnerabilities": [
+    {
+      "gems": {
+        "rails": ["3.2.12", "3.1.11", "2.3.17"]
+      },
+      "url": "http://weblog.rubyonrails.org/2013/2/11/SEC-ANN-Rails-3-2-12-3-1-11-and-2-3-17-have-been-released/",
+      "date": "2013-02-11T18:40Z"
+    },
+    {
+      "gems": {
+        "json": ["1.7.7", "1.6.8", "1.5.5"]
+      },
+      "url": "https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/4_YvCpLzL58",
+      "date": "2013-02-11T18:26Z"
+    },
+    {
+      "gems": {
+        "rack": ["1.5.2", "1.4.5", "1.3.10", "1.2.8", "1.1.6"]
+      },
+      "url": "http://rack.github.com/",
+      "date": "2013-02-08T03:14Z"
+    },
+    {
+      "gems": {
+        "rails": ["3.0.20", "2.3.16"]
+      },
+      "url": "http://weblog.rubyonrails.org/2013/1/28/Rails-3-0-20-and-2-3-16-have-been-released/",
+      "date": "2013-01-28T21:08Z"
+    },
+    {
+      "gems": {
+        "devise": ["2.2.3", "2.1.3", "2.0.5", "1.5.4"]
+      },
+      "url": "http://blog.plataformatec.com.br/2013/01/security-announcement-devise-v2-2-3-v2-1-3-v2-0-5-and-v1-5-3-released/",
+      "date": "2013-01-28T15:03Z"
+    },
+    {
+      "gems": {
+        "rails": ["3.2.11", "3.1.10", "3.0.19", "2.3.15"]
+      },
+      "url": "http://weblog.rubyonrails.org/2013/1/8/Rails-3-2-11-3-1-10-3-0-19-and-2-3-15-have-been-released/",
+      "date": "2013-01-08T20:26Z"
+    },
+    {
+      "gems": {
+        "rails": ["3.2.10", "3.1.9", "3.0.18", "2.3.15"]
+      },
+      "url": "http://weblog.rubyonrails.org/2013/1/2/Rails-3-2-10--3-1-9--and-3-0-18-have-been-released/",
+      "date": "2013-01-02T21:39Z"
+    }
+  ]
+}

data/lib/holepicker/database.rb

@@ -0,0 +1,22 @@
+require 'holepicker/vulnerability'
+require 'json'
+require 'rubygems'
+
+module HolePicker
+  class Database
+    attr_reader :vulnerabilities
+
+    def self.load_from_json_file(data)
+      new(JSON.parse(data))
+    end
+
+    def initialize(json)
+      @vulnerabilities = json['vulnerabilities'].reverse.map { |v| Vulnerability.new(v) }
+      @min_version = ::Gem::Version.new(json['min_version'])
+    end
+
+    def compatible?
+      HolePicker.version >= @min_version
+    end
+  end
+end

data/lib/holepicker/gem.rb

@@ -0,0 +1,21 @@
+require 'rubygems'
+
+module HolePicker
+  class Gem
+    GEM_LINE_PATTERN = /([\w\-]+) \(([^)]+)\)/
+
+    attr_reader :name, :version
+
+    def initialize(line)
+      result = line.match(GEM_LINE_PATTERN)
+      raise "Invalid gem format: #{line}" unless result
+
+      @name = result[1]
+      @version = ::Gem::Version.new(result[2])
+    end
+
+    def to_s
+      "#{name} (#{version})"
+    end
+  end
+end

data/lib/holepicker/offline_database.rb

@@ -0,0 +1,14 @@
+require 'holepicker/database'
+
+module HolePicker
+  class OfflineDatabase < Database
+    OFFLINE_JSON_FILE = File.expand_path('../data/data.json', __FILE__)
+
+    def self.load
+      load_from_json_file(File.read(OFFLINE_JSON_FILE))
+    rescue Exception => e
+      puts "Can't load local data file: #{e}"
+      exit 1
+    end
+  end
+end

data/lib/holepicker/online_database.rb

@@ -0,0 +1,56 @@
+require 'holepicker/database'
+require 'holepicker/utils'
+require 'net/http'
+
+module HolePicker
+  class OnlineDatabase < Database
+    # TODO temporary link
+    URL='http://pastie.org/pastes/6183429/download?key=qryhowarb9i7hoqqyvy0q'
+
+    def self.load
+      puts "Fetching list of vulnerabilities..."
+
+      load_from_json_file(http_get(URL)).tap do |db|
+        db.check_compatibility
+        db.report_new_vulnerabilities
+      end
+    rescue SystemExit
+      raise
+    rescue Exception => e
+      puts "Can't download latest data file: #{e}"
+      exit 1
+    end
+
+    def self.http_get(url)
+      uri = URI(url)
+      http = Net::HTTP.new(uri.host, uri.port)
+      http.use_ssl = url.start_with?('https')
+
+      response = http.get(uri.request_uri)
+      response.body
+    end
+
+    def check_compatibility
+      unless compatible?
+        puts "You need to upgrade holepicker to version #{@min_version} or later."
+        exit 1
+      end
+    end
+
+    def report_new_vulnerabilities
+      new_vulnerabilities = @vulnerabilities.select(&:recent?)
+      count = new_vulnerabilities.length
+
+      if count > 0
+        puts "#{count} new #{Utils.pluralize(count, 'vulnerability')} found in the last " +
+          "#{Vulnerability::NEW_VULNERABILITY_DAYS} days:"
+
+        new_vulnerabilities.each do |v|
+          puts "#{v.day} (#{v.gem_names.join(', ')}): #{v.url}"
+        end
+
+        puts
+      end
+    end
+  end
+end

data/lib/holepicker/scanner.rb

@@ -0,0 +1,128 @@
+require 'holepicker/gem'
+require 'holepicker/offline_database'
+require 'holepicker/online_database'
+require 'holepicker/utils'
+require 'rainbow'
+require 'set'
+
+module HolePicker
+  class Scanner
+    SKIPPED_DIRECTORIES = ["-name cached-copy", "-path '*/bundle/ruby'", "-name tmp", "-name '.*'"]
+    ROOT_LINE_PATTERN = %r{\b(?:root|DocumentRoot)\s+(.*)/public\b}
+    GEMFILE_GEM_PATTERN = %r(^ {4}[^ ])
+
+    def initialize(paths, options = {})
+      @paths = paths.is_a?(Array) ? paths : [paths]
+
+      @database = options[:offline] ? OfflineDatabase.load : OnlineDatabase.load
+
+      @ignored = options[:ignored_gems] || []
+      @skip = !options[:dont_skip]
+      @current = options[:current]
+      @roots = options[:follow_roots]
+    end
+
+    def scan
+      puts "Looking for gemfiles..."
+
+      @found_vulnerabilities = Set.new
+      @matched_gemfiles = 0
+      @matched_gems = 0
+
+      @paths.each { |p| scan_path(p) }
+
+      print_report
+
+      @matched_gems == 0
+    end
+
+
+    private
+
+    def vulnerabilities_for_gem(gem)
+      @database.vulnerabilities.select { |v| v.gem_vulnerable?(gem) }
+    end
+
+    def find_gemfiles_in_path(path)
+      skips = SKIPPED_DIRECTORIES.join(" -or ")
+      gemfiles = @current ? "-path '*/current/Gemfile.lock'" : "-name 'Gemfile.lock'"
+
+      command = if @skip
+        "find -L #{path} \\( #{skips} \\) -prune -or #{gemfiles} -print"
+      else
+        "find -L #{path} #{gemfiles}"
+      end
+
+      run_and_read_lines(command)
+    end
+
+    def find_gemfiles_in_configs(path)
+      configs = run_and_read_lines("find -L #{path} -type f -or -type l")
+      configs = select_existing(configs)
+
+      directories = configs.map { |f| File.read(f).scan(ROOT_LINE_PATTERN) }
+      gemfiles = directories.flatten.map { |dir| "#{dir}/Gemfile.lock" }
+
+      select_existing(gemfiles)
+    end
+
+    def read_gemfile(path)
+      File.readlines(path).select { |l| l =~ GEMFILE_GEM_PATTERN }.map { |l| Gem.new(l) }
+    end
+
+    def scan_path(path)
+      gemfiles = @roots ? find_gemfiles_in_configs(path) : find_gemfiles_in_path(path)
+      gemfiles.each { |f| scan_gemfile(f) }
+    end
+
+    def scan_gemfile(path)
+      print "#{path}: "
+
+      gems = read_gemfile(path)
+      gems.delete_if { |g| @ignored.include?(g.name) }
+
+      vulnerable_gems = gems.map { |g| [g, vulnerabilities_for_gem(g)] }
+      vulnerable_gems.delete_if { |g, v| v.empty? }
+
+      count = vulnerable_gems.length
+
+      if count == 0
+        puts "OK"
+      else
+        puts "#{count} vulnerable #{Utils.pluralize(count, 'gem')} found!".color(:red)
+
+        vulnerable_gems.each do |gem, vulnerabilities|
+          puts "- #{gem} [#{vulnerabilities.map(&:tag).join(',')}]"
+
+          @found_vulnerabilities.merge(vulnerabilities)
+          @matched_gems += 1
+        end
+
+        @matched_gemfiles += 1
+
+        puts
+      end
+    end
+
+    def print_report
+      if @matched_gemfiles == 0
+        puts "No vulnerabilities found."
+      else
+        puts ("#{@matched_gems} vulnerable #{Utils.pluralize(@matched_gems, 'gem')} found in " +
+          "#{@matched_gemfiles} #{Utils.pluralize(@matched_gemfiles, 'gemfile')}!").color(:red) + "\n\n"
+
+        @found_vulnerabilities.sort_by(&:id).each do |v|
+          puts "[#{v.tag}] #{v.day}: #{v.url}"
+        end
+      end
+    end
+
+    def select_existing(files)
+      files.select { |f| File.exist?(f) }
+    end
+
+    def run_and_read_lines(command)
+      %x(#{command}).lines.map(&:strip)
+    end
+  end
+end

data/lib/holepicker/utils.rb

@@ -0,0 +1,11 @@
+module HolePicker
+  module Utils
+    def self.pluralize(count, name)
+      if count == 1
+        name
+      else
+        name.gsub(/y$/, 'ie') + 's'
+      end
+    end
+  end
+end

data/lib/holepicker/version.rb

@@ -0,0 +1,9 @@
+require 'rubygems'
+
+module HolePicker
+  VERSION = "0.1"
+
+  def self.version
+    ::Gem::Version.new(VERSION)
+  end
+end

data/lib/holepicker/vulnerability.rb

@@ -0,0 +1,66 @@
+require 'rubygems'
+require 'time'
+
+module HolePicker
+  class Vulnerability
+    NEW_VULNERABILITY_DAYS = 7
+    NEW_VULNERABILITY_TIME = NEW_VULNERABILITY_DAYS * 86400
+
+    attr_reader :id, :date, :url, :gems
+
+    def self.next_id
+      @@count ||= 0
+      @@count += 1
+    end
+
+    def initialize(json)
+      @gems = {}
+
+      json['gems'].each do |name, versions|
+        @gems[name] = versions.map { |v| ::Gem::Version.new(v) }
+      end
+
+      @id = self.class.next_id
+      @url = json['url']
+      @date = Time.parse(json['date'])
+    end
+
+    def day
+      @date.strftime("%Y-%m-%d")
+    end
+
+    def recent?
+      date > Time.now - NEW_VULNERABILITY_TIME
+    end
+
+    def tag
+      "##{@id}"
+    end
+
+    def gem_names
+      @gems.keys
+    end
+
+    def gem_vulnerable?(gem)
+      !gem_safe?(gem)
+    end
+
+    def gem_safe?(gem)
+      fixes = @gems[gem.name]
+      !fixes || fixes.any? { |fix| fix_included?(fix, gem) } || fixes.all? { |fix| gem.version > fix }
+    end
+
+
+    private
+
+    def fix_included?(fix, gem)
+      gem.version == fix || (gem.version > fix && same_level?(fix, gem))
+    end
+
+    def same_level?(fix, gem)
+      segments_count = fix.segments.length
+
+      fix.segments[0...segments_count-1] == gem.version.segments[0...segments_count-1]
+    end
+  end
+end

data/lib/holepicker.rb

@@ -0,0 +1,2 @@
+require 'holepicker/scanner'
+require 'holepicker/version'

metadata

@@ -0,0 +1,94 @@
+--- !ruby/object:Gem::Specification
+name: holepicker
+version: !ruby/object:Gem::Version
+  version: '0.1'
+  prerelease: 
+platform: ruby
+authors:
+- Jakub Suder
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-02-16 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: json
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.7.7
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.7.7
+- !ruby/object:Gem::Dependency
+  name: rainbow
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.1.4
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.1.4
+description: 
+email: jakub.suder@gmail.com
+executables:
+- holepicker
+extensions: []
+extra_rdoc_files: []
+files:
+- MIT-LICENSE.txt
+- README.markdown
+- Changelog.markdown
+- Gemfile
+- Gemfile.lock
+- lib/holepicker/data/data.json
+- lib/holepicker/database.rb
+- lib/holepicker/gem.rb
+- lib/holepicker/offline_database.rb
+- lib/holepicker/online_database.rb
+- lib/holepicker/scanner.rb
+- lib/holepicker/utils.rb
+- lib/holepicker/validator.rb
+- lib/holepicker/version.rb
+- lib/holepicker/vulnerability.rb
+- lib/holepicker.rb
+- bin/holepicker
+homepage: http://github.com/psionides/holepicker
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.24
+signing_key: 
+specification_version: 3
+summary: A tool for checking gem versions in Gemfile.lock files for known vulnerabilities
+test_files: []