irrc 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: d0cea0cd7090c9cdc0050e02cc02368a03e48f12
+  data.tar.gz: 022c5535b1e8acb15e3641fcff65959105ebd5c7
+SHA512:
+  metadata.gz: f3883807ba6822641f396177a1fb187f0d50c96595ac9b4b33c6d2693a789ac720cd7b324aa1b780ee76898760afd247cf93803a0a9d606c668e0bf826bb9f51
+  data.tar.gz: 0e89f38a1c5b90241301ff54839949cc1778dc0194163659f8e60c23b457fe5e29cfbb25806ab41c707dffb9348eb07f29ef27a0230620445076e0d0460f728f

data/.gitignore

@@ -0,0 +1,22 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+*.bundle
+*.so
+*.o
+*.a
+mkmf.log

data/.rspec

@@ -0,0 +1,2 @@
+--color
+--require spec_helper

data/.travis.yml

@@ -0,0 +1,6 @@
+language: ruby
+rvm:
+  - 2.0.0
+  - 2.1.0
+  - 2.1.1
+  - ruby-head

data/Gemfile

@@ -0,0 +1,7 @@
+source 'https://rubygems.org'
+
+group :test do
+  gem 'rspec'
+end
+
+gemspec

data/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2014 Shintaro Kojima
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,195 @@
+# irrc
+
+[![Build Status](https://travis-ci.org/codeout/irrc.svg)](https://travis-ci.org/codeout/irrc)
+[![Code Climate](https://codeclimate.com/github/codeout/irrc.png)](https://codeclimate.com/github/codeout/irrc)
+[![Inline docs](http://inch-ci.org/github/codeout/irrc.svg)](http://inch-ci.org/github/codeout/irrc)
+
+irrc is a lightweight and flexible client of IRR / Whois Database to expand arbitrary as-set and route-set objects into a list of origin ASs and prefixes belonging to those ASs. It will concurrently queries multiple IRR / Whois Databases for performance.
+
+## Features
+
+* Fast. irrc runs multi-threaded micro clients to process simultaneous IRR / Whois queries for performance. It also uses object caches.
+* Handy. irrc's CLI client provides an easy way to resolve prefixes from as-set and route-set objects. It works even when multiple objects given as arguments.
+* Dual stack. irrc returns both IPv4 and IPv6 prefixes by default. There is no need to kick a command twice for dual stacked result.
+* Flexible. irrc provides an extensible ruby library which allows to modify IRR / Whois queries more flexibly.
+* Pretty print. irrc shows prefixes in YAML format like [this](#example).
+* Pure ruby. irrc doesn't depend on any other ruby gem.
+* Lightweight. irrc is designed to gather prefixes from arbitrary as-set and route-set. It's implemented as simple as possible to achieve that. In other words, domain name related features are not supported.
+
+
+## Installation
+
+For bundler:
+
+    gem 'irrc'
+
+And then:
+
+    $ bundle
+
+Otherwise:
+
+    $ gem install irrc
+
+
+## Usage
+
+### CLI
+
+irrc privides a peval-like CLI interface.
+
+* Query JPIRR about AS-JPNIC and AS-OCN
+
+    ```shell
+    $ irrc -h jpirr AS-JPNIC AS-OCN
+    ```
+
+* Query JPIRR about AS-JPNIC with authoritative IRR (SOURCE:) based filter
+
+    ```shell
+    $ irrc -h jpirr -s radb -s apnic AS-JPNIC
+    ```
+
+* Query JPIRR about AS-JPNIC for IPv4 only
+
+    ```shell
+    $ irrc -h jpirr -4 AS-JPNIC
+    ```
+
+### As a Library
+
+You can load irrc as a library and use it easily in your own code.
+
+```ruby
+require 'irrc'
+
+client = Irrc::Client.new
+client.query(:jpirr, 'AS-JPNIC', source: :jpirr)    # queries JPIRR about AS-JPNIC with a SOURCE: filter
+client.query(:ripe, 'AS-RIPENCC', protocol: :ipv4)  # queries RIPE Whoisd about AS-RIPENCC for IPv4 only
+client.perform                                      # returns the results in a Hash
+```
+
+
+## Example
+
+```shell
+$ irrc -h jpirr AS-JPNIC
+```
+
+will result in a YAML:
+
+```yaml
+---
+AS-JPNIC:               # queried object
+  :ipv4:
+    AS2515:             # AS-JPNIC has AS2515 as a origin AS
+    - 202.12.30.0/24    # 4 IPv4 prefixes belonging to AS2515
+    - 192.41.192.0/24   #
+    - 211.120.240.0/21  #
+    - 211.120.248.0/24  #
+  :ipv6:
+    AS2515:
+    - 2001:0fa0::/32
+    - 2001:dc2::/32
+    - 2001:DC2::/32
+```
+
+
+## Supported Ruby Versions
+
+* Ruby >= 2.0.0
+
+Successfully tested with 2.0.0, 2.1.0, 2.1.1, 2.1.2.
+
+
+## Threading
+
+irrc will send queries to multiple IRR / Whois servers simultaneously in multi-threads. Single-thread processing for each server by default.
+
+To configure the number of threads per server:
+
+### CLI
+
+```shell
+$ irrc -h jpirr -t 2 AS-JPNIC AS-OCN  # 2 threads to query JPIRR
+```
+
+### AS a Library
+
+```ruby
+client = Irrc::Client.new(2)  # 2 threads per IRR / Whois server
+```
+
+
+## Debugging
+
+irrc uses ```STDERR``` printer for a logger by default, which reports more severe messages than INFO.
+
+### CLI
+
+To display debug information including raw messages of IRR / Whois protocol:
+
+```shell
+$ irrc -h jpirr -d AS-JPNIC
+```
+
+### As a Library
+
+To use modified Logger:
+
+```ruby
+client = Irrc::Client.new {|c| c.logger = Rails.logger }
+```
+
+
+## Quick Benchmark
+
+Here is a quick performance comparison with peval and irrpt.
+
+* Ordinary as-set
+
+| CLI command                     |      user |     system |     cpu |      total |
+| :------------------------------ | --------: | ---------: | ------: | ---------: |
+| peval -h jpirr.nic.ad.jp AS-OCN |     0.14s |      0.04s |      1% |      9.397 |
+| irrpt_eval AS-OCN               |     0.35s |      0.06s |      5% |      8.021 |
+| **irrc -h jpirr AS-OCN**        | **0.42s** |  **0.09s** |  **4%** | **10.639** |
+
+* Huge as-set
+
+Object caching may contribute to the query performance.
+
+| CLI command                               |        user |     system |     cpu |       total |
+| :---------------------------------------- | ----------: | ---------: | ------: | ----------: |
+| peval -h jpirr.nic.ad.jp AS-HURRICANE     |      67.29s |      1.21s |     22% |     4:58.16 |
+| irrpt_eval AS-HURRICANE                   |     163.10s |      1.38s |     43% |     6:18.17 |
+| **irrc -h jpirr AS-HURRICANE**            |  **20.26s** |  **1.84s** |  **7%** | **4:54.37** |
+
+* Multiple as-set
+
+Multi-threading and object caching may contribute to the query performance.
+
+| CLI command                                                    |        user |     system |     cpu |       total |
+| :------------------------------------------------------------- | ----------: | ---------: | ------: | ----------: |
+| for i in AS-HURRICANE AS-GBLX; peval -h jpirr.nic.ad.jp $i     |      76.47s |      1.08s |     20% |     6:15.80 |
+| irrpt_fetch                                                    |     174.21s |      1.29s |     44% |     6:38.50 |
+| irrc -h jpirr AS-HURRICANE AS-GBLX                             |      22.28s |      1.75s |      8% |     4:34.31 |
+| **irrc -h jpirr -t 2 AS-HURRICANE AS-GBLX**                    |  **23.04s** |  **1.81s** |  **9%** | **4:09.27** |
+
+
+## Contributing
+
+Please fork it, fix and then send a pull request. :tada:
+
+To run tests just type:
+
+```shell
+$ rake
+```
+
+Please report issues or enhancement requests to [GitHub issues](https://github.com/codeout/irrc/issues).
+For questions or feedbacks write to my twitter @codeout.
+
+
+## Copyright and License
+
+Copyright (c) 2014 Shintaro Kojima. Code released under the [MIT license](LICENSE).

data/Rakefile

@@ -0,0 +1,8 @@
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec) do |spec|
+  spec.pattern = FileList['spec/**/*_spec.rb']
+end
+
+task default: :spec

data/bin/irrc

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby -Ilib
+
+require 'irrc/cli'
+
+Irrc::Cli::Client.new(ARGV).start

data/irrc.gemspec

@@ -0,0 +1,23 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'irrc/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = 'irrc'
+  spec.version       = Irrc::VERSION
+  spec.authors       = ['Shintaro Kojima']
+  spec.email         = ['goodies@codeout.net']
+  spec.summary       = 'IRR / Whois client to expand as-set and route-set into a list of origin ASs and prefixes'
+  spec.description   = 'irrc is a lightweight and flexible client of IRR / Whois Database to expand arbitrary as-set and route-set objects into a list of origin ASs and prefixes belonging to the ASs. It allows concurrent queries to IRR / Whois Databases for performance.'
+  spec.homepage      = 'https://github.com/codeout/irrc'
+  spec.license       = 'MIT'
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ['lib']
+
+  spec.add_development_dependency 'bundler', '~> 1.6'
+  spec.add_development_dependency 'rake'
+end

data/lib/irrc.rb

@@ -0,0 +1,2 @@
+require 'irrc/client'
+require 'irrc/version'

data/lib/irrc/cli.rb

@@ -0,0 +1 @@
+require 'irrc/cli/client'

data/lib/irrc/cli/client.rb

@@ -0,0 +1,108 @@
+require 'optparse'
+
+require 'irrc'
+require 'irrc/cli/yaml_printer'
+
+module Irrc
+  module Cli
+    class Client
+      def initialize(args)
+        @args = args
+        @options = Struct.new(:host, :source, :protocol, :threads, :debug).new(nil, [], [], 1, nil)
+      end
+
+      def start
+        OptionParser.new(&method(:options)).parse!(@args)
+
+        verify_arguments
+        set_default_arguments
+
+        Irrc::Cli::YamlPrinter.print perform
+
+      rescue
+        $stderr.print "#{$!.class}: " unless $!.instance_of?(RuntimeError)
+        $stderr.puts $!.message
+
+        exit 1
+      end
+
+
+      private
+
+      def options(opts)
+        opts.banner = <<-EOS
+Usage: #{opts.program_name} [options] [objects ...]
+
+Description:
+  Better IRR client to resolve as-set, route-set or aut-num object into prefixes.
+
+  If no [-4|-6|--ipv4|--ipv6] option given, it tries both of ipv4 and ipv6.
+
+Options:
+        EOS
+
+        opts.on '-h HOST', 'Specify FQDN of IRR / Whois server to send queries.',
+                           'IRR / Whois name is also acceptable. This switch is mandatory.' do |host|
+          @options.host = host
+        end
+
+        opts.on '-s SOURCE', '--source', 'Specify an authoritative IRR / Whois source name.',
+                                         'Multiply this option for multiple SOURCE.',
+                                         "eg) #{opts.program_name} -s jpirr -s radb AS-JPNIC" do |source|
+          @options.source |= [source]
+        end
+
+        opts.on '-4', '--ipv4', 'Resolve IPv4 prefixes.' do
+          @options.protocol |= [:ipv4]
+        end
+
+        opts.on '-6', '--ipv6', 'Resolve IPv6 prefixes.' do
+          @options.protocol |= [:ipv6]
+        end
+
+        opts.on '-t NUMBER', '--threads', 'Number of threads to resolve prefixes per IRR / Whois server.' do |threads|
+          @options.threads = threads
+        end
+
+        opts.on '-d', '--debug', 'Print raw queries, answers and additional informations.' do
+          @options.debug = true
+        end
+      end
+
+      def verify_arguments
+        if @args.empty?
+          $stderr.puts <<-EOS
+Missing Argument: objects required.
+
+Use --help for usage.
+          EOS
+          exit 1
+        end
+
+        unless @options.host
+          $stderr.puts <<-EOS
+Missing Argument: -h option is required.
+
+Use --help for usage.
+          EOS
+          exit 1
+        end
+      end
+
+      def set_default_arguments
+        @options.protocol = [:ipv4, :ipv6] if @options.protocol.empty?
+      end
+
+      def perform
+        client = if @options.debug
+                   Irrc::Client.new(@options.threads) {|c| c.logger = Logger.new(STDERR) }
+                 else
+                   Irrc::Client.new(@options.threads)
+                 end
+
+        client.query(@options.host, @args, source: @options.source, protocol: @options.protocol)
+        client.perform
+      end
+    end
+  end
+end

data/lib/irrc/cli/yaml_printer.rb

@@ -0,0 +1,13 @@
+require 'yaml'
+
+module Irrc
+  module Cli
+    class YamlPrinter
+      class << self
+        def print(hash)
+          puts hash.to_yaml
+        end
+      end
+    end
+  end
+end

data/lib/irrc/client.rb

@@ -0,0 +1,109 @@
+require 'irrc/irr'
+require 'irrc/irrd'
+require 'irrc/whoisd'
+require 'irrc/query'
+
+module Irrc
+
+  # Public: IRR / Whois client to manage child client workers and queues.
+  class Client
+    # Public: Create a new IRR / Whois client worker manager.
+    #         You can customize the logger by specifying a block.
+    #         The default logger is STDERR printer of more severe messages than INFO.
+    #
+    # threads - Number of threads to resolve prefixes per IRR / Whois server. (default: 1)
+    # block   - An optional block that can be used to customize the logger.
+    #
+    # Examples
+    #
+    #   Irrc::Client.new(2) {|c|
+    #     c.logger = Logger.new('irrc.log')
+    #   }
+    def initialize(threads=1, &block)
+      @thread_limit = threads.to_i
+      @block = block
+    end
+
+    # Public: Enqueue an IRR / Whois query.
+    #
+    # host    - FQDN of IRR / Whois server. IRR name is also acceptable. (eg: jpirr)
+    # objects - IRR objects to extract. (eg: as-set, route-set, aut-num object)
+    #           Array form is also acceptable for multiple objects.
+    # options - The Hash options to pass to IRR. (default: {procotol: [:ipv4, :ipv6]})
+    #           :source   - Specify authoritative IRR source names.
+    #                       If not given, any source will be accepted. (optional)
+    #           :protocol - :ipv4, :ipv6 or [:ipv4, :ipv6]
+    #                       A String or Symbol of protcol name is acceptable. (optional)
+    #
+    # Examples
+    #
+    #   client.query(:jpirr, 'AS-JPNIC', source: :jpirr, protocol: :ipv4)
+    #   client.query(:jpirr, 'AS-JPNIC', source: [:jpirr, :radb])
+    def query(host, objects, options={})
+      raise ArgumentError, 'host required.' unless host
+      fqdn = Irrc::Irr.host(host) || host
+
+      queues[fqdn] ||= Queue.new
+      Array(objects).map {|object|
+        queues[fqdn] << Irrc::Query.new(object, options)
+      }
+    end
+
+    # Public: Run the query threads.
+    #
+    # Returns Raw level Array of Queries.
+    def run
+      done = []
+
+      queues.each_with_object([]) {|(fqdn, queue), workers|
+        @thread_limit.times.map {
+          workers << Thread.start {
+            done.push *worker_class(fqdn).new(fqdn, queues[fqdn], &@block).run
+          }
+        }
+      }.each {|t| t.join }
+
+      done
+    end
+
+    # Public: Run the query threads.
+    #
+    # Returns Decorated result Hash. See an example below:
+    #
+    #   {"as-jpnic"=>                 # IRR object to query
+    #     {:ipv4=>                    # protocol
+    #       {"AS2515"=>               # origin aut-num object
+    #         ["202.12.30.0/24",      # prefixes
+    #          "192.41.192.0/24",     #
+    #          "211.120.240.0/21",    #
+    #          "211.120.248.0/24"]},  #
+    #      :ipv6=>
+    #        {"AS2515"=>
+    #          ["2001:dc2::/32",
+    #           "2001:0fa0::/32",
+    #           "2001:DC2::/32"]}}}
+    def perform
+      decorate(run)
+    end
+
+
+    private
+
+    def queues
+      @_queues ||= {}
+    end
+
+    def worker_class(fqdn)
+      type = Irrc::Irr.type(fqdn) or raise "Unknown type of IRR for '#{fqdn}'."
+      Module.const_get("Irrc::#{type.capitalize}::Client")
+    end
+
+    def decorate(queries)
+      Hash[
+        queries.map {|query|
+          [query.object, query.result.to_h.select {|_, val| val }] if query.succeeded?
+        }.compact
+      ]
+    end
+  end
+end