stubby 0.0.1

data/README.md

@@ -0,0 +1,219 @@
+# Stubby
+
+A local DNS and HTTP server combo that provides a package manager
+solution to configuring network systems on a development machine. This
+is currently only designed to run on OS X.
+
+Use it to:
+
+* manage your dev domains (like pow, with lethal power)
+
+* distribute a spec for your API so developers can run basic tests without
+hitting your dev server.
+
+* point a client to the right version of an app without editing a hosts file.
+
+* lock down access to a dev system only to users running a stubby.json config
+from your project.
+
+## Installation
+
+Install the stubby gem:
+
+		> $ sudo gem install stubby
+
+## Available Options
+
+                > $ sudo stubby -h
+                > Commands:
+                >  stubby env NAME           # Switch stubby environment
+                >  stubby help [COMMAND]     # Describe available commands or one specific command
+                >  stubby search             # View all available stubs
+                >  stubby start ENVIRONMENT  # Starts stubby HTTP and DNS servers, default env ENVIRONMENT
+                >  stubby status             # View current rules
+
+## Getting Started
+
+Stubby uses `Stubfile.json` for configuration. This file includes a mapping of
+environments to a number of rules that define server configurations and stub
+usage for the project. 
+
+                > cd ~/MyProject
+                > cat Stubfile.json
+                > {
+                >  "test": {
+                >    "dependencies": {
+                >      "example": "staging"
+                >    },
+                >
+                >    "(https?:\/\/)?example.com": "http://localhost:3000"
+                >  },
+                >
+                >  "staging": {
+                >    "dependencies": {
+                >      "example": "staging"
+                >    },
+                >
+                >    "example.com": "dns-cname://aws..."
+                >  }
+                > }
+
+                > $ sudo stubby start
+
+The 'test' and 'staging' modes for this project both include rules for the
+'example' stub, and then define a single rule of their own. Stubby starts
+by default in the 'development' environment, so with this `Stubfile.json`,
+the stubby server is not yet modifying any requests. In a new terminal:
+
+                > $ sudo stubby env test
+
+Switches stubby to test mode. Now the 'example' stub is activated, and
+additionally any requests to http or https versions of example.com are
+routed to http://localhost:3000. Let's take a look at the rules applied:
+
+                > $ sudo bin/stubby status
+                > {
+                >  "rules":{
+                >    "example":{
+                >      "_comment":"All SMTP traffic (NOT YET FUNCTIONAL)",
+                >      "admin.example.com":"10.0.1.1",
+                >      "admin2.example.com":"dns-cname://admin.example.com",
+                >      "(http?://)?merchant.example.com":"http://10.0.1.1",
+                >      "(https?://)?.*.example.io":"http://10.0.1.1",
+                >      "(https?://)?.*mail.*yahoo.*":"http://en.wikipedia.org/wiki/RTFM",
+                >      "(https?://)?yahoo.com":"https-redirect://duckduckgo.com",
+                >      "stubby\\..*":"file:///var/www/tmp",
+                >      "api.example.com":"file://~/.stubby/example/files",
+                >      "smtp://.*":"log:///var/log/out.txt"
+                >    },
+                >    "_":{
+                >      "dependencies":{
+                >        "example":"staging"
+                >      },
+                >      "(https?://)?example.com":"http://localhost:3000"
+                >    }
+                >  },
+                >  "environment":"test"
+                > }
+
+This shows us all activated rules. the "_" indicates that the rules are loaded
+from the current `Stubfile.json`. We also see that requests to yahoo.com are
+redirected to https://duckduckgo.com:
+
+To revert the system back to normal, just CTRL-C from the main stubby process.
+This will revert any changes made to configure DNS servers for all network 
+interfaces and will shut down the stubby server.
+
+## Stubbing
+
+A stub is a folder named with the name of the stub that contains a stubby.json file. The stubby.json file contains a hash with the available
+modes. Each mode contains a set of rules that define how to route DNS and how to handle potential extension requests (redirects, file server, etc).
+
+Installed stubs are in the ~/.stubby folder:
+
+		> $ ls ~/.stubby 
+		> example		system.json
+
+The example folder is the `example` stub, and the system.json file contains the agent configurations. You don't need to manually edit it.
+
+		> $ find ~/.stubby/example
+		> ... example
+		> ... example/files
+		> ... example/hello.html
+		> ... example/stubby.json
+		
+The example/stubby.json file has two modes, staging, and production:
+
+		> cat ~/.stubby/example/stubby.json
+		{ "staging": {...}, "production": {...} }
+
+Each environment contains a number of rules:
+
+		{ "staging": {
+			"MATCH_REGEXP": "INSTRUCTION"
+		} ... }		
+		
+When a request is made, either DNS or HTTP (important), the request is
+compared against the MATCH_REGEXP. If matched, the INSTRUCTION is executed. Since the same rules are consulted for DNS and HTTP, if you are
+trying to overwrite a domain, you should make sure the match won't exclude
+simply the host. For example, to proxy web traffic from test.example.com
+to a server at 10.0.1.5:
+
+		"test.example.com": "http://10.0.1.5"
+		
+This results in 
+
+		> $ dig test.example.com
+		...
+		;; ANSWER SECTION:
+		test.example.com.   0       IN      A       172.16.123.1
+ 
+172.16.123.1 is the stubby host url (TODO: configurable). All requests
+to http://test.example.com are routed to the stubby web server at that
+address.
+ 
+		> $ curl test.example.com
+ 		
+Issues a request handled by the stubby web server, which proxies the request to 172.16.123.1.
+
+## Contributing a Stub
+
+Fork this repository, update the index.json file, and submit a pull request. For
+this major version, the remote registry will just be the index.json file from
+this project's github.
+
+### DNS Only
+
+To simply override DNS for test.example.com, you can create an A record on lookup:
+
+		"test.example.com": "10.0.1.5"		
+		
+Or a CNAME, if no IP is given:
+
+		"test.example.com": "test.example2.com"
+		
+But you can be explicit in the INSTRUCTION:
+
+		"test.example.com": "dns-a://10.0.1.5"
+		"test.example.com": "dns-cname://test.example2.com"
+		
+Using the dns-#{name} convention, you can create simple references to 
+any dns record type. TODO: need to allow mx record priority somehow.
+ 
+### File Server
+
+Because stubby can intercept HTTP requests, it includes a base set of functionality that allows you two serve files directly from the stub. Given a rule:
+
+		"api.example.com": "file://~/.stubby/example/files"
+		
+DNS will resolve to the stubby server:
+
+		> $ dig api.example.com
+		... 
+		api.example.com.   0       IN      A       172.16.123.1
+		
+And a web request to api.example.com will serve files from the ~/.stubby/example/files directory:
+
+		> $ curl http://api.example.com/hello.html
+		> <html><head></head><body>Hello</body></html>
+
+This is designed to allow you to create API stubs (success responses, for instance).
+
+
+### HTTP Redirects
+
+Given a rule:
+
+		"(https?:\/\/)?yahoo.com": "http-redirect://duckduckgo.com"
+		
+DNS will resolve to the stubby server, and the web request to http://yahoo.com will redirect to http://duckduckgo.com.
+
+### Vision
+
+* protocol in instruction becomes a plugin system. dns-cname:, for instance,
+  could be handled by the dns plugin. If it didn't exist when Stubfile.json was
+  being installed, it would be installed. 
+* proxy traffic on ports and send to log systems:
+  ":25": "log-smtp://"
+  ":3306": "log-mysql://"
+* web app front-end: show emails sent, mysql queries made, etc.

data/bin/stubby

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+
+$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
+
+require 'stubby'
+require 'stubby/cli'
+
+Stubby::CLI::Application.start(ARGV)

data/lib/stubby/cli/application.rb

@@ -0,0 +1,160 @@
+require 'thor'
+
+require 'stubby/registry'
+require 'stubby/extensions/dns'
+require 'stubby/extensions/http'
+
+module Stubby
+  module CLI
+    class Application < Thor
+      default_task :start
+
+      # TODO: filesystem watch all config directories for change
+      desc "start ENVIRONMENT", "Starts stubby HTTP and DNS servers"
+      long_desc <<-LONGDESC
+        > $ sudo stubby start [ENVIRONMENT='development']
+
+        Starts the stubby HTTP and DNS servers and loads the configuration
+        from `Stubfile.json` for the named environment. If no environment
+        is given, we default to 'development'
+
+        An environment need not actually match a name in `Stubfile.json`.
+        This allows you to use environments named in dependencies but not
+        in the application. If no rules match the environment, Stubby 
+        just won't override any behaviors.
+      LONGDESC
+
+      def start(environment="development")
+        unless File.exists?("Stubfile.json")
+          puts "[ERROR]: Stubfile.json not found!"
+          return
+        end
+
+        unless permissions?
+          puts "[ERROR]: ATM I need to be run with sudo..."
+          return
+        end
+
+        if master_running?
+          puts "[ERROR]: Stubby's already running!"
+          return
+        end
+
+        environments = MultiJson.load(File.read("Stubfile.json"))
+
+        File.write(pidfile, Process.pid)
+
+        master = Stubby::Master.new(environments)
+        master.environment = environment
+        master.run!
+      end
+
+      desc "env NAME", "Switch stubby environment"
+      long_desc <<-LONGDESC
+        > $ sudo stubby env test
+        > {"status":"ok"}
+      LONGDESC
+      def env(name=nil)
+        unless master_running?
+          puts "[ERROR]: Stubby must be running to run 'environment'"
+          return
+        end
+
+        puts HTTPI.post("http://#{STUBBY_MASTER}:9000/environment.json", environment: name).body
+      end
+    
+      desc "search", "View all available stubs"
+      long_desc <<-LONGDESC
+        View all available registered stubs. These are stubs that you can use
+        as dependencies in Stubfile.json.
+
+        > $ sudo stubby search
+        > {
+        >   "example":[
+        >     {
+        >       "name":"example",
+        >       "version":"v0.0.1",
+        >       ...
+        >    }
+        >  ],
+        >  "spreedly":[
+        >    {
+        >      "name":"spreedly",
+        >      "version":"v0.0.1",
+        >       ...
+        >    }
+        >  ]
+        > }
+
+        Wildcard supported for search:
+
+        > $ sudo stubby search ex*
+        > {
+        >   "example":[
+        >     {
+        >       "name":"example",
+        >       "version":"v0.0.1",
+        >       ...
+        >    }
+        >  ]
+        > }
+
+      LONGDESC
+      def search(name=nil)
+        if master_running?
+          available = MultiJson.load(HTTPI.get("http://#{STUBBY_MASTER}:9000/stubs/available.json").body)
+        else
+          available = Stubby::Api.registry.index
+        end
+
+        puts MultiJson.dump(available.select { |key, ri| 
+          File.fnmatch(name || "*", key)
+        }, pretty: true)
+      end
+
+      desc "status", "View current rules"
+      long_desc <<-LONGDESC
+        > $ sudo bin/stubby status
+        > {
+        >   "rules":{
+        >     "example":{
+        >       "admin.example.com":"10.0.1.1",
+        >       ...
+        >     },
+        >     "_":{
+        >        "dependencies":{
+        >          "example":"staging"
+        >        },
+        >        "(https?://)?example.com":"http://localhost:3000"
+        >      }
+        >    },
+        >    "environment":"test"
+        >  }
+      LONGDESC
+      def status
+        if master_running?
+          environment = MultiJson.load(HTTPI.get("http://#{STUBBY_MASTER}:9000/environment.json").body)["environment"]
+          activated = MultiJson.load(HTTPI.get("http://#{STUBBY_MASTER}:9000/stubs/activated.json").body)
+          puts MultiJson.dump({ "rules" => activated, "environment" => environment }, pretty: true)
+        else
+          puts MultiJson.dump(status: "error", message: "Stubby currently not running")
+        end
+      end
+
+      private
+      def pidfile
+        @pidfile ||= File.expand_path("~/.stubby/pid")
+      end
+
+      def master_running?
+        Process.kill(0, File.read(pidfile).to_i)
+      rescue
+        false
+      end
+
+      def permissions?
+        `whoami`.strip == "root"
+      end
+    end
+  end
+end

data/lib/stubby/cli.rb

@@ -0,0 +1,3 @@
+require 'thor'
+
+require 'stubby/cli/application'

data/lib/stubby/extensions/dns/osx.rb

@@ -0,0 +1,58 @@
+# This module extracts the OS level DNS configuration dependency for OSX.
+module Extensions
+  module DNS
+    module OSX
+      private
+        def servers_for(interface)
+          servers = `networksetup -getdnsservers '#{interface}'`
+
+          if servers =~ /There aren't any DNS Servers/
+            return ["empty"]
+          else
+            return servers.split("\n")
+          end
+        end
+
+        def setup_reference(interface)
+          return if interface.include? "*"
+          @network_interfaces[interface] = servers_for(interface)
+          puts "[INFO] #{interface} configured with Stubby DNS. Will restore to #{@network_interfaces[interface]}"
+          `networksetup -setdnsservers '#{interface}' #{STUBBY_MASTER}`
+        end
+
+        def teardown_reference(interface, servers)
+          `networksetup -setdnsservers '#{interface}' #{servers.join(" ")}`
+          puts "[INFO] #{interface} original DNS settings restored #{servers.join(" ")}"
+        rescue => e
+          puts e.inspect
+        end
+
+        def setup_references
+          # TODO: if we connect to a new network, we'd like that to use us, too
+          return if @network_interfaces
+
+          @network_interfaces = {}
+          `networksetup listallnetworkservices`.split("\n").each do |interface|
+            next if interface.include? '*'
+            setup_reference(interface)
+          end
+
+          flush
+        end
+
+        def teardown_references
+          interfaces, @network_interfaces = @network_interfaces, nil
+
+          (interfaces || []).each do |interface, servers| 
+            teardown_reference(interface, servers)
+          end
+
+          flush
+        end
+
+        def flush
+          `dscacheutil -flushcache`
+        end
+    end
+  end
+end

data/lib/stubby/extensions/dns.rb

@@ -0,0 +1,115 @@
+require 'rubydns'
+require 'ipaddress'
+require 'uri'
+require 'pry'
+
+module Extensions
+  module DNS
+    class UnsupportedOS < Exception; end
+
+    class Server < RubyDNS::Server
+      UPSTREAM = RubyDNS::Resolver.new([[:udp, "8.8.8.8", 53], [:tcp, "8.8.8.8", 53]])
+	    IN = Resolv::DNS::Resource::IN
+
+      case RbConfig::CONFIG['host_os']
+      when /mswin|msys|mingw|cygwin|bccwin|wince|emc/
+        raise UnsupportedOS, "Sorry, Windows is not currently supported"
+      when /darwin|mac os/
+        include Extensions::DNS::OSX
+      when /linux/
+        raise UnsupportedOS, "Sorry, Linux is not currently supported"
+      else
+        raise UnsupportedOS, "Sorry, #{RbConfig::CONFIG['host_os']} wasn't recognized"
+      end
+
+      def process(name, resource_class, transaction)
+          body = HTTPI.post("http://#{STUBBY_MASTER}:9000/rules/search.json", trigger: name).body
+
+          instruction = MultiJson.load(body)
+
+    		if instruction.nil? or instruction == "@"
+    			transaction.passthrough!(UPSTREAM)
+    			return
+    		end
+
+    		url = URI.parse(instruction)
+
+    		if url.scheme.to_s.empty?
+    			url = URI.parse("dns-a://" + instruction)
+    		elsif (url.scheme.to_s =~ /^dns-.*/).nil?
+    			url.host = STUBBY_MASTER
+    		end
+
+    		response_resource_class = resource url.scheme.gsub('dns-', '')
+
+    		if url.host.to_s.empty?
+    			url.host = STUBBY_MASTER
+    		end
+
+    		if !IPAddress.valid?(url.host) and response_resource_class == IN::A
+    			response_resource_class = IN::CNAME
+    		end
+
+    		response = url.host
+
+    		if response_resource_class == IN::CNAME
+    			response = Resolv::DNS::Name.create(url.host)
+    		end
+
+    		puts "DNS: #{name} => #{response}-#{resource_class.name})"
+
+    		transaction.respond!(response, 
+    			:resource_class => response_resource_class, 
+    			:ttl => 0)
+      end
+
+      def run!(session, options)
+        return if options[:dns] == false
+        trap("INT"){ stop!  }
+
+        @session = session
+        setup_references and run_dns_server
+      end
+
+      def stop!
+        teardown_references and stop_dns_server
+      end
+
+      
+      private
+
+      def resource(pattern)
+        return IN::A unless pattern.respond_to? :to_sym
+
+        {
+          a:      IN::A,
+          aaaa:   IN::AAAA,
+          srv:    IN::SRV,
+          wks:    IN::WKS,
+          minfo:  IN::MINFO,
+          mx:     IN::MX,
+          ns:     IN::NS,
+          ptr:    IN::PTR,
+          soa:    IN::SOA,
+          txt:    IN::TXT,
+          cname:  IN::CNAME
+        }[pattern.to_sym] || IN::A
+      end
+
+      def run_dns_server
+        logger.level = Logger::INFO
+
+        EventMachine.run do
+          run(:listen => [[:tcp, STUBBY_MASTER, 53],
+            [:udp, STUBBY_MASTER, 53]])
+        end
+      end
+
+      def stop_dns_server
+        fire :stop
+        EventMachine::stop_event_loop
+      rescue
+      end
+    end
+  end
+end