reyes 1.2.6 → 1.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: aee98a9cbc8ed42310ebf0edeac2193d8c919f84
-  data.tar.gz: 461a416f7ea2a33953792d0bbbe4677c1e895e0f
+  metadata.gz: 8e9225619362879c180db3a50563afd569ff3c71
+  data.tar.gz: 241984d8db909dc5e34b2444b1906abd88d48f06
 SHA512:
-  metadata.gz: dfa634f9ec9e5f317faf5fdd6736c6e9342f88d6f88ba7c25e5b7a9cb38bb5d56b5308dada3db28d351eb0726bed005eb25146486cb14eb55e1548e28ce50d7b
-  data.tar.gz: 17b0c2e6fa7d23906845ee5edbf4a0e7327f4baff113e679e80b0a7601f26a38777c5179fc443918f158e49b97e779b820f8a1614e27cefe7252fd987fd2b746
+  metadata.gz: 7119c776a76ba54cd231c048b4ea06f18ba572538522d292bd1fd3d90f86019cc8bf38cd28c5fa7c196843faa257843ea251e0f204b4df223d98440f3778572d
+  data.tar.gz: aa90b05a80ce985576c6ea0ee81df8891bd17dd34a281129a4d543218ea4f839752701e6e7d143586163e13f1ae1fb06fa9488ec4d90bead254d944879f277ac

data/CHANGES.md

@@ -0,0 +1,3 @@
+# 1.3.0
+
+- Initial public release of Reyes

data/LICENSE

@@ -0,0 +1,22 @@
+The MIT License
+
+Copyright (c) 2015 Stripe Inc <security@stripe.com>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+

data/README.md

@@ -1,12 +1,15 @@
 Reyes
 =====
 
+<a href="https://en.wikipedia.org/wiki/Point_Reyes_Lighthouse">
 ![Pt. Reyes Lighthouse](http://upload.wikimedia.org/wikipedia/commons/thumb/5/56/Point_Reyes_Lighthouse_%28April_2012%29.jpg/1266px-Point_Reyes_Lighthouse_%28April_2012%29.jpg)
+</a>
 
 Reyes populates IPTables firewall rules based on EC2 security group rules.
-Named after the Pt. Reyes Lighthouse, which shines light through the fog,
-preventing your ships from crashing on the rocks as they make their way to
-port.
+It is named after the
+[Pt. Reyes Lighthouse](https://en.wikipedia.org/wiki/Point_Reyes_Lighthouse),
+which shines light through the fog, preventing your ships from crashing on the
+rocks as they make their way to port.
 
 Use Case
 --------
@@ -15,3 +18,101 @@ Reyes is designed to apply security group rules to IPsec VPN traffic that would
 otherwise be injected past security group protection. This is useful for
 enforcing firewalls on VPNs between EC2 instances and security groups in other
 VPCs, even in other regions.
+
+Stripe uses Reyes to ensure that we apply security group rules to traffic
+running through our IPsec VPNs between VPCs and between EC2 classic and VPC.
+
+Requirements
+------------
+
+Reyes is designed to have as few requirements as possible. It uses Amazon S3 to
+distribute rule updates in JSON, and it signs these updates with a GPG key to
+ensure rule integrity and authenticity.
+
+Gem dependencies:
+
+- [AWS SDK for Ruby Version 1](
+   https://github.com/aws/aws-sdk-ruby/tree/aws-sdk-v1)
+- [Subprocess](https://github.com/stripe/subprocess)
+- [Chalk::Log](https://github.com/stripe/chalk-log)
+
+Currently Reyes assumes that your VPC uses a CIDR block in 10.0.0.0/8. It also
+only understands TCP and UDP traffic.
+
+Architecture
+------------
+
+To avoid pummeling the EC2 API, Reyes uses a leader and follower model.
+
+A reyes leader queries the Amazon EC2 API to list the instances and security
+group rules in all desired regions and VPCs. It then uploads a signed JSON file
+to an S3 bucket. We recommend running multiple hot spare leaders for
+redundancy. (All of them will upload the same data, and the last one will win.)
+The leader should be run periodically, e.g. with a cron job every 5 minutes.
+
+A reyes follower downloads the JSON data from S3 and verifies the GPG
+signature. (This GPG key should be managed by your normal configuration
+management process.) It then creates Linux kernel
+[IP sets](http://ipset.netfilter.org/) via `ipset` that replicate the security
+group members, and populates IPTables rules with the security group data. To
+avoid replay attacks, the rule file contains a `not_after` field defaulting to
+1 hour after generation.
+
+Cutover to the new rules is atomic (using `iptables-restore`). To avoid
+accidentally locking you out of your whole infrastructure, Reyes has a few
+safeguards against applying its rules to traffic that did not travel over
+IPsec.
+
+**Caveat emptor:** Due to the use of `iptables-restore`, Reyes assumes that
+there are **no other rules** in the IPTables filter table (that includes the
+`INPUT`, `OUTPUT`, and `FORWARD` chains). All other rules in the filter table
+will be silently discarded. Reyes followers should *not* be run on servers that
+rely on having other rules in the filter table. (This may be addressed in a
+future release, but for now there is only support for a flag file to
+temporarily disable Reyes in the event that you need to change the filter table
+by some other process.)
+
+Getting Started
+---------------
+
+Questions? Feedback? Please email us! There may be hidden assumptions or
+gotchas specific to Stripe's infrastructure even though we strove to write it
+in a generic way.
+
+Reyes can be installed as a stand alone system gem: `gem install reyes`.
+
+It uses a YAML configuration file to specify the S3 bucket, GPG signing key,
+and various information about which EC2 regions and VPCs to cover.
+
+You'll want to generate a GPG signing key and place it in a keyring in some
+directory referred to by the `keyring_directory` config option. The Reyes
+leaders will need the secret keyring, while the followers should *only* have
+access to the public key. Reyes does not currently support having a passphrase
+on the key.
+
+On the EC2 classic side, Reyes assumes that any given EC2 instance will be
+running IPsec directly, so it will ignore any traffic that didn't arrive over
+IPsec. (It ignores traffic that doesn't match `-A INPUT -m policy --pol
+ipsec`.)
+
+On the VPC side, Reyes assumes that there will be dedicated VPN instances
+acting as IP routers, so on an EC2 instance it won't be possible to filter by
+policy IPsec. Instead, Reyes will filter all traffic in `10.0.0.0/8`,
+whitelisting the CIDR block of the current VPC so that all VPC-local traffic is
+allowed through (it should be filtered by normal security group rules).
+
+To configure security groups properly for Reyes in VPC, allow all traffic from
+foreign VPC (or EC2 classic) CIDR blocks that will be routed by IPsec VPN
+servers. Make sure that this security group is excluded in `config.yaml` by
+`excluded_group_names`, or else Reyes will dutifully mirror the allow rules and
+you will have no firewall whatsoever.
+
+It is strongly recommended to use `--log-accept` and `--log-drop` during
+testing. This will add IPTables logging so that you can validate that the
+firewall is taking the expected action on traffic.
+
+License
+-------
+
+Reyes is distribued under the terms of the MIT license, which can be found in
+this repository in the file called LICENSE.

data/bin/reyes

@@ -40,7 +40,7 @@ def command_fetch(region, instance_id, options)
   r.apply_data!(data, options.fetch(:apply_options))
 
   if options[:prune]
-    r.prune_ipsets
+    r.prune_ipsets(options.fetch(:prune_opts, {}))
   end
 end
 
@@ -65,7 +65,7 @@ def command_install(json_file, region, instance_id, options)
   r.apply_data!(data, options.fetch(:apply_options))
 
   if options[:prune]
-    r.prune_ipsets
+    r.prune_ipsets(options.fetch(:prune_opts, {}))
   end
 end
 
@@ -146,6 +146,11 @@ Options:
       options[:prune] = true
     end
 
+    opts.on('--prune-future', 'Prune ipsets from future run generations') do
+      options[:prune] = true
+      options[:prune_opts] = {:future_ok => true}
+    end
+
     opts.on('-a', '--archive', 'Store an archived copy of these rules') do
       options[:archive] = true
     end

data/config.yaml.example

@@ -28,6 +28,6 @@ aws:
       :max_retries: 4
 
 reyes:
-  pgp:
-    signing_key: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
+  signing_key: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
+  keyring_directory: /etc/reyes/keyring/
   nf_conntrack_max: 262144

data/lib/reyes/run_manager.rb

@@ -92,7 +92,6 @@ module Reyes
         set_nf_conntrack_max(Integer(ct_max))
       end
 
-      # XXX(richo) Should we be pruning inside run! ?
       log.info('Finished RunManager.apply_data!')
     end
 
@@ -138,9 +137,15 @@ module Reyes
     end
 
     # Remove old IPSets from previous run generations.
-    def prune_ipsets
+    #
+    # @param options [Hash]
+    #
+    # @option options :future_ok [Boolean] Whether to prune ipsets from future
+    #   generations, which are normally skipped.
+    #
+    def prune_ipsets(options={})
       log.info('Pruning old IPSets')
-      old_ipsets = []
+      prune_ipsets = []
       current_ipsets = []
 
       current_gen = @group_manager.run_generation
@@ -152,13 +157,14 @@ module Reyes
         end
 
         if s[:generation] < current_gen
-          old_ipsets << set
+          prune_ipsets << set
         elsif s[:generation] == current_gen
           current_ipsets << set
         else
           log.error("IPSet from a future generation detected: #{set.inspect}")
-          log.error("Cowardly refusing to proceed")
-          raise Reyes::Error.new("IPSet from future generation detected")
+          if options[:future_ok]
+            prune_ipsets << set
+          end
         end
       end
 
@@ -168,7 +174,7 @@ module Reyes
         raise Reyes::Error.new("Pruning would remove all IPSets")
       end
 
-      old_ipsets.each do |set|
+      prune_ipsets.each do |set|
         log.info("Pruning IPSet: #{set.name}")
         set.drop!
       end

data/lib/reyes/version.rb

@@ -1,6 +1,6 @@
 module Reyes
   # The Reyes version number
-  VERSION = '1.2.6' unless defined?(self::VERSION)
+  VERSION = '1.3.0' unless defined?(self::VERSION)
 
   # Number defining the JSON serialization format
   JSON_FORMAT_VERSION = 2 unless defined?(self::JSON_FORMAT_VERSION)

data/misc/reyes-default-drop

@@ -0,0 +1,71 @@
+#!/bin/sh
+set -eu
+
+# This script creates a default drop set of firewall rules that will apply at
+# boot before the first Reyes run. (This ensures fail-closed behavior at boot.)
+# Install this script in /etc/network/if-pre-up.d/ to ensure that it runs
+# before network interfaces are brought up.
+
+# You can generate these rules with the help of `reyes --empty`
+
+# USAGE: you must select one of the two -A INPUT choices depending on whether
+# this will be running on an instance in EC2 classic or VPC.
+
+# INPUT_RULES=''
+
+SAMPLE_INPUT_CLASSIC='
+# filter IPsec in EC2 classic
+-A INPUT -m policy --pol ipsec --dir in -j reyes-input
+'
+
+SAMPLE_INPUT_VPC='
+# filter 10.0.0.0/8 in VPC, whitelisting the self VPC
+-A INPUT -s 10.0.0.0/8 -j reyes-input
+-A reyes-input -s $LOCAL_VPC_CIDR_BLOCK -j ACCEPT
+'
+
+logger -t reyes "Importing default drop firewall rules"
+
+# generate these with reyes --empty
+
+iptables-restore <<EOM
+# Generated by Reyes, modified by hand
+*filter
+:INPUT ACCEPT
+:FORWARD ACCEPT
+:OUTPUT ACCEPT
+
+:reyes-input -
+:reyes-accept -
+:reyes-drop -
+:reyes-log-accept -
+:reyes-log-drop -
+
+$INPUT_RULES
+
+-A reyes-accept -j reyes-log-accept
+-A reyes-accept -j ACCEPT
+-A reyes-drop -j reyes-log-drop
+-A reyes-drop -j DROP
+
+-A reyes-input -p icmp -m icmp --icmp-type 3 -j ACCEPT
+-A reyes-input -p icmp -m icmp --icmp-type 4 -j ACCEPT
+-A reyes-input -p icmp -m icmp --icmp-type 11 -j ACCEPT
+-A reyes-input -p icmp -m icmp --icmp-type 12 -j ACCEPT
+-A reyes-input -p icmp -m icmp --icmp-type 8 -j ACCEPT
+-A reyes-input -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A reyes-input -m conntrack --ctstate INVALID -j reyes-drop
+
+# dynamic rules from security groups
+
+# default drop
+-A reyes-input -j reyes-drop
+
+# log rules
+-A reyes-log-accept -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[REYES ACCEPT] "
+-A reyes-log-drop -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[REYES BLOCK] "
+
+COMMIT
+EOM
+
+logger -t reyes "Done importing default drop firewall rules"

data/reyes.gemspec

@@ -20,11 +20,11 @@ Gem::Specification.new do |gem|
   gem.email     = ["security@stripe.com"]
   gem.summary   = "Reyes manages IPTables rules based on EC2 security groups."
   gem.description = <<-EOM
-    Reyes is a gem...
-
-    TO DO
+    Reyes populates IPTables firewall rules based on EC2 security group rules.
+    It applies security group rules to IPsec VPN traffic that would otherwise
+    be injected past security group protection. This is useful for enforcing
+    firewalls on VPNs between EC2 instances across VPCs, even across regions.
   EOM
-  # TODO ^
   gem.homepage = "https://github.com/stripe/reyes/"
 
   gem.files         = list_files

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: reyes
 version: !ruby/object:Gem::Version
-  version: 1.2.6
+  version: 1.3.0
 platform: ruby
 authors:
 - Andy Brody
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-08-12 00:00:00.000000000 Z
+date: 2015-10-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: aws-sdk
@@ -96,9 +96,10 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 description: |2
-      Reyes is a gem...
-
-      TO DO
+      Reyes populates IPTables firewall rules based on EC2 security group rules.
+      It applies security group rules to IPsec VPN traffic that would otherwise
+      be injected past security group protection. This is useful for enforcing
+      firewalls on VPNs between EC2 instances across VPCs, even across regions.
 email:
 - security@stripe.com
 executables:
@@ -110,7 +111,9 @@ files:
 - ".rubocop-disables.yml"
 - ".rubocop.yml"
 - ".ruby-version"
+- CHANGES.md
 - Gemfile
+- LICENSE
 - README.md
 - Rakefile
 - bin/reyes
@@ -133,6 +136,7 @@ files:
 - lib/reyes/tmp_persistent_file.rb
 - lib/reyes/utils.rb
 - lib/reyes/version.rb
+- misc/reyes-default-drop
 - reyes.gemspec
 homepage: https://github.com/stripe/reyes/
 licenses: []