trusted-sandbox 0.0.2.pre

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: f04a7997f512db9c4d422f4b5ca6f13e26b96c0e
+  data.tar.gz: 4e1ff6efa79e0353d880a4ea7bac92f680e464e3
+SHA512:
+  metadata.gz: 72fb2983a31e0908a2d085b53c43d4e8452fb3edc3e81ca4d7af264bec9f8583211bc57a28c436dc6f3bade87645a94650e18f924ee27ba2780efd2afd56bd92
+  data.tar.gz: 6305312ad3faafa2295f16c47cbbfb4e8287185f986ac0c9ead45ef4340438e220c6f59f912e30f726c1fc689e1d4f92a459bb0e8c01211494f95fc6b4f00521

data/.gitignore

@@ -0,0 +1,8 @@
+.idea
+.DS_Store
+tmp/
+.swp
+pkg/
+config/trusted_sandbox.yml
+trusted_sandbox.yml
+trusted_sandbox_images/

data/.ruby-gemset

@@ -0,0 +1 @@
+trusted-sandbox

data/.ruby-version

@@ -0,0 +1 @@
+ruby-2.1.2

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in trusted_sandbox.gemspec
+gemspec

data/Gemfile.lock

@@ -0,0 +1,27 @@
+PATH
+  remote: .
+  specs:
+    trusted-sandbox (0.0.2.pre)
+      docker-api (~> 1.13)
+      thor (~> 0.19)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    archive-tar-minitar (0.5.2)
+    docker-api (1.13.6)
+      archive-tar-minitar
+      excon (>= 0.38.0)
+      json
+    excon (0.40.0)
+    json (1.8.1)
+    rake (10.1.0)
+    thor (0.19.1)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  bundler (~> 1.3)
+  rake
+  trusted-sandbox!

data/LICENSE

@@ -0,0 +1,24 @@
+The MIT License (MIT)
+
+Copyright (c) 2014 Amit Aharoni
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation
+files (the "Software"), to deal in the Software without
+restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,400 @@
+# Trusted Sandbox
+
+Run untrusted ruby code in a contained sandbox, using Docker. This gem was inspired by [Harry Marr's work][1].
+
+## Instant gratification
+
+Trusted Sandbox makes it simple to execute Ruby classes that `eval` untrusted code in a resource-controlled docker container.
+```ruby
+# lib/my_function.rb
+
+class MyFunction
+  attr_reader :input
+
+  def initialize(user_code, input)
+    @user_code = user_code
+    @input = input
+  end
+
+  def run
+    eval @user_code
+  end
+end
+```
+```ruby
+# somewhere_else.rb
+require 'trusted_sandbox'
+require 'lib/my_function'
+
+untrusted_code = "input[:number] ** 2"
+
+# The following will run inside a Docker container
+output = TrustedSandbox.run! MyFunction, untrusted_code, {number: 10}
+# => 100
+```
+
+Classes you want to run in a container need to respond to #initialize and #run. Trusted Sandbox serializes the
+arguments sent to #initialize, loads the container, instantiates an object, and calls #run.
+
+## Installing
+
+### Step 1
+Add this line to your application's Gemfile:
+```
+gem 'trusted-sandbox'
+```
+
+And then execute:
+```
+$ bundle
+```
+Or install it yourself as:
+```
+$ gem install trusted-sandbox
+```
+
+Then, run the following command which will copy the `trusted_sandbox.yml` file into your current directory, or
+`config` directory if it exists:
+```
+$ trusted_sandbox install
+```
+
+### Step 2
+Install Docker. Refer to the Docker documentation to see how to install Docker on your environment.
+
+### Step 3
+Install the image. This step is optional, as Docker automatically installs images when you first run them. However,
+since it takes a few minutes we suggest you do this in advance.
+```
+$ docker run --rm vaharoni/trusted_sandbox:2.1.2.v1
+```
+If you see the message "you must provide a uid", then you are set.
+
+If you receive an error that looks like this: `Error response from daemon: Cannot start container 9f3bd8d72f0704980cedacc068261c38e280e7314916245550a6d48431ea8f11: fork/exec /var/lib/docker/init/dockerinit-1.0.1: cannot allocate memory`
+consider restarting docker:
+```
+$ sudo service docker.io restart
+```
+and then try again.
+
+### Step 4
+
+If you'd like to limit swap memory or set user quotas you'll have to install additional programs on your server.
+Follow the instructions in the relevant sections of the configuration guide.
+
+## Configuring Trusted Sandbox
+
+Let's go over the sections of the YAML configuration file you created in step 1 above.
+
+### Docker access
+```ruby
+  # ENV['DOCKER_HOST'] is used if omitted
+  docker_url: https://192.168.59.103:2376
+
+  # ENV['DOCKER_CERT_PATH'] is used if omitted
+  docker_cert_path: ~/.boot2docker/certs/boot2docker-vm
+
+  docker_image_name: vaharoni/trusted_sandbox:2.1.2.v1
+
+  # Optional authentication
+  docker_login:
+    user: my_user
+    password: my_password
+    email: email@email.com
+
+```
+
+Trusted Sandbox uses the `docker-api` gem to communicate with docker. Some of the parameters above are used to setup
+the global `Docker` class. For finer control of its configuration, you can add a `docker_options` hash entry to the
+YAML file which will override any configuration and passed through to `Docker.options`.
+
+### Limiting resources
+CPU:
+```ruby
+  cpu_shares:        1              # In relative units
+```
+Memory:
+```ruby
+  memory_limit:      52_428_800     # In bytes
+  enable_swap_limit: false
+  memory_swap_limit: 52_428_800     # In bytes. Relevant only if enable_swap_limit is true.
+```
+Execution
+```ruby
+  execution_timeout: 15             # In seconds
+  network_access:    false
+```
+Quotas
+```ruby
+  enable_quotas:     false
+```
+Settings for UID-pool used for assigning user quotas. Always used, even if quota functionality is disabled.
+It's very unlikely you'll need to touch these:
+```ruby
+  pool_size:         5000
+  pool_min_uid:      20000
+  pool_timeout:      3
+  pool_retries:      5
+  pool_delay:        0.5
+```
+Note that controlling memory swap limits and user quotas requires additional steps as outlined below.
+
+### Execution parameters
+
+A temporary directory under which sub directories are created and mounted to containers.
+The code and args exchange between the host and containers is done via these sub directories.
+
+```ruby
+  host_code_root_path: tmp/code_dirs
+```
+
+When set to true, the temporary sub directories will not be erased. This allows you to login to the container to
+troubleshoot issues as explained in the "Troubleshooting" section.
+```ruby
+  keep_code_folders: false
+```
+
+A directory used by the UID-pool to handle locks.
+```ruby
+  host_uid_pool_lock_path: tmp/uid_pool_lock
+```
+
+### Limiting swap memory
+
+In order to limit swap memory, you'll need to set up your host server to allow that.
+The following should work for Debian / Ubuntu.
+
+First, run:
+```
+$ sudoedit /etc/default/grub
+```
+and edit the following line:
+```
+GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"
+```
+Then run:
+```
+$ sudo update-grub
+```
+Reboot the server, and you should be set. Read more about it [here][2].
+Remember to set `enable_swap_limit: true` in the YAML file.
+
+### Limiting user quotas
+
+Note: due to permission setting scheme, limiting user quota does not work on OS or Windows.
+
+In order to control quotas we follow the technique suggested by [Harry Marr][3]. It makes use of the fact that
+UIDs (user IDs) and GIDs (Group IDs) are shared between the host and its containers. When a container starts, we
+run the untrusted code under an unprivileged user whose UID has a quota enforced by the host.
+
+In order to enable quotas do the following on the server:
+```
+$ sudo apt-get install quota
+```
+And follow [these instructions][4], which are also brought here for completeness:
+```
+$ sudo vim /etc/fstab
+```
+Add `,usrquota` in the end of column no. 4 so it looks something like:
+```
+LABEL=cloudimg-rootfs   /        ext4   defaults,discard,usrquota       0 0
+```
+Then do:
+```
+$ mount -o remount
+```
+and reboot the server. Finally, run the following (quota is in KB):
+```
+$ trusted_sandbox set_quotas 10000
+```
+This sets ~10MB quota on all UIDs that are in the range defined by `pool_size` and `pool_min_uid` parameters. If you
+change these configuration parameters you must rerun the `set_quotas` command.
+
+Remember to set `enable_quotas: true` in the YAML file.
+
+Note: At this time, there is no way to assign different quotas to different users.
+
+### Limiting network
+
+The only option available is to turn on and off network access using `enable_network`. Finer control of network
+access is currently not supported. If you need this feature please open an issue and share your use case.
+
+## Using Trusted Sandbox
+
+### Class and argument serialization
+
+The class you send to a container can be as elaborate as you want, providing a context of execution for the user code.
+When you call `run` or `run!` with a class constant, the file where that class is defined is copied to the
+`/home/sandbox/src` folder inside the container. Any arguments needed to instantiate an object from that class are
+serialized. When the container starts, it deserializes these arguments, invokes the `new` method with them, and runs
+`run` on the instantiated object. The output of that method is then serialized back to the host.
+
+A less trivial example:
+```ruby
+# my_function.rb
+
+# Example for requiring a gem, assuming it is in the Gemfile of both the container and the
+# host. If you want to access a gem that is only available to the container, put the require
+#  directive inside `initialize` or `run` methods.
+require 'hashie/mash'
+
+class MyFunction
+
+  attr_reader :a, :b
+  def initialize(first_user_func, second_user_func, a, b)
+    @first_user_func = first_user_func
+    @second_user_func = second_user_func
+    @a = a
+    @b = b
+  end
+
+  def run
+    # Will have access to #a and #b through attr_reader
+    result1 = eval(@first_user_func)
+
+    result2 = Context.new(result1).run(@second_user_func)
+    [result1, result2]
+  end
+
+  class Context
+    attr_reader :x
+    def initialize(x)
+      @x = x
+    end
+
+    def run(code)
+      eval code
+    end
+  end
+end
+```
+```ruby
+# Somewhere else
+require 'trusted_sandbox'
+require 'my_function'
+a, b = TrustedSandbox.run! MyFunction, "a + b", "x ** 2", 2, 5
+# => 49
+```
+Because serialization occurs through Marshalling, you should use primitive Ruby classes for your inputs as much as
+possible. You can prepare a docker image with additional gems and custom Ruby classes, as explained in the
+"Using custom docker images" section.
+
+### Running containers
+
+There are two ways to run a container. Use `run!` to retrieve output from the container. If the user code raised
+an exception, it will be raised by `run!`.
+
+```ruby
+output = TrustedSandbox.run! MyFunction, "input ** 2", 10
+# => 100
+```
+Use `run` to retrieve a response object. The response object provides additional useful information about the
+container execution.
+
+Here is an error scenario:
+```ruby
+response = TrustedSandbox.run MyFunction, "raise 'error!'", 10
+
+response.status
+# => "error"
+
+response.valid?
+# => false
+
+response.output
+# => nil
+
+response.output!
+# => TrustedSandbox::UserCodeError: error!
+
+response.error
+# => #<RuntimeError: error!>
+
+response.error.backtrace
+# => /home/sandbox/src/my_function.rb:14:in `eval'
+# => /home/sandbox/src/my_function.rb:14:in `eval'
+# => /home/sandbox/src/my_function.rb:14:in `run'
+
+# Can be useful if MyFunction prints to stdout
+puts response.stdout
+
+# Can be useful for environment related errors
+puts response.stderr
+```
+Here is a success scenario:
+```ruby
+response = TrustedSandbox.run MyFunction, "input ** 2", 10
+
+response.status
+# => "success"
+
+response.valid?
+# => true
+
+response.output
+# => 100
+
+response.output!
+# => 100
+
+response.error
+# => nil
+```
+### Overriding specific invocations
+
+To override a configuration parameter for a specific invocation, use `with_options`:
+```ruby
+TrustedSandbox.with_options(cpu_shares: 2) do |s|
+  s.run! MyFunction, untrusted_code, input
+end
+```
+You should not override user quota related parameters, as they must be prepared on the host in advance of execution.
+
+## Using custom docker images
+
+Trusted Sandbox comes with one ready-to-use image that includes Ruby 2.1.2. It is hosted on Docker Hub under
+`vaharoni/trusted_sandbox:2.1.2.v1`.
+
+To use a different image from your Docker Hub account simply change the configuration parameters in the YAML file.
+
+To customize the provided images, run the following. It will copy the image definition to your current directory under
+`trusted_sandbox_images/2.1.2`.
+```
+$ trusted_sandbox generate_image
+```
+
+After modifying the files to your satisfaction, you can either push it to your Docker Hub account, or build directly
+on the server. Assuming you kept the image under trusted_sandbox_images/2.1.2:
+```
+$ docker build -t "your_user/your_image_name:your_image_version" trusted_sandbox_images/2.1.2
+```
+
+## Troubleshooting
+
+If you encounter issues, try troubleshooting them by accessing your container's bash. Make the following change in the
+YAML file:
+
+```ruby
+keep_code_folders: true
+```
+This will keep your code folders from getting deleted when containers stop running. This allows you to do the
+following from your command line (adjust to your environment):
+```
+$ docker run -it -v /home/MyUser/my_app/tmp/code_dirs/20000:/home/sandbox/src --entrypoint="/bin/bash" my_user/my_image:my_tag
+```
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
+
+## License
+Licensed under the [MIT license](http://opensource.org/licenses/MIT).
+
+[1]: http://hmarr.com/2013/oct/16/codecube-runnable-gists/
+[2]: https://www.digitalocean.com/community/articles/how-to-enable-user-quotas
+[3]: http://hmarr.com/2013/oct/16/codecube-runnable-gists/
+[4]: https://www.digitalocean.com/community/tutorials/how-to-enable-user-quotas

data/Rakefile

@@ -0,0 +1 @@
+require 'bundler/gem_tasks'

data/bin/trusted_sandbox

@@ -0,0 +1,4 @@
+#!/usr/bin/env ruby
+
+require 'trusted_sandbox/cli'
+TrustedSandbox::Cli.start(ARGV)

data/lib/trusted_sandbox/cli.rb

@@ -0,0 +1,43 @@
+require 'thor'
+
+module TrustedSandbox
+  class Cli < Thor
+    desc 'install', 'Creates trusted_sandbox.yml in `config`, if this directory exists, or in the current directory otherwise'
+    def install
+      curr_dir_file = 'trusted_sandbox.yml'
+      config_dir_file = 'config/trusted_sandbox.yml'
+
+      puts "#{curr_dir_file} already exists" or return if File.exist?(curr_dir_file)
+      puts "#{config_dir_file} already exists" or return if File.exist?(config_dir_file)
+
+      target_file = Dir.exist?('config') ? config_dir_file : curr_dir_file
+
+      puts "Creating #{target_file}"
+      FileUtils.cp File.expand_path('../config/trusted_sandbox.yml', __FILE__), target_file
+    end
+
+    desc 'generate_image VERSION', 'Creates the Docker image files and places them into the `trusted_sandbox_images` directory. Default version is 2.1.2'
+    def generate_image(image_version = '2.1.2')
+      target_dir = 'trusted_sandbox_images'
+      target_image_path = "#{target_dir}/#{image_version}"
+      gem_image_path = File.expand_path("../server_images/#{image_version}", __FILE__)
+
+      puts "Image #{image_version} does not exist" unless Dir.exist?(gem_image_path)
+      puts "Directory #{target_image_path} already exists" or return if Dir.exist?(target_image_path)
+
+      puts "Copying #{image_version} into #{target_image_path}"
+      FileUtils.mkdir_p target_dir
+      FileUtils.cp_r gem_image_path, target_image_path
+    end
+
+    desc 'set_quotas QUOTA_KB', 'Sets the quota for all the UIDs in the pool. This requires additional installation. Refer to the README file.'
+    def set_quotas(quota_kb)
+      from = TrustedSandbox.config.pool_min_uid
+      to = TrustedSandbox.config.pool_max_uid
+      puts "Configuring quota for UIDs [#{from}..#{to}]"
+      (from..to).each do |uid|
+        `setquota -u #{uid} 0 #{quota_kb} 0 0 /`
+      end
+    end
+  end
+end

data/lib/trusted_sandbox/config/trusted_sandbox.yml

@@ -0,0 +1,32 @@
+development:
+  # Optional login information
+  docker_login:
+    user:                       my_user
+    password:                   my_password
+    email:                      email@email.com
+
+  docker_url:                   https://192.168.59.103:2376
+  docker_cert_path:             ~/.boot2docker/certs/boot2docker-vm
+
+  docker_image_name:            vaharoni/trusted_sandbox:2.1.2.v1
+
+  cpu_shares:                   1
+
+  memory_limit:                 52_428_800            # 50 MB
+  enable_swap_limit:            false
+  memory_swap_limit:            52_428_800            # 50 MB
+
+  execution_timeout:            15
+  network_access:               false
+
+  enable_quotas:                false
+
+  host_code_root_path:          tmp/code_dirs
+  host_uid_pool_lock_path:      tmp/uid_pool_lock
+  keep_code_folders:            false
+
+  pool_size:                    5000
+  pool_min_uid:                 20000
+  pool_timeout:                 3
+  pool_retries:                 5
+  pool_delay:                   0.5

data/lib/trusted_sandbox/config.rb

@@ -0,0 +1,103 @@
+module TrustedSandbox
+
+  # Allows chaining so that specific invocations can override configurations.
+  # Usage:
+  #   general_config = Defaults.new.override(pool_size: 10, memory_limit: 100)
+  #   specific_invocation = general_config.override(memory_limit: 200)
+  #
+  class Config
+    attr_reader :other_config
+
+    def self.attr_reader_with_fallback(*names)
+      names.each do |name|
+        define_method name do
+          value = instance_variable_get("@#{name}")
+          return value unless value.nil?
+          return other_config.send(name) if other_config.respond_to?(name)
+          nil
+        end
+      end
+    end
+
+    def self.attr_accessor_with_fallback(*names)
+      names.each do |name|
+        attr_reader_with_fallback(name)
+        attr_writer(name)
+      end
+    end
+
+    attr_accessor_with_fallback :pool_size, :pool_min_uid, :pool_timeout, :pool_retries, :pool_delay, :docker_options,
+                                :memory_limit, :memory_swap_limit, :cpu_shares, :docker_image_name,
+                                :execution_timeout, :network_access, :enable_swap_limit, :enable_quotas,
+                                :container_code_path, :container_input_filename, :container_output_filename,
+                                :keep_code_folders
+
+    attr_reader_with_fallback :host_code_root_path, :host_uid_pool_lock_path
+
+    attr_reader :docker_url, :docker_cert_path, :docker_auth_email, :docker_auth_user, :docker_auth_password,
+                :docker_auth_needed
+
+    def override(params={})
+      Config.send :new, self, params
+    end
+
+    def pool_max_uid
+      pool_min_uid + pool_size - 1
+    end
+
+    def docker_url=(value)
+      @docker_url = value
+      Docker.url = value
+    end
+
+    def docker_cert_path=(value)
+      @docker_cert_path = File.expand_path(value)
+      Docker.options = {
+        private_key_path: "#{@docker_cert_path}/key.pem",
+        certificate_path: "#{@docker_cert_path}/cert.pem",
+        ssl_verify_peer: false
+      }.merge(docker_options)
+    end
+
+    def host_code_root_path=(path)
+      @host_code_root_path = File.expand_path(path)
+    end
+
+    def host_uid_pool_lock_path=(path)
+      @host_uid_pool_lock_path = File.expand_path(path)
+    end
+
+    # All keys are mandatory
+    # @option :user [String]
+    # @option :password [String]
+    # @option :email [String]
+    def docker_login=(options={})
+      @docker_auth_needed = true
+      @docker_auth_user = options[:user] || options['user']
+      @docker_auth_password = options[:password] || options['password']
+      @docker_auth_email = options[:email] || options['email']
+    end
+
+    # Called to do any necessary setup to allow staged configuration
+    # @return [Config] self for chaining
+    def finished_configuring
+      return self unless @docker_auth_needed
+      Docker.authenticate! username: @docker_auth_user, password: @docker_auth_password, email: @docker_auth_email
+      @docker_auth_needed = false
+      self
+    end
+
+    private_class_method :new
+
+    # @params other_config [Config] config object that will be deferred to if the current config object does not
+    #   contain a value for the requested configuration options
+    # @params params [Hash] hash containing configuration options
+    def initialize(other_config, params={})
+      @other_config = other_config
+      params.each do |key, value|
+        send "#{key}=", value
+      end
+    end
+
+  end
+end