haconiwa 0.1.2 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 9ddb95a0442d9007bda5a95609229cf0cef771a6
-  data.tar.gz: e56fd0275206c68766efc6e7c27a406aaf5db059
+  metadata.gz: fa21fa8f08a50b66d878955312eb2537b1a8c44d
+  data.tar.gz: 378fe5fe360b75569622b78a9cf0c1c0e5300659
 SHA512:
-  metadata.gz: 96efacf5d423f067a4fcf33bc624d32db67bb803c9b3123f93c546e4acf435bef82fc48fe58395a84a8eaec3556ac19651b09385102f8631b89e1bb71bb4b541
-  data.tar.gz: aeaadfc43ff779eb60d295e05a7f10589795f1e520ac453a426d31b4cc181ba00bc5d9d439fc110e29dafcfeb1995aa4eef4c5d1459768c0a026ee80923b4602
+  metadata.gz: 575dcfbc3d35f4c8800e3b865c82fa96f785dc422f8481c6d7b1c783579a846c391514a9f0aefa9bde71c85d00fde2e36ec325899b2428b229899306e8dba708
+  data.tar.gz: df4a100b6a234b8e6fd42550f2078cbb7ca916619a5de9e9b1c74d1ce306f78b9c528f3c30ad92fc2c5941e5585e4ff306354da81e2c3231428d1688817e9826

data/README.md

@@ -6,51 +6,99 @@ Ruby on Container / helper tools with DSL for your handmade linux containers
 
 ## Installation
 
-Add this line to your application's Gemfile:
+This gem only works on Linux systems, as using cgroup, Linux namespace and Linux capabilities.
 
-```ruby
-gem 'haconiwa'
-```
+Some packages/dylibs/tools are used internally, so please install below before gem setup:
 
-And then execute:
+* `libcap` (`libcap.so.2` is used via FFI)
+* `nsenter` (e.g. `yum install util-linux` or such one)
 
-    $ bundle
+Then, install it yourself globally(recommended using rbenv) as:
 
-Or install it yourself as:
+```console
+$ gem install haconiwa
+$ rbenv rehash # if needed
+```
+
+Or add the line to your application's Gemfile:
+
+```ruby
+gem 'haconiwa'
+```
 
-    $ gem install haconiwa
+And then execute `bundle`.
 
 ## Usage
 
-```ruby
-require "haconiwa"
+Create the file `example001.haco`:
 
-haconiwa = Haconiwa::Base.define do |config|
+```ruby
+Haconiwa::Base.define do |config|
   config.name = "new-haconiwa001" # to be hostname
 
   config.cgroup["cpu.shares"] = 2048
   config.cgroup["memory.limit_in_bytes"] = "256M"
   config.cgroup["pid.max"] = 1024
 
+  config.add_mount_point "/var/another/root/etc", to: "/var/your_rootfs/etc", readonly: true
+  config.add_mount_point "/var/another/root/home", to: "/var/your_rootfs/home"
+  config.mount_independent_procfs
   config.chroot_to "/var/your_rootfs"
-  config.add_mount_point "/var/another/root/etc", to: "/etc", readonly: true
-  config.add_mount_point "/var/another/root/home", to: "/home"
-  config.add_mount_point "proc", to: "/proc", fs: "proc"
 
   config.namespace.unshare "ipc"
+  config.namespace.unshare "uts"
   config.namespace.unshare "mount"
   config.namespace.unshare "pid"
-  config.namespace.use_netns "foobar"
 
   config.capabilities.allow :all
   config.capabilities.drop "CAP_SYS_TIME"
 end
+```
+
+Then use `haconiwa` binary installed with thie gem.
+
+```console
+$ haconiwa run example001.haco
+```
+
+When you want to attach existing container:
+
+```console
+$ haconiwa attach example001.haco
+```
+
+Note: `attach` subcommand allows to set PID(`--target`) or container name(`--name`) for dynamic configuration.
+And `attach` is not concerned with capabilities which is granted to container. So you can drop or allow specific caps with `--drop/--allow`.
+
+### DSL spec
+
+* `config.cgroup` - Assign cgroup parameters via `[]=`
+* `config.namespace.unshare` - Unshare the namespaces like `"mount"`, `"ipc"` or `"pid"`
+* `config.capabilities.allow` - Allow capabilities on container root. Setting parameters other than `:all` should make this acts as whitelist
+* `config.capabilities.drop` - Drop capabilities of container root. Default to act as blacklist
+* `config.add_mount_point` - Add the mount point odf container
+* `config.mount_independent_procfs` - Mount the independent /proc directory in the container. Useful if `"pid"` is unshared
+* `config.chroot_to` - The new chroot root
 
-haconiwa.start
+You can pick your own parameters for your use case of container.
+e.g. just using `mount` namespace unshared, container with common filesystem, limit the cgroups for big resource job and so on.
 
-## or to attach running container
+Please look into `example` directory.
 
-Haconiwa.attach haconiwa.name
+### Library use case
+
+```ruby
+require 'haconiwa'
+base = Haconiwa::Base.define do |config|
+  config.name = "new-haconiwa001" # to be hostname
+  ...
+end
+
+# Run the container
+base.run("/bin/bash")
+
+# Or attach existing one
+base.attach("/bin/bash")
 ```
 
 ## Development
@@ -63,8 +111,13 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/udzura/haconiwa. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
+## TODOs
+
+* [ ] netns attachment
+* [ ] more utilities such as `ps`
+* [ ] better daemon handling
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
+This gem is created under [GNU General Public License Version 3](./LICENSE).
 

data/examples/{cpu.rb → cpu.haco}

@@ -1,5 +1,4 @@
-require 'haconiwa'
-require 'pathname'
+# -*- mode: ruby -*-
 haconiwa = Haconiwa::Base.define do |config|
   config.name = "cpu-quota001" # to be hostname
 
@@ -20,5 +19,3 @@ haconiwa = Haconiwa::Base.define do |config|
   config.cgroup["cpu.cfs_period_us"] = 100000
   config.cgroup["cpu.cfs_quota_us"]  =  30000
 end
-
-haconiwa.start("/bin/bash")

data/examples/{drop_cap_sys_time.rb → drop_cap_sys_time.haco}

@@ -1,5 +1,4 @@
-require 'haconiwa'
-require 'pathname'
+# -*- mode: ruby -*-
 haconiwa = Haconiwa::Base.define do |config|
   config.name = "drop-time001" # to be hostname
 
@@ -20,6 +19,7 @@ haconiwa = Haconiwa::Base.define do |config|
 
   config.capabilities.allow :all
   config.capabilities.drop "cap_sys_time"
-end
 
-haconiwa.start("/bin/bash")
+  # or use white list:
+  # config.capabilities.allow "cap_sys_time"
+end

data/exe/haconiwa

@@ -8,4 +8,17 @@ when "version"
   puts Haconiwa::VERSION
 when "run"
   Haconiwa::Cli.run(ARGV[1..-1])
+when "attach"
+  Haconiwa::Cli.attach(ARGV[1..-1])
+else
+  STDERR.puts <<-EOH
+Usage of #{$0}:
+    run - Run the container with init command
+        Example: haconiwa run path/to/definition.haco -- /bin/your command
+    attach - Attach to the existing container with command
+        Example: haconiwa attach path/to/definition.haco -- /bin/your command
+
+Version:
+    #{Haconiwa::VERSION}
+  EOH
 end

data/lib/haconiwa/base.rb

@@ -10,10 +10,12 @@ module Haconiwa
     attr_accessor :name,
                   :init_command,
                   :container_pid_file,
+                  :pid,
                   :filesystem,
                   :cgroup,
                   :namespace,
-                  :capabilities
+                  :capabilities,
+                  :attached_capabilities
 
     def self.define(&b)
       new.tap(&b)
@@ -24,8 +26,10 @@ module Haconiwa
       @cgroup = CGroup.new
       @namespace = Namespace.new
       @capabilities = Capabilities.new
+      @attached_capabilities = nil
       @name = "haconiwa-#{Time.now.to_i}"
-      @container_pid_file = "/var/run/haconiwa-#{@name}.pid"
+      @container_pid_file = nil
+      @pid = nil
     end
 
     # aliases
@@ -44,9 +48,19 @@ module Haconiwa
     end
 
     def start(*init_command)
+      self.container_pid_file ||= default_container_pid_file
       Runners::Linux.run(self, init_command)
     end
     alias run start
+
+    def attach(*run_command)
+      self.container_pid_file ||= default_container_pid_file
+      Runners::Linux.attach(self, run_command)
+    end
+
+    def default_container_pid_file
+      "/var/run/haconiwa-#{@name}.pid"
+    end
   end
 
   def self.define(&b)

data/lib/haconiwa/cgroup.rb

@@ -18,6 +18,7 @@ module Haconiwa
     def to_dirs
       groups.keys.map{|k| k.split('.').first }.uniq
     end
+    alias dirs to_dirs
 
     def register_all!(to: nil)
       cg = SmallCgroup.new(name: to)
@@ -25,5 +26,12 @@ module Haconiwa
         cg.register k, v
       end
     end
+
+    def attach(to: nil)
+      cg = SmallCgroup.new(name: to)
+      dirs.each do |dir|
+        cg.attach(dir)
+      end
+    end
   end
 end

data/lib/haconiwa/cli.rb

@@ -1,15 +1,48 @@
 module Haconiwa
   module Cli
     def self.run(args)
+      base, init = get_script_and_eval(args)
+      base.run(*init)
+    end
+
+    def self.attach(args)
+      require 'optparse'
+      opt = OptionParser.new
+      pid = nil
+      name = nil
+      allow = nil
+      drop = nil
+
+      opt.program_name = "haconiwa attach"
+      opt.on('-t', '--target PID', "Container's PID to attatch. If not set, use pid file of definition") {|v| pid = v }
+      opt.on('-n', '--name CONTAINER_NAME', "Container's name. Set if the name is dynamically defined") {|v| name = v }
+      opt.on('--allow CAPS[,CAPS...]', "Capabilities to allow attached process. Independent container's own caps") {|v| allow = v.split(',') }
+      opt.on('--drop CAPS[,CAPS...]', "Capabilities to drop from attached process. Independent container's own caps") {|v| drop = v.split(',') }
+      args = opt.parse(args)
+
+      base, exe = get_script_and_eval(args)
+      base.pid = pid if pid
+      base.name = name if name
+      if allow || drop
+        base.attached_capabilities = Capabilities.new
+        base.attached_capabilities.allow(*allow) if allow
+        base.attached_capabilities.drop(*drop) if drop
+      end
+
+      base.attach(*exe)
+    end
+
+    private
+
+    def self.get_script_and_eval(args)
       require 'pathname'
       script = File.read(args[0])
-      init   = args[1..-1]
-      if init.first == "--"
-        init.shift
+      exe    = args[1..-1]
+      if exe.first == "--"
+        exe.shift
       end
 
-      container = eval(script)
-      container.run(*init)
+      return [eval(script), exe]
     end
   end
 end

data/lib/haconiwa/namespace.rb

@@ -1,6 +1,7 @@
 module Haconiwa
   class Namespace
     UNSHARE = 272
+    SETNS   = 308
 
     # from linux/sched.h
 
@@ -25,6 +26,16 @@ module Haconiwa
       "uts"    => CLONE_NEWUTS,
     }
 
+    FLAG_TO_PARAM = {
+      CLONE_NEWCGROUP => "cgroup",
+      CLONE_NEWIPC    => "ipc",
+      CLONE_NEWNET    => "net",
+      CLONE_NEWNS     => "mount",
+      CLONE_NEWPID    => "pid",
+      CLONE_NEWUSER   => "user",
+      CLONE_NEWUTS    => "uts",
+    }
+
     def initialize
       @use_ns = []
       @netns_name = nil
@@ -55,8 +66,20 @@ module Haconiwa
       Kernel.syscall(UNSHARE, flag)
     end
 
+    def enter(pid: nil, wrapper_path: nil)
+      ns_params = use_ns_all.map{|f| "--#{FLAG_TO_PARAM[f]}" }
+      exec "nsenter",
+           "--target", "#{pid}",
+           *ns_params,
+           "--", wrapper_path.to_s
+    end
+
     private
 
+    def use_ns_all
+      @use_ns.uniq + (@use_pid_ns ? [CLONE_NEWPID] : [])
+    end
+
     def to_ns_flag
       @use_ns.inject(0x00000000) { |dst, flag|
         dst |= flag

data/lib/haconiwa/runners/linux.rb

@@ -1,5 +1,6 @@
 require 'tempfile'
 require 'fileutils'
+require 'shellwords'
 require 'bundler'
 
 module Haconiwa::Runners
@@ -53,7 +54,8 @@ module Haconiwa::Runners
                              container
                            end
       File.open(base.container_pid_file, "w") {|pid| pid.write real_container_pid }
-      puts "New container: PID = #{real_container_pid}"
+      Signal.trap(:INT) { Process.kill :TERM, container }
+      puts "New container: Name = #{base.name}, PID = #{real_container_pid}"
 
       res = Process.waitpid2 container
       if res[1].success?
@@ -65,6 +67,42 @@ module Haconiwa::Runners
       at_exit { FileUtils.rm_f base.container_pid_file }
     end
 
+    def self.attach(base, run_command=[])
+      if run_command.empty?
+        run_command << "/bin/bash"
+      end
+
+      wrapper = Tempfile.open("haconiwa-attacher-#{$$}-#{Time.now.to_i}.sh")
+
+      wrapper.puts "#!/bin/bash"
+      wrapper.puts "chroot #{base.filesystem.chroot} bash -c '"
+      wrapper.puts "cd / ;"
+      wrapper.puts Shellwords.shelljoin(["exec", *run_command]).gsub(/'/, %<'"'"'>)
+      wrapper.puts "'"
+      wrapper.close
+      FileUtils.chmod 0700, wrapper.path
+
+      runner = fork {
+        base.pid ||= File.read(base.container_pid_file).to_i
+
+        if base.attached_capabilities
+          base.attached_capabilities.apply!
+        end
+
+        base.cgroup.attach(to: base.name)
+        Bundler.with_clean_env { base.namespace.enter(pid: base.pid, wrapper_path: wrapper.path) }
+      }
+
+      puts "Attached to contanier: Runner PID = #{runner}"
+
+      res = Process.waitpid2 runner
+      if res[1].success?
+        puts "Successfully exit."
+      else
+        puts "Attached process exited with status code <#{res[1].to_i}>."
+      end
+    end
+
     private
     def self.find_by_ppid(ppid)
       s = Dir.glob('/proc/*/stat')

data/lib/haconiwa/small_cgroup.rb

@@ -31,10 +31,15 @@ module Haconiwa
     def activate(dir)
       dirroot = root_of(dir)
       FileUtils.mkdir_p dirroot
-      append_write dirroot.join("tasks"), self.pid
+      attach(dir)
       @active_dirs << dir
     end
 
+    def attach(dir)
+      dirroot = root_of(dir)
+      append_write dirroot.join("tasks"), self.pid
+    end
+
     def activated?(dir)
       @active_dirs.include? dir
     end

data/lib/haconiwa/version.rb

@@ -1,3 +1,3 @@
 module Haconiwa
-  VERSION = "0.1.2"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: haconiwa
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.2.0
 platform: ruby
 authors:
 - Uchio KONDO
@@ -121,16 +121,14 @@ files:
 - CODE_OF_CONDUCT.md
 - Gemfile
 - LICENSE
-- LICENSE.txt
 - README.md
 - Rakefile
 - Vagrantfile
 - bin/console
 - bin/setup
 - examples/chroot.haco
-- examples/chroot.rb
-- examples/cpu.rb
-- examples/drop_cap_sys_time.rb
+- examples/cpu.haco
+- examples/drop_cap_sys_time.haco
 - exe/haconiwa
 - haconiwa.gemspec
 - lib/haconiwa.rb

data/LICENSE.txt

@@ -1,21 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2016 Uchio KONDO
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.

data/examples/chroot.rb

@@ -1,21 +0,0 @@
-require 'haconiwa'
-require 'pathname'
-haconiwa = Haconiwa::Base.define do |config|
-  config.name = "chroot001" # to be hostname
-
-  root = Pathname.new("/var/haconiwa/root")
-  config.add_mount_point "/var/haconiwa/rootfs", to: root, readonly: true
-  config.add_mount_point "/lib64", to: root.join("lib64"), readonly: true
-  config.add_mount_point "/usr/bin", to: root.join("usr/bin"), readonly: true
-  config.add_mount_point "tmpfs", to: root.join("tmp"), fs: "tmpfs"
-  config.add_mount_point "/var/haconiwa/user_homes/hakoniwa-test001/home/hakoniwa", to: root.join("home/hakoniwa")
-  config.mount_independent_procfs
-  config.chroot_to root
-
-  config.namespace.unshare "mount"
-  config.namespace.unshare "ipc"
-  config.namespace.unshare "uts"
-  config.namespace.unshare "pid"
-end
-
-haconiwa.start("/bin/bash")