freighter 0.0.1 → 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +2 -0
- data/.rspec +2 -0
- data/Gemfile +8 -0
- data/README.md +78 -11
- data/Rakefile +10 -0
- data/bin/freighter +56 -17
- data/config/freighter.example.yml +56 -0
- data/lib/freighter/deploy.rb +166 -0
- data/lib/freighter/docker_rest_api.rb +68 -0
- data/lib/freighter/helpers.rb +10 -0
- data/lib/freighter/logger.rb +18 -8
- data/lib/freighter/parser.rb +79 -0
- data/lib/freighter/ssh.rb +47 -0
- data/lib/freighter/version.rb +1 -1
- data/lib/freighter.rb +12 -4
- data/spec/lib/freighter/deploy_spec.rb +39 -0
- data/spec/lib/freighter/parser_spec.rb +62 -0
- data/spec/spec_helper.rb +95 -0
- metadata +16 -4
- data/lib/freighter/parse.rb +0 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: af0c274acce168e3b23f92161069ddc4271fe463
|
4
|
+
data.tar.gz: 20ef8715cd51bb1dae81d921e8029d2653f045e1
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: b759d8a9dcfe91c6c9d03eaacb1662e084eb556fc26b0ea096c96bd7f8c06cfd97e7dfb6d18d9f19425b9364f51bdc8888ac63b8cdfca6fe9d46f29860a9d549
|
7
|
+
data.tar.gz: 6dee9418ec37852e3c8a925cb78dbc68d11146d635137f9a0f7f5f1482224d9dc54e9fb7188b0ae795823fe8e7877ceac49a4836ea4e3e0b729ae4aef569e8f7
|
data/.gitignore
CHANGED
data/.rspec
ADDED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,28 +1,95 @@
|
|
1
|
-
# This gem is not ready yet. Just a gem stub at this point.
|
2
|
-
|
3
1
|
# Freighter
|
4
2
|
|
5
|
-
|
3
|
+
Freighter's goal is to make it easy to deploy docker containers over ssh. Freighter uses one YAML file to describe the environments, servers, images, and containers in your environment.
|
4
|
+
|
5
|
+
Freighter goals:
|
6
|
+
* Simple docker container deployment
|
7
|
+
* Straight forward configuration
|
8
|
+
* Users new to freighter should be able to deploy in minutes
|
9
|
+
* Minimal server-side configuration
|
10
|
+
* Fast and reliable
|
6
11
|
|
7
12
|
## Installation
|
13
|
+
Freighter is a ruby gem and requires ruby 1.9 or higher.
|
8
14
|
|
9
|
-
|
15
|
+
gem install freighter
|
10
16
|
|
11
|
-
|
12
|
-
|
17
|
+
## Configuration
|
18
|
+
|
19
|
+
After freighter is installed, run the configuration installer.
|
20
|
+
```
|
21
|
+
freighter configure
|
13
22
|
```
|
23
|
+
This copies an example template of a YAML configuration file into ./config/freighter.yml
|
14
24
|
|
15
|
-
|
25
|
+
### Docker REST API
|
26
|
+
|
27
|
+
The way that freighter does not require that users have sudo access on the hosts it deploys to is that it interacts with the docker rest api running on the hosts. This means that docker must be configured to expose its REST API on each host.
|
28
|
+
|
29
|
+
```
|
30
|
+
echo 'DOCKER_OPTS="-H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock"' | sudo cat >> /etc/default/docker
|
31
|
+
```
|
16
32
|
|
17
|
-
|
33
|
+
The docker service, on the host(s), will need to be restarted.
|
18
34
|
|
19
|
-
|
35
|
+
Running the docker REST API this way should be secure since all communication to the API is over an SSH tunnel, and the REST API is only available locally on the host.
|
20
36
|
|
21
|
-
|
37
|
+
### Authentication
|
38
|
+
|
39
|
+
Currently, this gem supports pulling images from hub.docker.com. This means that you must authenticate.
|
40
|
+
It is not recommended to store your personal authentication credentials in freighter.yml since that file
|
41
|
+
should be added to source control. Freighter needs the following environment variables set on your local machine:
|
42
|
+
|
43
|
+
* DOCKER_HUB_USER_NAME
|
44
|
+
* DOCKER_HUB_PASSWORD
|
45
|
+
* DOCKER_HUB_EMAIL
|
46
|
+
|
47
|
+
A recommendation would be to create a file only accessible to your machine's user account that defines these environment variables.
|
48
|
+
|
49
|
+
```shell
|
50
|
+
export DOCKER_HUB_USER_NAME=<yourDockerHubUserName>
|
51
|
+
export DOCKER_HUB_PASSWORD=<yourDockerHubPassword>
|
52
|
+
export DOCKER_HUB_EMAIL=<yourDockerHubEmail>
|
53
|
+
```
|
22
54
|
|
23
55
|
## Usage
|
24
56
|
|
25
|
-
|
57
|
+
For quick reference:
|
58
|
+
```
|
59
|
+
freighter --help
|
60
|
+
```
|
61
|
+
|
62
|
+
Example of how to deploy:
|
63
|
+
```
|
64
|
+
freighter -e staging --all deloy
|
65
|
+
```
|
66
|
+
|
67
|
+
If you want to deploy one app:
|
68
|
+
```
|
69
|
+
freighter -e staging --app my_app deploy
|
70
|
+
```
|
71
|
+
|
72
|
+
The apps are defined in freighter.yml.
|
73
|
+
|
74
|
+
## freighter.yml
|
75
|
+
|
76
|
+
When you run `freighter configure` it will copy an example freighter.yml file that you can use as a template. The structure of the YAML file is important. More documentation on configuration to come.
|
77
|
+
|
78
|
+
## Fun facts
|
79
|
+
|
80
|
+
If you find yourself in a pickle of not being able to Ctrl+c (interupt) the command. Ctrl+z (suspend) the process and the kill the pid with `kill -6 <pid>`. I'll try to fix it so that this scenario is more avoidable.
|
81
|
+
|
82
|
+
# Status
|
83
|
+
|
84
|
+
Needed:
|
85
|
+
* Needs more testing with more complex scenarios
|
86
|
+
|
87
|
+
Nice to haves:
|
88
|
+
* Container linking options
|
89
|
+
* Volume mounting options
|
90
|
+
* Container cleanup
|
91
|
+
|
92
|
+
Freighter is currently deploying quickly and reliably as far as I can tell.
|
26
93
|
|
27
94
|
## Contributing
|
28
95
|
|
data/Rakefile
CHANGED
data/bin/freighter
CHANGED
@@ -2,45 +2,84 @@
|
|
2
2
|
|
3
3
|
require 'optparse'
|
4
4
|
require 'bundler/setup'
|
5
|
-
require 'ostruct'
|
6
5
|
require 'freighter'
|
7
6
|
require 'pry'
|
8
7
|
|
9
|
-
|
8
|
+
options = Freighter.options
|
10
9
|
|
11
10
|
if defined?(Rails)
|
12
|
-
|
11
|
+
options.config_path = "#{Rails.root}/config/freighter.yml"
|
13
12
|
else
|
14
|
-
|
13
|
+
options.config_path = "./config/freighter.yml"
|
15
14
|
end
|
16
15
|
|
17
16
|
OptionParser.new do |opts|
|
18
|
-
opts.banner = "Usage: freigter [options] (deploy)"
|
17
|
+
opts.banner = "Usage: freigter [options] (deploy|configure|verify-config)"
|
19
18
|
|
20
|
-
opts.on('-
|
21
|
-
|
19
|
+
opts.on('-e ENV', 'environment * required') do |env|
|
20
|
+
options.environment = env
|
22
21
|
end
|
23
22
|
|
24
|
-
opts.on('-
|
23
|
+
opts.on('-v', '--verbose', 'verbose logging') do
|
24
|
+
options.verbose = true
|
25
|
+
end
|
26
|
+
|
27
|
+
opts.on('-c', '--config PATH', 'path to yml config. Default is ./config/freighter.yml') do |opt|
|
25
28
|
if opt
|
26
|
-
|
29
|
+
options.config_path = opt
|
27
30
|
elsif defiend?(Rails) && Rails.root
|
28
|
-
|
31
|
+
options.config_path = "#{Rails.root}/config/freighter.yml"
|
29
32
|
end
|
30
33
|
end
|
31
|
-
end.parse!
|
32
34
|
|
33
|
-
|
35
|
+
opts.on('-a', '--app APP_NAME', 'The name of the app to deploy') do |app_name|
|
36
|
+
options.app_name = app_name
|
37
|
+
end
|
38
|
+
|
39
|
+
opts.on('--all', 'Deploy all apps for the environment') do
|
40
|
+
raise "Can not use options --app and --all as they conflict" if options.app_name
|
41
|
+
options.deploy_all = true
|
42
|
+
end
|
43
|
+
|
44
|
+
# This option is not yet implemented because the docker REST api is not the best at searching available images
|
45
|
+
# opts.on('--no-pull', 'specify if you do not wish to pull the image') do
|
46
|
+
# options.pull_image = false
|
47
|
+
# end
|
48
|
+
end.parse!
|
34
49
|
|
35
|
-
logger
|
36
|
-
logger.log "app is chatty", :verbose
|
50
|
+
logger = Freighter.logger
|
37
51
|
|
38
|
-
|
52
|
+
# todo - brush up on nautical jargon to find a better way of saying the freighter is to depart
|
53
|
+
logger.info "All hand on deck. Toot toot."
|
39
54
|
|
40
55
|
case ARGV[0]
|
41
56
|
when "deploy"
|
42
|
-
|
57
|
+
unless options.environment
|
58
|
+
logger.error "Must specify an environment with the -e option"
|
59
|
+
end
|
60
|
+
# check to see if the container name is specified or all containers should be deployed
|
61
|
+
unless (options.app_name || options.deploy_all)
|
62
|
+
logger.error "Must specify either container(s) with --app or --all option"
|
63
|
+
end
|
64
|
+
Freighter::Deploy.new
|
65
|
+
when "configure"
|
66
|
+
if File.exist?('./config/freighter.yml')
|
67
|
+
logger.info "./config/freighter.yml already exists"
|
68
|
+
else
|
69
|
+
example_path = File.expand_path('../../config/freighter.example.yml', __FILE__)
|
70
|
+
begin
|
71
|
+
FileUtils.mkdir('config')
|
72
|
+
rescue Errno::EEXIST
|
73
|
+
end
|
74
|
+
FileUtils.cp(example_path, 'config/freighter.yml')
|
75
|
+
logger.info "Example configuration copied to ./config/freighter.yml"
|
76
|
+
logger.info "Edit the file with your configuration."
|
77
|
+
logger.info "Please remember to prefer to use ENV vars instead of passwords when editing the configuration."
|
78
|
+
end
|
79
|
+
when "verify-config"
|
80
|
+
Freighter::Parser.new(options.config_path)
|
81
|
+
logger.info "Your configuration checks out"
|
43
82
|
else
|
44
|
-
logger.
|
83
|
+
logger.error "unexpected command given: #{ARGV[0]}. See usage. freighter --help"
|
45
84
|
end
|
46
85
|
|
@@ -0,0 +1,56 @@
|
|
1
|
+
# Please use the examples in this file as a starting place for your own configuration.
|
2
|
+
# This file should contain all the information needed to deploy with freighter,
|
3
|
+
# aside from environment variables: DOCKER_HUB_USER_NAME, DOCKER_HUB_PASSWORD, DOCKER_HUB_EMAIL
|
4
|
+
|
5
|
+
connection:
|
6
|
+
type: ssh
|
7
|
+
ssh_options:
|
8
|
+
user_name: user name on host
|
9
|
+
keys:
|
10
|
+
- "~/.config/id_rsa"
|
11
|
+
|
12
|
+
docker:
|
13
|
+
port: 2375
|
14
|
+
|
15
|
+
environments:
|
16
|
+
staging:
|
17
|
+
hosts:
|
18
|
+
- host: staging.example.com
|
19
|
+
images:
|
20
|
+
- name: organization/imageName:latest
|
21
|
+
containers:
|
22
|
+
- name: app
|
23
|
+
port_mapping: 0.0.0.0:80->80
|
24
|
+
env:
|
25
|
+
DB_USERNAME: fooBar
|
26
|
+
DB_PASSWORD: My53cr37
|
27
|
+
|
28
|
+
# a more complex example for production showing a way to reduce redundancy
|
29
|
+
production:
|
30
|
+
shared_env: &shared_app_env
|
31
|
+
DB_USERNAME: fooBar
|
32
|
+
DB_PASSWORD: My53cr37
|
33
|
+
hosts:
|
34
|
+
- host: prod1.example.com
|
35
|
+
images:
|
36
|
+
- name: organization/imageName:latest
|
37
|
+
containers:
|
38
|
+
- name: app
|
39
|
+
port_mapping: 0.0.0.0:80->80
|
40
|
+
env:
|
41
|
+
<<: *shared_app_env
|
42
|
+
ADDITIONAL_VAR: someValue
|
43
|
+
- name: organization/otherImage:latest
|
44
|
+
containers:
|
45
|
+
- name: otherApp
|
46
|
+
port_mapping: 0.0.0.0:2000->80
|
47
|
+
env:
|
48
|
+
<<: *shared_app_env
|
49
|
+
- host: prod2.example.com
|
50
|
+
images:
|
51
|
+
- name: organization/imageName:latest
|
52
|
+
containers:
|
53
|
+
- name: otherApp
|
54
|
+
port_mapping: 0.0.0.0:2000->80
|
55
|
+
env:
|
56
|
+
<<: *shared_app_env
|
@@ -0,0 +1,166 @@
|
|
1
|
+
require 'docker'
|
2
|
+
|
3
|
+
module Freighter
|
4
|
+
class Deploy
|
5
|
+
attr_reader :logger, :config
|
6
|
+
|
7
|
+
def initialize
|
8
|
+
@parser = Parser.new OPTIONS.config_path
|
9
|
+
@logger = LOGGER
|
10
|
+
@config = OPTIONS.config
|
11
|
+
@connection_config = @config.fetch('connection')
|
12
|
+
environments = @config.fetch('environments')
|
13
|
+
@environment = environments.fetch(OPTIONS.environment) rescue logger.config_error("environments/#{OPTIONS.environment}")
|
14
|
+
|
15
|
+
connection_type = @connection_config['type']
|
16
|
+
case connection_type
|
17
|
+
when 'ssh'
|
18
|
+
deploy_with_ssh
|
19
|
+
else
|
20
|
+
logger.error "Unknown configuration option for type: #{connection_type}"
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
def deploy_with_ssh
|
25
|
+
ssh_options = @connection_config.fetch('ssh_options')
|
26
|
+
ssh_options.extend Helpers::Hash
|
27
|
+
ssh_options = ssh_options.symbolize_keys
|
28
|
+
|
29
|
+
@environment.fetch('hosts').each_with_index do |host, i|
|
30
|
+
host_name = host.fetch('host')
|
31
|
+
images = @parser.images(host_name)
|
32
|
+
|
33
|
+
ssh = SSH.new(host_name, ssh_options)
|
34
|
+
local_port = 7000 + i
|
35
|
+
# docker_api = DockerRestAPI.new("http://localhost:#{local_port}")
|
36
|
+
|
37
|
+
ssh.tunneled_proxy(local_port) do |session|
|
38
|
+
msg = ->(m) { "#{host_name}: #{m}" }
|
39
|
+
|
40
|
+
logger.debug msg["Connected"]
|
41
|
+
begin
|
42
|
+
# The timeout is needed in the case that we are unable to communicate with the docker REST API
|
43
|
+
Timeout::timeout(5) do
|
44
|
+
setup_docker_client(local_port)
|
45
|
+
end
|
46
|
+
rescue Timeout::Error
|
47
|
+
ssh.thread.exit
|
48
|
+
logger.error msg["Could not reach the docker REST API"]
|
49
|
+
end
|
50
|
+
|
51
|
+
|
52
|
+
images.each do |image|
|
53
|
+
image_name = image['name']
|
54
|
+
# pull image
|
55
|
+
if OPTIONS.pull_image
|
56
|
+
logger.info msg["Pulling image: #{image_name}"]
|
57
|
+
pull_response = Docker::Image.create 'fromImage' => image_name
|
58
|
+
else
|
59
|
+
logger.info msg["Skip pull image"]
|
60
|
+
logger.error msg["Skipping is not yet implemented. Please run again without the --no-pull option"]
|
61
|
+
end
|
62
|
+
|
63
|
+
# find existing images on the machine
|
64
|
+
image_ids = Docker::Image.all.select do |img|
|
65
|
+
img.info['RepoTags'].member?(image_name)
|
66
|
+
end.map { |img| img.id[0...12] }
|
67
|
+
|
68
|
+
logger.info msg["Existing image(s) found #{image_ids.join(', ')}"]
|
69
|
+
|
70
|
+
# determine if a the latest version of the image is currently running
|
71
|
+
matching_containers = containers_matching_port_map(Docker::Container.all, image['containers'].map { |c| c['port_mapping'] })
|
72
|
+
if image_ids.member?(pull_response.id) && !matching_containers.empty?
|
73
|
+
logger.info msg["Container already running with the latest image: #{pull_response.id}"]
|
74
|
+
else
|
75
|
+
# stop previous container and start up a new container with the latest image
|
76
|
+
results = update_containers matching_containers, image
|
77
|
+
logger.info msg["Finished:"]
|
78
|
+
logger.info msg[" started: #{results[:started]}"]
|
79
|
+
logger.info msg[" stopped: #{results[:stopped]}"]
|
80
|
+
logger.info msg[" started container ids: #{results[:container_ids_started]}"]
|
81
|
+
end
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
85
|
+
end
|
86
|
+
|
87
|
+
private
|
88
|
+
|
89
|
+
# Sets up the Docker gem by setting the local URL and authenticating to the host's REST API
|
90
|
+
def setup_docker_client(local_port)
|
91
|
+
Docker.url = "http://localhost:#{local_port}"
|
92
|
+
begin
|
93
|
+
logger.debug "Requesting docker version"
|
94
|
+
response = Docker.version
|
95
|
+
logger.debug "Docker version: #{response.inspect}"
|
96
|
+
logger.debug "Requesting docker authenticaiton"
|
97
|
+
response = Docker.authenticate!('username' => ENV['DOCKER_HUB_USER_NAME'], 'password' => ENV['DOCKER_HUB_PASSWORD'], 'email' => ENV['DOCKER_HUB_EMAIL'])
|
98
|
+
logger.debug "Docker authentication: #{response.inspect}"
|
99
|
+
rescue Excon::Errors::SocketError => e
|
100
|
+
logger.error e.message
|
101
|
+
end
|
102
|
+
end
|
103
|
+
|
104
|
+
def containers_matching_port_map(containers, port_mappings)
|
105
|
+
port_mappings.map do |port_map|
|
106
|
+
ports = ports(port_map)
|
107
|
+
containers.select do |c|
|
108
|
+
c.info['Ports'].detect do |p|
|
109
|
+
p['PrivatePort'] == ports.container && p['PublicPort'] == ports.host
|
110
|
+
end
|
111
|
+
end
|
112
|
+
end.flatten
|
113
|
+
end
|
114
|
+
|
115
|
+
PortMap = Struct.new(:ip, :host, :container)
|
116
|
+
def ports(port_map)
|
117
|
+
port_map.match(/^(\d{1,3}\.[\.0-9]*)?:?(\d+)->(\d+)$/)
|
118
|
+
begin
|
119
|
+
raise if $2.nil? or $3.nil?
|
120
|
+
PortMap.new($1, $2.to_i, $3.to_i)
|
121
|
+
rescue
|
122
|
+
raise "port_mappings needs to be in the format of <ip-address>:<host-port-number>-><container-port-number>. received: #{port_map}"
|
123
|
+
end
|
124
|
+
end
|
125
|
+
|
126
|
+
def update_containers existing_containers=[], image
|
127
|
+
totals = { stopped: 0, started: 0, container_ids_started: [] }
|
128
|
+
# stop the existing matching containers
|
129
|
+
existing_containers.map do |container|
|
130
|
+
Thread.new do
|
131
|
+
existing_container = Docker::Container.get(container.id)
|
132
|
+
logger.info "Stopping container: #{contianer.id}"
|
133
|
+
existing_container.stop
|
134
|
+
existing_container.wait()
|
135
|
+
logger.info "Container stopped (#{container.id}"
|
136
|
+
totals[:stopped] += 1
|
137
|
+
end
|
138
|
+
end.join
|
139
|
+
|
140
|
+
# start up some new containers
|
141
|
+
image['containers'].map do |container|
|
142
|
+
port_map = ports(container['port_mapping'])
|
143
|
+
|
144
|
+
# env = container['env'].inject("") { |r, (k,v)| r << "#{k}='#{v}',\n" }
|
145
|
+
env = container['env'].map { |k,v| "#{k}=#{v}" }
|
146
|
+
container_options = {
|
147
|
+
"Image" => image['name'],
|
148
|
+
"ExposedPorts" => { "#{port_map.container}/tcp" => {} },
|
149
|
+
"Env" => env
|
150
|
+
}
|
151
|
+
|
152
|
+
new_container = Docker::Container.create container_options
|
153
|
+
logger.info "Starting container with port_mapping: host #{[port_map.ip, port_map.host].join(':')}, container #{port_map.container}"
|
154
|
+
new_container.start(
|
155
|
+
"PortBindings" => { "#{port_map.container}/tcp" => [{ "HostPort" => port_map.host.to_s, "HostIp" => port_map.ip }] }
|
156
|
+
)
|
157
|
+
totals[:container_ids_started] << new_container.id
|
158
|
+
logger.info "New container started with id: #{new_container.id}"
|
159
|
+
totals[:started] += 1
|
160
|
+
end
|
161
|
+
|
162
|
+
totals
|
163
|
+
end
|
164
|
+
|
165
|
+
end
|
166
|
+
end
|
@@ -0,0 +1,68 @@
|
|
1
|
+
require 'excon'
|
2
|
+
require 'json'
|
3
|
+
|
4
|
+
module Freighter
|
5
|
+
class DockerRestAPI
|
6
|
+
|
7
|
+
def initialize(url)
|
8
|
+
@base_url = url
|
9
|
+
@http = Excon.new url
|
10
|
+
end
|
11
|
+
|
12
|
+
# authentication should not be necessary if the user is already authenticated with the docker client on the host
|
13
|
+
def authenticate
|
14
|
+
request { post(path: '/auth', body: JSON.dump({ 'username' => ENV['DOCKER_HUB_USER_NAME'], 'password' => ENV['DOCKER_HUB_PASSWORD'], 'email' => ENV['DOCKER_HUB_EMAIL'] }), headers: { "Content-Type" => "application/json" }) }
|
15
|
+
end
|
16
|
+
|
17
|
+
# This pulls a specified image
|
18
|
+
# def pull(image, repo, tag='latest')
|
19
|
+
def pull(tag)
|
20
|
+
request do
|
21
|
+
post(path: "/images/create", query: { tag: tag })
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
# returns all running containers
|
26
|
+
def running_containers
|
27
|
+
request do
|
28
|
+
get(path: '/containers/json')
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
def list_images
|
33
|
+
request { get path: '/images/json' }
|
34
|
+
end
|
35
|
+
|
36
|
+
protected
|
37
|
+
|
38
|
+
ResponseObject = Struct.new(:body_hash, :status)
|
39
|
+
|
40
|
+
def request(&block)
|
41
|
+
begin
|
42
|
+
binding.pry
|
43
|
+
response = yield
|
44
|
+
rescue Excon::Errors::SocketError => e
|
45
|
+
logger.error e.message
|
46
|
+
end
|
47
|
+
|
48
|
+
status = response.status
|
49
|
+
if status >= 200 and status < 300
|
50
|
+
begin
|
51
|
+
ResponseObject.new JSON.parse(response.body), status
|
52
|
+
rescue JSON::ParserError => e
|
53
|
+
binding.pry
|
54
|
+
end
|
55
|
+
else
|
56
|
+
LOGGER.error "Could not process request:\n request: #{@last_request_args.inspect}\n response: #{response.inspect}"
|
57
|
+
end
|
58
|
+
end
|
59
|
+
|
60
|
+
%w[get post put delete].each do |verb|
|
61
|
+
define_method verb.to_sym do |*args|
|
62
|
+
@last_request_args = args
|
63
|
+
@http.send(verb.to_sym, *args)
|
64
|
+
end
|
65
|
+
end
|
66
|
+
|
67
|
+
end
|
68
|
+
end
|
data/lib/freighter/logger.rb
CHANGED
@@ -1,19 +1,29 @@
|
|
1
|
+
require 'logger'
|
2
|
+
require 'pry'
|
3
|
+
|
1
4
|
module Freighter
|
2
5
|
class Logger
|
3
|
-
|
4
|
-
|
6
|
+
attr_reader :logger
|
7
|
+
|
8
|
+
def initialize
|
9
|
+
@logger = ::Logger.new(STDOUT)
|
10
|
+
logger.formatter = ->(severity, time, progname, msg) { "#{severity}: #{msg}\n" }
|
11
|
+
end
|
12
|
+
|
13
|
+
def method_missing(meth, *args, &block)
|
14
|
+
logger.level = OPTIONS.verbose ? ::Logger::DEBUG : ::Logger::INFO
|
15
|
+
logger.send(meth, *args)
|
5
16
|
end
|
6
17
|
|
7
|
-
def
|
8
|
-
|
9
|
-
puts str
|
10
|
-
end
|
18
|
+
def config_error(str)
|
19
|
+
error "Config error: #{str} not defined"
|
11
20
|
end
|
12
21
|
|
13
22
|
def error(str)
|
14
|
-
|
15
|
-
|
23
|
+
logger.error str
|
24
|
+
logger.error "Freighter hit an iceburg. To the life boats. All is lost. A truely unfortunate day in nautical affairs :("
|
16
25
|
exit -1
|
17
26
|
end
|
27
|
+
|
18
28
|
end
|
19
29
|
end
|
@@ -0,0 +1,79 @@
|
|
1
|
+
require 'yaml'
|
2
|
+
|
3
|
+
module Freighter
|
4
|
+
class Parser
|
5
|
+
attr_reader :config
|
6
|
+
|
7
|
+
def initialize(config_path)
|
8
|
+
begin
|
9
|
+
@config = opts.config = YAML.load_file(config_path)
|
10
|
+
LOGGER.debug "config file parsed"
|
11
|
+
rescue Errno::ENOENT, Psych::SyntaxError => e
|
12
|
+
LOGGER.error "Error parsing freighter config file.\n path: #{config_path}\n #{e}"
|
13
|
+
rescue
|
14
|
+
LOGGER.error "There is something wrong with the path to your yaml config file: #{config_path}\n #{$!.message}"
|
15
|
+
end
|
16
|
+
|
17
|
+
# Do some basic checking to make sure the config file has what we need
|
18
|
+
%w[environments connection/type].each { |option| test_config_option option }
|
19
|
+
set_defaults
|
20
|
+
end
|
21
|
+
|
22
|
+
def opts
|
23
|
+
OPTIONS
|
24
|
+
end
|
25
|
+
|
26
|
+
# recursively tests for keys in a nested hash by separating nested keys with '/'
|
27
|
+
def test_config_option(option, opt_array=[], context=nil)
|
28
|
+
opts_2_test = option.split('/')
|
29
|
+
opts_2_test.each_with_index do |opt, i|
|
30
|
+
opt_array << opt
|
31
|
+
context ||= opts.config
|
32
|
+
begin
|
33
|
+
if next_opt = opts_2_test[i+1]
|
34
|
+
new_context = context.fetch(opt)
|
35
|
+
test_config_option(next_opt, opt_array.clone, new_context.clone)
|
36
|
+
end
|
37
|
+
rescue KeyError
|
38
|
+
LOGGER.config_error opt_array.join('/')
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
def images(host)
|
44
|
+
host_config = environment.fetch('hosts').detect { |h| h.fetch('host') == host }
|
45
|
+
host_images = host_config.fetch('images')
|
46
|
+
raise "app(s) to deploy not specified" unless opts.deploy_all or opts.app_name
|
47
|
+
if opts.deploy_all
|
48
|
+
host_images
|
49
|
+
else
|
50
|
+
host_images.select do |host_image|
|
51
|
+
!host_image.fetch('containers').detect do |container|
|
52
|
+
container['name'] == opts.app_name
|
53
|
+
end.nil?
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
def environment
|
59
|
+
begin
|
60
|
+
config.fetch('environments').fetch(opts.environment)
|
61
|
+
rescue KeyError => e
|
62
|
+
LOGGER.error "Error fetching environment: #{e.message}"
|
63
|
+
end
|
64
|
+
end
|
65
|
+
|
66
|
+
private
|
67
|
+
|
68
|
+
def set_defaults
|
69
|
+
opts.config.tap do |conf|
|
70
|
+
conf['connection']['docker'] ||= {}
|
71
|
+
conf['connection']['docker']['socket'] ||= 'unix:///var/run/docker.sock'
|
72
|
+
conf['connection']['docker']['port'] ||= nil
|
73
|
+
end
|
74
|
+
opts.deploy_all ||= false
|
75
|
+
opts.pull_image = true if opts.pull_image.nil?
|
76
|
+
end
|
77
|
+
|
78
|
+
end
|
79
|
+
end
|
@@ -0,0 +1,47 @@
|
|
1
|
+
require 'net/ssh'
|
2
|
+
require 'net/ssh/proxy/socks5'
|
3
|
+
|
4
|
+
module Freighter
|
5
|
+
class SSH
|
6
|
+
attr_reader :thread
|
7
|
+
|
8
|
+
def initialize(host, ssh_conf)
|
9
|
+
@host = host
|
10
|
+
@user = ssh_conf.fetch(:user_name)
|
11
|
+
ssh_conf.delete(:user_name)
|
12
|
+
@ssh_options = ssh_conf
|
13
|
+
end
|
14
|
+
|
15
|
+
def proxy
|
16
|
+
docker_port = OPTIONS.config['connection']['docker']['port']
|
17
|
+
Net::SSH::Proxy::SOCKS5.new(@host, docker_port, {user: @user}.merge(@ssh_options))
|
18
|
+
end
|
19
|
+
|
20
|
+
def tunneled_proxy(local_port, use_proxy=false, &block)
|
21
|
+
options = use_proxy ? { proxy: proxy } : @ssh_options
|
22
|
+
docker_port = OPTIONS.config['docker']['port']
|
23
|
+
|
24
|
+
LOGGER.debug "Connecting\n host: #{@host}, user: #{@user}, options: #{options.inspect}"
|
25
|
+
@thread = Thread.new do
|
26
|
+
Thread.current.thread_variable_set(:ssh_tunnel_established, false)
|
27
|
+
|
28
|
+
Net::SSH.start(@host, @user, options) do |session|
|
29
|
+
session.forward.local(local_port, "0.0.0.0", docker_port)
|
30
|
+
LOGGER.debug "Connected to #{@host} and port #{local_port} is forwarded to host's docker REST API port ##{docker_port}."
|
31
|
+
|
32
|
+
Thread.current.thread_variable_set(:ssh_tunnel_established, true)
|
33
|
+
int_pressed = false
|
34
|
+
trap("INT") { int_pressed = true }
|
35
|
+
session.loop(0.1) { !session.closed? && !int_pressed }
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
while @thread.thread_variable_get(:ssh_tunnel_established) != true
|
40
|
+
sleep 0.1
|
41
|
+
end
|
42
|
+
|
43
|
+
yield
|
44
|
+
end
|
45
|
+
|
46
|
+
end
|
47
|
+
end
|
data/lib/freighter/version.rb
CHANGED
data/lib/freighter.rb
CHANGED
@@ -1,7 +1,15 @@
|
|
1
|
-
|
2
|
-
require
|
3
|
-
|
1
|
+
Dir[File.join(File.dirname(__FILE__), 'freighter', '*')].each do |file|
|
2
|
+
require file
|
3
|
+
end
|
4
|
+
require 'ostruct'
|
4
5
|
|
5
6
|
module Freighter
|
6
|
-
|
7
|
+
|
8
|
+
def self.options; OPTIONS end
|
9
|
+
def self.logger; LOGGER end
|
10
|
+
|
11
|
+
OPTIONS = OpenStruct.new
|
12
|
+
LOGGER = Logger.new
|
13
|
+
|
7
14
|
end
|
15
|
+
|
@@ -0,0 +1,39 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Freighter::Deploy do
|
4
|
+
|
5
|
+
before do
|
6
|
+
options = Freighter.options
|
7
|
+
options.config_path = "#{BASE_DIR}/config/freighter.example.yml"
|
8
|
+
options.environment = "staging"
|
9
|
+
expect_any_instance_of(Freighter::Deploy).to receive(:deploy_with_ssh)
|
10
|
+
end
|
11
|
+
|
12
|
+
context "private methods" do
|
13
|
+
subject { Freighter::Deploy.new }
|
14
|
+
|
15
|
+
describe "ports" do
|
16
|
+
|
17
|
+
it "should match port mapping with IP address" do
|
18
|
+
mapping = "0.0.0.0:80->90"
|
19
|
+
port_map = subject.send(:ports, mapping)
|
20
|
+
expect(port_map.ip).to eq "0.0.0.0"
|
21
|
+
expect(port_map.host).to eq 80
|
22
|
+
expect(port_map.container).to eq 90
|
23
|
+
end
|
24
|
+
|
25
|
+
it "should be able to accept a mapping without an IP address" do
|
26
|
+
mapping = "80->90"
|
27
|
+
port_map = subject.send(:ports, mapping)
|
28
|
+
expect(port_map.ip).to be_nil
|
29
|
+
expect(port_map.host).to eq 80
|
30
|
+
expect(port_map.container).to eq 90
|
31
|
+
end
|
32
|
+
|
33
|
+
it "should raise an exception if port mapping format is incorrect" do
|
34
|
+
mapping = ""
|
35
|
+
expect { subject.send(:ports, mapping) }.to raise_error
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
@@ -0,0 +1,62 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'pry'
|
3
|
+
|
4
|
+
describe Freighter::Parser do
|
5
|
+
let(:config_path) { File.expand_path('../../../../config/freighter.example.yml', __FILE__) }
|
6
|
+
|
7
|
+
before do
|
8
|
+
# allow(YAML).to receive(:load_file).and_return(:sample_config)
|
9
|
+
Freighter.options.environment = 'staging'
|
10
|
+
end
|
11
|
+
|
12
|
+
subject { Freighter::Parser.new(config_path) }
|
13
|
+
|
14
|
+
describe "images" do
|
15
|
+
subject do
|
16
|
+
Freighter.options.environment = 'production'
|
17
|
+
Freighter::Parser.new(config_path).images('prod1.example.com')
|
18
|
+
end
|
19
|
+
|
20
|
+
context "deploy all" do
|
21
|
+
before do
|
22
|
+
Freighter.options.deploy_all = true
|
23
|
+
Freighter.options.app_name = nil
|
24
|
+
end
|
25
|
+
|
26
|
+
it "should retrieve images" do
|
27
|
+
expect(subject.map { |h| h['name'] }).to eq ["organization/imageName:latest", "organization/otherImage:latest"]
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
context "deploy one app" do
|
32
|
+
before do
|
33
|
+
Freighter.options.deploy_all = false
|
34
|
+
Freighter.options.app_name = 'otherApp'
|
35
|
+
end
|
36
|
+
|
37
|
+
it "should find one image" do
|
38
|
+
expect(subject.length).to eq 1
|
39
|
+
expect(subject.first['name']).to eq "organization/otherImage:latest"
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
context "app not found" do
|
44
|
+
before do
|
45
|
+
Freighter.options.deploy_all = false
|
46
|
+
Freighter.options.app_name = nil
|
47
|
+
end
|
48
|
+
|
49
|
+
it "should raise an exception" do
|
50
|
+
expect { subject }.to raise_error RuntimeError, "app(s) to deploy not specified"
|
51
|
+
end
|
52
|
+
end
|
53
|
+
end # images
|
54
|
+
|
55
|
+
describe "environment" do
|
56
|
+
it "should fetch the right environment" do
|
57
|
+
expect(subject.environment).to have_key('hosts')
|
58
|
+
expect(subject.environment['hosts'].first.fetch('host')).to eq 'staging.example.com'
|
59
|
+
end
|
60
|
+
end # environment
|
61
|
+
|
62
|
+
end
|
data/spec/spec_helper.rb
ADDED
@@ -0,0 +1,95 @@
|
|
1
|
+
# This file was generated by the `rspec --init` command. Conventionally, all
|
2
|
+
# specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`.
|
3
|
+
# The generated `.rspec` file contains `--require spec_helper` which will cause this
|
4
|
+
# file to always be loaded, without a need to explicitly require it in any files.
|
5
|
+
#
|
6
|
+
# Given that it is always loaded, you are encouraged to keep this file as
|
7
|
+
# light-weight as possible. Requiring heavyweight dependencies from this file
|
8
|
+
# will add to the boot time of your test suite on EVERY test run, even for an
|
9
|
+
# individual file that may not need all of that loaded. Instead, consider making
|
10
|
+
# a separate helper file that requires the additional dependencies and performs
|
11
|
+
# the additional setup, and require it from the spec files that actually need it.
|
12
|
+
#
|
13
|
+
# The `.rspec` file also contains a few flags that are not defaults but that
|
14
|
+
# users commonly want.
|
15
|
+
#
|
16
|
+
# See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
|
17
|
+
require 'freighter'
|
18
|
+
require 'pry-nav'
|
19
|
+
BASE_DIR = File.expand_path('../../', __FILE__)
|
20
|
+
|
21
|
+
RSpec.configure do |config|
|
22
|
+
# rspec-expectations config goes here. You can use an alternate
|
23
|
+
# assertion/expectation library such as wrong or the stdlib/minitest
|
24
|
+
# assertions if you prefer.
|
25
|
+
config.expect_with :rspec do |expectations|
|
26
|
+
# This option will default to `true` in RSpec 4. It makes the `description`
|
27
|
+
# and `failure_message` of custom matchers include text for helper methods
|
28
|
+
# defined using `chain`, e.g.:
|
29
|
+
# be_bigger_than(2).and_smaller_than(4).description
|
30
|
+
# # => "be bigger than 2 and smaller than 4"
|
31
|
+
# ...rather than:
|
32
|
+
# # => "be bigger than 2"
|
33
|
+
expectations.include_chain_clauses_in_custom_matcher_descriptions = true
|
34
|
+
end
|
35
|
+
|
36
|
+
# rspec-mocks config goes here. You can use an alternate test double
|
37
|
+
# library (such as bogus or mocha) by changing the `mock_with` option here.
|
38
|
+
config.mock_with :rspec do |mocks|
|
39
|
+
# Prevents you from mocking or stubbing a method that does not exist on
|
40
|
+
# a real object. This is generally recommended, and will default to
|
41
|
+
# `true` in RSpec 4.
|
42
|
+
mocks.verify_partial_doubles = true
|
43
|
+
end
|
44
|
+
|
45
|
+
config.filter_run :focus
|
46
|
+
config.run_all_when_everything_filtered = true
|
47
|
+
# The settings below are suggested to provide a good initial experience
|
48
|
+
# with RSpec, but feel free to customize to your heart's content.
|
49
|
+
=begin
|
50
|
+
# These two settings work together to allow you to limit a spec run
|
51
|
+
# to individual examples or groups you care about by tagging them with
|
52
|
+
# `:focus` metadata. When nothing is tagged with `:focus`, all examples
|
53
|
+
# get run.
|
54
|
+
config.filter_run :focus
|
55
|
+
config.run_all_when_everything_filtered = true
|
56
|
+
|
57
|
+
# Limits the available syntax to the non-monkey patched syntax that is recommended.
|
58
|
+
# For more details, see:
|
59
|
+
# - http://myronmars.to/n/dev-blog/2012/06/rspecs-new-expectation-syntax
|
60
|
+
# - http://teaisaweso.me/blog/2013/05/27/rspecs-new-message-expectation-syntax/
|
61
|
+
# - http://myronmars.to/n/dev-blog/2014/05/notable-changes-in-rspec-3#new__config_option_to_disable_rspeccore_monkey_patching
|
62
|
+
config.disable_monkey_patching!
|
63
|
+
|
64
|
+
# This setting enables warnings. It's recommended, but in some cases may
|
65
|
+
# be too noisy due to issues in dependencies.
|
66
|
+
config.warnings = true
|
67
|
+
|
68
|
+
# Many RSpec users commonly either run the entire suite or an individual
|
69
|
+
# file, and it's useful to allow more verbose output when running an
|
70
|
+
# individual spec file.
|
71
|
+
if config.files_to_run.one?
|
72
|
+
# Use the documentation formatter for detailed output,
|
73
|
+
# unless a formatter has already been configured
|
74
|
+
# (e.g. via a command-line flag).
|
75
|
+
config.default_formatter = 'doc'
|
76
|
+
end
|
77
|
+
|
78
|
+
# Print the 10 slowest examples and example groups at the
|
79
|
+
# end of the spec run, to help surface which specs are running
|
80
|
+
# particularly slow.
|
81
|
+
config.profile_examples = 10
|
82
|
+
|
83
|
+
# Run specs in random order to surface order dependencies. If you find an
|
84
|
+
# order dependency and want to debug it, you can fix the order by providing
|
85
|
+
# the seed, which is printed after each run.
|
86
|
+
# --seed 1234
|
87
|
+
config.order = :random
|
88
|
+
|
89
|
+
# Seed global randomization in this process using the `--seed` CLI option.
|
90
|
+
# Setting this allows you to use `--seed` to deterministically reproduce
|
91
|
+
# test failures related to randomization by passing the same `--seed` value
|
92
|
+
# as the one that triggered the failure.
|
93
|
+
Kernel.srand config.seed
|
94
|
+
=end
|
95
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: freighter
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0
|
4
|
+
version: 0.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Sean McCleary
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-11-
|
11
|
+
date: 2014-11-12 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -61,16 +61,25 @@ extensions: []
|
|
61
61
|
extra_rdoc_files: []
|
62
62
|
files:
|
63
63
|
- ".gitignore"
|
64
|
+
- ".rspec"
|
64
65
|
- Gemfile
|
65
66
|
- LICENSE.txt
|
66
67
|
- README.md
|
67
68
|
- Rakefile
|
68
69
|
- bin/freighter
|
70
|
+
- config/freighter.example.yml
|
69
71
|
- freighter.gemspec
|
70
72
|
- lib/freighter.rb
|
73
|
+
- lib/freighter/deploy.rb
|
74
|
+
- lib/freighter/docker_rest_api.rb
|
75
|
+
- lib/freighter/helpers.rb
|
71
76
|
- lib/freighter/logger.rb
|
72
|
-
- lib/freighter/
|
77
|
+
- lib/freighter/parser.rb
|
78
|
+
- lib/freighter/ssh.rb
|
73
79
|
- lib/freighter/version.rb
|
80
|
+
- spec/lib/freighter/deploy_spec.rb
|
81
|
+
- spec/lib/freighter/parser_spec.rb
|
82
|
+
- spec/spec_helper.rb
|
74
83
|
homepage: https://github.com/mrinterweb/freighter
|
75
84
|
licenses:
|
76
85
|
- MIT
|
@@ -95,4 +104,7 @@ rubygems_version: 2.2.2
|
|
95
104
|
signing_key:
|
96
105
|
specification_version: 4
|
97
106
|
summary: Ruby gem to deploy docker containers
|
98
|
-
test_files:
|
107
|
+
test_files:
|
108
|
+
- spec/lib/freighter/deploy_spec.rb
|
109
|
+
- spec/lib/freighter/parser_spec.rb
|
110
|
+
- spec/spec_helper.rb
|
data/lib/freighter/parse.rb
DELETED
@@ -1,18 +0,0 @@
|
|
1
|
-
require 'yaml'
|
2
|
-
require 'pry'
|
3
|
-
require 'freighter/logger'
|
4
|
-
|
5
|
-
module Freighter
|
6
|
-
class Parse
|
7
|
-
def initialize(options)
|
8
|
-
@logger = Logger.new options
|
9
|
-
@config_path = options.config_path
|
10
|
-
begin
|
11
|
-
@config = YAML.load_file(@config_path)
|
12
|
-
@logger.log "config file parsed", :verbose
|
13
|
-
rescue Errno::ENOENT => e
|
14
|
-
@logger.error "Error parsing freighter config file.\n path: #{@config_path}\n #{e}"
|
15
|
-
end
|
16
|
-
end
|
17
|
-
end
|
18
|
-
end
|