pups 1.0.3 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6637fcde0de413ee3d9925e644f84547563f86ba44d67692daa9f431520ab598
4
- data.tar.gz: 52ed370e7071a7eb53cd87bc455dec4f48ca84c691ac28f1429e6985673a4d21
3
+ metadata.gz: e76a5c569771df90ca70c5e87fcef1f1252192b487fa0e772f43adcc7118adc0
4
+ data.tar.gz: 71a81d1ed1c2a4fe0dc1618b399dab1fc3a0e4b47d4afd9e8a5fe03b17411cf1
5
5
  SHA512:
6
- metadata.gz: dc4828cccbe30f2d64d265c5bbed1558e17754a1866ebee549fc590eb26e80de8388deb40905fd3715af029e68c7e5592520b6eaeb1a61904f36114fae1f49ff
7
- data.tar.gz: 9f5da00c947a36c01d574a497afad7f9d218870ab898195d2427622817b0f260f3e8ccf2650113ba0f20bf585d9673a80af9d6174d81b4cd6301cacfa3ce84af
6
+ metadata.gz: 2cb538356ede8e3554047bd59818500533dfa4ab7ab2446020d481c38121e7f88a3bfac481ae3985f45dde11384c3cf915db2be60dba1f5fee54e09886f53337
7
+ data.tar.gz: fedb98ab255593cb8804d05bed9fc1367135e5d891e3ea813795dafe4da5407c59123fdf52f99d42a402695a47c74dc828fac9be15d475d1dfc3a1317958d125
@@ -0,0 +1,29 @@
1
+ name: Tests
2
+
3
+ on:
4
+ push:
5
+ branches:
6
+ - master
7
+ pull_request: {}
8
+
9
+ jobs:
10
+ test:
11
+ name: "pups tests"
12
+ runs-on: ${{ matrix.os }}
13
+ timeout-minutes: 5
14
+
15
+ strategy:
16
+ fail-fast: true
17
+ matrix:
18
+ os: [ubuntu-latest]
19
+ ruby: ["2.7"]
20
+
21
+ steps:
22
+ - uses: actions/checkout@v2
23
+ - uses: ruby/setup-ruby@v1
24
+ with:
25
+ ruby-version: ${{ matrix.ruby }}
26
+ bundler-cache: true
27
+ - name: Run minitest
28
+ run: |
29
+ rake test
@@ -0,0 +1,27 @@
1
+ name: Lint
2
+
3
+ on:
4
+ push:
5
+ branches:
6
+ - master
7
+ pull_request: {}
8
+
9
+ jobs:
10
+ lint:
11
+ name: "pups lint"
12
+ runs-on: ${{ matrix.os }}
13
+ timeout-minutes: 5
14
+
15
+ strategy:
16
+ fail-fast: true
17
+ matrix:
18
+ os: [ubuntu-latest]
19
+ ruby: ["2.7"]
20
+
21
+ steps:
22
+ - uses: actions/checkout@v2
23
+ - uses: ruby/setup-ruby@v1
24
+ with:
25
+ ruby-version: ${{ matrix.ruby }}
26
+ bundler-cache: true
27
+ - run: bundle exec rubocop
data/.rubocop.yml ADDED
@@ -0,0 +1,3 @@
1
+ inherit_gem:
2
+ rubocop-discourse: default.yml
3
+
data/Gemfile CHANGED
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  source 'https://rubygems.org'
2
4
 
3
5
  # Specify your gem's dependencies in pups.gemspec
data/Guardfile CHANGED
@@ -1,6 +1,8 @@
1
+ # frozen_string_literal: true
2
+
1
3
  guard :minitest do
2
4
  # with Minitest::Unit
3
- watch(%r{^test/(.*)\/?(.*)_test\.rb$})
5
+ watch(%r{^test/(.*)/?(.*)_test\.rb$})
4
6
  watch(%r{^lib/(.*/)?([^/]+)\.rb$}) { |m| "test/#{m[2]}_test.rb" }
5
7
  watch(%r{^test/test_helper\.rb$}) { 'test' }
6
8
  end
data/README.md CHANGED
@@ -20,6 +20,19 @@ Or install it yourself as:
20
20
 
21
21
  pups is a small library that allows you to automate the process of creating Unix images.
22
22
 
23
+ ```
24
+ Usage: pups [options] [FILE|--stdin]
25
+ --stdin Read input from stdin.
26
+ --quiet Don't print any logs.
27
+ --ignore <elements> Ignore specific configuration elements, multiple elements can be provided (comma-delimited).
28
+ Useful if you want to skip over config in a pups execution.
29
+ e.g. `--ignore env,params`.
30
+ --gen-docker-run-args Output arguments from the pups configuration for input into a docker run command. All other pups config is ignored.
31
+ -h, --help
32
+ ```
33
+
34
+ pups requires input either via a stdin stream or a filename. The entire input is parsed prior to any templating or command execution.
35
+
23
36
  Example:
24
37
 
25
38
  ```
@@ -35,6 +48,14 @@ Running: `pups somefile.yaml` will execute the shell script resulting in a file
35
48
 
36
49
  ### Features
37
50
 
51
+ #### Docker run argument generation
52
+
53
+ The `--gen-docker-run-args` argument is used to make pups output arguments be in the format of `docker run <arguments output>`. Specifically, pups
54
+ will take any `env`, `volume`, `labels`, `links`, and `expose` configuration, and coerce that into the format expected by `docker run`. This can be useful
55
+ when pups is being used to configure an image (e.g. by executing a series of commands) that is then going to be run as a container. That way, the runtime and image
56
+ configuration can be specified within the same yaml files.
57
+
58
+
38
59
  #### Environment Variables
39
60
 
40
61
  By default, pups automatically imports your environment variables and includes them as params.
@@ -149,15 +170,31 @@ Will merge the yaml file with the inline contents.
149
170
 
150
171
  #### A common environment
151
172
 
152
- This is implemented in discourse_docker's launcher, not in pups - therefore it does not work in standalone pups.
173
+ Environment variables can be specified under the `env` key, which will be included in the environment for the template.
174
+
175
+ ```
176
+ env:
177
+ MY_ENV: "a couple of words"
178
+ run:
179
+ - exec: echo $MY_ENV > tmpfile
180
+ ```
181
+
182
+ `tmpfile` will contain `a couple of words`.
183
+
184
+ You can also specify variables to be templated within the environment, such as:
153
185
 
154
186
  ```
155
187
  env:
156
- MY_ENV: 1
188
+ greeting: "hello, {{location}}!"
189
+ env_template:
190
+ location: world
157
191
  ```
158
192
 
159
- All executions will get this environment set up
193
+ In this example, the `greeting` environment variable will be set to `hello, world!` during initialisation as the `{{location}}` variable will be templated as `world`.
194
+ Pups will also look in the environment itself at runtime for template variables, prefixed with `env_template_<variable name>`.
195
+ Note that strings should be quoted to prevent YAML from parsing the `{ }` characters.
160
196
 
197
+ All commands executed will inherit the environment once parsing and variable interpolation has been completed.
161
198
 
162
199
  ## Contributing
163
200
 
data/Rakefile CHANGED
@@ -1,10 +1,12 @@
1
- require "bundler/gem_tasks"
2
- require "rake/testtask"
1
+ # frozen_string_literal: true
2
+
3
+ require 'bundler/gem_tasks'
4
+ require 'rake/testtask'
3
5
 
4
6
  Rake::TestTask.new do |t|
5
- t.libs << "test"
6
- t.libs << "lib"
7
- t.test_files = FileList["test/*_test.rb"]
7
+ t.libs << 'test'
8
+ t.libs << 'lib'
9
+ t.test_files = FileList['test/*_test.rb']
8
10
  end
9
11
 
10
- task :default => :test
12
+ task default: :test
data/bin/pups CHANGED
@@ -1,9 +1,9 @@
1
1
  #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
2
3
 
3
- $LOAD_PATH.unshift File.dirname(__FILE__) + "/../lib"
4
+ $LOAD_PATH.unshift "#{File.dirname(__FILE__)}/../lib"
4
5
 
5
- require "pups"
6
- require "pups/cli"
6
+ require 'pups'
7
+ require 'pups/cli'
7
8
 
8
9
  Pups::Cli.run(ARGV)
9
-
data/lib/pups.rb CHANGED
@@ -1,15 +1,17 @@
1
- require "logger"
2
- require "yaml"
1
+ # frozen_string_literal: true
3
2
 
4
- require "pups/version"
5
- require "pups/config"
6
- require "pups/command"
7
- require "pups/exec_command"
8
- require "pups/merge_command"
9
- require "pups/replace_command"
10
- require "pups/file_command"
3
+ require 'logger'
4
+ require 'yaml'
11
5
 
12
- require "pups/runit"
6
+ require 'pups/version'
7
+ require 'pups/config'
8
+ require 'pups/command'
9
+ require 'pups/exec_command'
10
+ require 'pups/merge_command'
11
+ require 'pups/replace_command'
12
+ require 'pups/file_command'
13
+ require 'pups/docker'
14
+ require 'pups/runit'
13
15
 
14
16
  module Pups
15
17
  class ExecError < RuntimeError
@@ -18,10 +20,18 @@ module Pups
18
20
 
19
21
  def self.log
20
22
  # at the moment docker likes this
21
- @logger ||= Logger.new(STDERR)
23
+ @logger ||= Logger.new($stderr)
22
24
  end
23
25
 
24
26
  def self.log=(logger)
25
27
  @logger = logger
26
28
  end
29
+
30
+ def self.silence
31
+ if @logger
32
+ @logger.close
33
+ end
34
+
35
+ @logger = Logger.new(File.open(File::NULL, "w"))
36
+ end
27
37
  end
data/lib/pups/cli.rb CHANGED
@@ -1,35 +1,70 @@
1
- class Pups::Cli
1
+ # frozen_string_literal: true
2
2
 
3
- def self.usage
4
- puts "Usage: pups FILE or pups --stdin"
5
- exit 1
6
- end
7
- def self.run(args)
8
- if args.length != 1
9
- usage
3
+ require 'optparse'
4
+
5
+ module Pups
6
+ class Cli
7
+ def self.opts
8
+ OptionParser.new do |opts|
9
+ opts.banner = 'Usage: pups [FILE|--stdin]'
10
+ opts.on('--stdin', 'Read input from stdin.')
11
+ opts.on('--quiet', "Don't print any logs.")
12
+ opts.on('--ignore <element(s)>', Array, "Ignore these template configuration elements, multiple elements can be provided (comma-delimited).")
13
+ opts.on('--gen-docker-run-args', 'Output arguments from the pups configuration for input into a docker run command. All other pups config is ignored.')
14
+ opts.on('-h', '--help') do
15
+ puts opts
16
+ exit
17
+ end
18
+ end
10
19
  end
11
20
 
12
- Pups.log.info("Loading #{args[0]}")
13
- if args[0] == "--stdin"
14
- conf = STDIN.readlines.join
15
- split = conf.split("_FILE_SEPERATOR_")
16
-
17
- conf = nil
18
- split.each do |data|
19
- current = YAML.load(data.strip)
20
- if conf
21
- conf = Pups::MergeCommand.deep_merge(conf, current, :merge_arrays)
22
- else
23
- conf = current
21
+ def self.parse_args(args)
22
+ options = {}
23
+ opts.parse!(args, into: options)
24
+ options
25
+ end
26
+
27
+ def self.run(args)
28
+ options = parse_args(args)
29
+ input_file = options[:stdin] ? 'stdin' : args.last
30
+ unless input_file
31
+ puts opts.parse!(%w[--help])
32
+ exit
33
+ end
34
+
35
+ if options[:quiet]
36
+ Pups.silence
37
+ end
38
+
39
+ Pups.log.info("Reading from #{input_file}")
40
+
41
+ if options[:stdin]
42
+ conf = $stdin.readlines.join
43
+ split = conf.split('_FILE_SEPERATOR_')
44
+
45
+ conf = nil
46
+ split.each do |data|
47
+ current = YAML.safe_load(data.strip)
48
+ conf = if conf
49
+ Pups::MergeCommand.deep_merge(conf, current, :merge_arrays)
50
+ else
51
+ current
52
+ end
24
53
  end
54
+
55
+ config = Pups::Config.new(conf, options[:ignore])
56
+ else
57
+ config = Pups::Config.load_file(input_file, options[:ignore])
58
+ end
59
+
60
+ if options[:"gen-docker-run-args"]
61
+ print config.generate_docker_run_arguments
62
+ return
25
63
  end
26
64
 
27
- config = Pups::Config.new(conf)
28
- else
29
- config = Pups::Config.load_file(args[0])
65
+ config.run
66
+ ensure
67
+ Pups::ExecCommand.terminate_async
30
68
  end
31
- config.run
32
- ensure
33
- Pups::ExecCommand.terminate_async
34
69
  end
35
70
  end
data/lib/pups/command.rb CHANGED
@@ -1,17 +1,20 @@
1
- class Pups::Command
1
+ # frozen_string_literal: true
2
2
 
3
- def self.run(command,params)
4
- case command
5
- when String then self.from_str(command,params).run
6
- when Hash then self.from_hash(command,params).run
3
+ module Pups
4
+ class Command
5
+ def self.run(command, params)
6
+ case command
7
+ when String then from_str(command, params).run
8
+ when Hash then from_hash(command, params).run
9
+ end
7
10
  end
8
- end
9
11
 
10
- def self.interpolate_params(cmd,params)
11
- Pups::Config.interpolate_params(cmd,params)
12
- end
12
+ def self.interpolate_params(cmd, params)
13
+ Pups::Config.interpolate_params(cmd, params)
14
+ end
13
15
 
14
- def interpolate_params(cmd)
15
- Pups::Command.interpolate_params(cmd,@params)
16
+ def interpolate_params(cmd)
17
+ Pups::Command.interpolate_params(cmd, @params)
18
+ end
16
19
  end
17
20
  end
data/lib/pups/config.rb CHANGED
@@ -1,128 +1,171 @@
1
- class Pups::Config
1
+ # frozen_string_literal: true
2
2
 
3
- attr_reader :config, :params
3
+ module Pups
4
+ class Config
5
+ attr_reader :config, :params
4
6
 
5
- def self.load_file(config_file)
6
- begin
7
- new YAML.load_file(config_file)
7
+ def initialize(config, ignored = nil)
8
+ @config = config
9
+
10
+ # remove any ignored config elements prior to any more processing
11
+ ignored&.each { |e| @config.delete(e) }
12
+
13
+ # set some defaults to prevent checks in various functions
14
+ ['env_template', 'env', 'labels', 'params'].each { |key| @config[key] = {} unless @config.has_key?(key) }
15
+
16
+ # Order here is important.
17
+ Pups::Config.combine_template_and_process_env(@config, ENV)
18
+ Pups::Config.prepare_env_template_vars(@config['env_template'], ENV)
19
+
20
+ # Templating is supported in env and label variables.
21
+ Pups::Config.transform_config_with_templated_vars(@config['env_template'], ENV)
22
+ Pups::Config.transform_config_with_templated_vars(@config['env_template'], @config['env'])
23
+ Pups::Config.transform_config_with_templated_vars(@config['env_template'], @config['labels'])
24
+
25
+ @params = @config["params"]
26
+ ENV.each do |k, v|
27
+ @params["$ENV_#{k}"] = v
28
+ end
29
+ inject_hooks
30
+ end
31
+
32
+ def self.load_file(config_file, ignored = nil)
33
+ Config.new(YAML.load_file(config_file), ignored)
8
34
  rescue Exception
9
- STDERR.puts "Failed to parse #{config_file}"
10
- STDERR.puts "This is probably a formatting error in #{config_file}"
11
- STDERR.puts "Cannot continue. Edit #{config_file} and try again."
35
+ warn "Failed to parse #{config_file}"
36
+ warn "This is probably a formatting error in #{config_file}"
37
+ warn "Cannot continue. Edit #{config_file} and try again."
12
38
  raise
13
39
  end
14
- end
15
40
 
16
- def self.load_config(config)
17
- new YAML.load(config)
18
- end
41
+ def self.load_config(config, ignored = nil)
42
+ Config.new(YAML.safe_load(config), ignored)
43
+ end
19
44
 
20
- def initialize(config)
21
- @config = config
22
- validate!(@config)
23
- @params = @config["params"]
24
- @params ||= {}
25
- ENV.each do |k,v|
26
- @params["$ENV_#{k}"] = v
45
+ def self.prepare_env_template_vars(env_template, env)
46
+ # Merge env_template variables from env and templates.
47
+ env.each do |k, v|
48
+ if k.include?('env_template_')
49
+ key = k.gsub('env_template_', '')
50
+ env_template[key] = v.to_s
51
+ end
52
+ end
27
53
  end
28
- inject_hooks
29
- end
30
54
 
31
- def validate!(conf)
32
- # raise proper errors if nodes are missing etc
33
- end
55
+ def self.transform_config_with_templated_vars(env_template, to_transform)
56
+ # Transform any templated variables prior to copying to params.
57
+ # This has no effect if no env_template was provided.
58
+ env_template.each do |k, v|
59
+ to_transform.each do |key, val|
60
+ if val.to_s.include?("{{#{k}}}")
61
+ to_transform[key] = val.gsub("{{#{k}}}", v.to_s)
62
+ end
63
+ end
64
+ end
65
+ end
66
+
67
+ def self.combine_template_and_process_env(config, env)
68
+ # Merge all template env variables and process env variables, so that env
69
+ # variables can be provided both by configuration and runtime variables.
70
+ config["env"].each { |k, v| env[k] = v.to_s }
71
+ end
34
72
 
35
- def inject_hooks
36
- return unless hooks = @config["hooks"]
73
+ def inject_hooks
74
+ return unless hooks = @config['hooks']
37
75
 
38
- run = @config["run"]
76
+ run = @config['run']
77
+
78
+ positions = {}
79
+ run.each do |row|
80
+ next unless row.is_a?(Hash)
39
81
 
40
- positions = {}
41
- run.each do |row|
42
- if Hash === row
43
82
  command = row.first
44
- if Hash === command[1]
45
- hook = command[1]["hook"]
83
+ if command[1].is_a?(Hash)
84
+ hook = command[1]['hook']
46
85
  positions[hook] = row if hook
47
86
  end
48
87
  end
49
- end
50
-
51
- hooks.each do |full, list|
52
88
 
53
- offset = nil
54
- name = nil
89
+ hooks.each do |full, list|
90
+ offset = nil
91
+ name = nil
55
92
 
56
- if full =~ /^after_/
57
- name = full[6..-1]
58
- offset = 1
59
- end
93
+ if full =~ /^after_/
94
+ name = full[6..-1]
95
+ offset = 1
96
+ end
60
97
 
61
- if full =~ /^before_/
62
- name = full[7..-1]
63
- offset = 0
64
- end
98
+ if full =~ /^before_/
99
+ name = full[7..-1]
100
+ offset = 0
101
+ end
65
102
 
66
- index = run.index(positions[name])
103
+ index = run.index(positions[name])
67
104
 
68
- if index && index >= 0
69
- run.insert(index + offset, *list)
70
- else
71
- Pups.log.info "Skipped missing #{full} hook"
105
+ if index && index >= 0
106
+ run.insert(index + offset, *list)
107
+ else
108
+ Pups.log.info "Skipped missing #{full} hook"
109
+ end
72
110
  end
73
-
74
111
  end
75
- end
76
112
 
77
- def run
78
- run_commands
79
- rescue => e
80
- exit_code = 1
81
- if Pups::ExecError === e
82
- exit_code = e.exit_code
113
+ def generate_docker_run_arguments
114
+ output = []
115
+ output << Pups::Docker.generate_env_arguments(config['env'])
116
+ output << Pups::Docker.generate_link_arguments(config['links'])
117
+ output << Pups::Docker.generate_expose_arguments(config['expose'])
118
+ output << Pups::Docker.generate_volume_arguments(config['volumes'])
119
+ output << Pups::Docker.generate_label_arguments(config['labels'])
120
+ output.sort!.join(" ").strip
83
121
  end
84
- unless exit_code == 77
85
- puts
86
- puts
87
- puts "FAILED"
88
- puts "-" * 20
89
- puts "#{e.class}: #{e}"
90
- puts "Location of failure: #{e.backtrace[0]}"
91
- if @last_command
92
- puts "#{@last_command[:command]} failed with the params #{@last_command[:params].inspect}"
122
+
123
+ def run
124
+ run_commands
125
+ rescue StandardError => e
126
+ exit_code = 1
127
+ exit_code = e.exit_code if e.is_a?(Pups::ExecError)
128
+ unless exit_code == 77
129
+ puts
130
+ puts
131
+ puts 'FAILED'
132
+ puts '-' * 20
133
+ puts "#{e.class}: #{e}"
134
+ puts "Location of failure: #{e.backtrace[0]}"
135
+ puts "#{@last_command[:command]} failed with the params #{@last_command[:params].inspect}" if @last_command
93
136
  end
137
+ exit exit_code
94
138
  end
95
- exit exit_code
96
- end
97
139
 
98
- def run_commands
99
- @config["run"].each do |item|
100
- item.each do |k,v|
101
- type = case k
102
- when "exec" then Pups::ExecCommand
103
- when "merge" then Pups::MergeCommand
104
- when "replace" then Pups::ReplaceCommand
105
- when "file" then Pups::FileCommand
106
- else raise SyntaxError.new("Invalid run command #{k}")
107
- end
108
-
109
- @last_command = { command: k, params: v }
110
- type.run(v, @params)
140
+ def run_commands
141
+ @config['run']&.each do |item|
142
+ item.each do |k, v|
143
+ type = case k
144
+ when 'exec' then Pups::ExecCommand
145
+ when 'merge' then Pups::MergeCommand
146
+ when 'replace' then Pups::ReplaceCommand
147
+ when 'file' then Pups::FileCommand
148
+ else raise SyntaxError, "Invalid run command #{k}"
149
+ end
150
+
151
+ @last_command = { command: k, params: v }
152
+ type.run(v, @params)
153
+ end
111
154
  end
112
155
  end
113
- end
114
156
 
115
- def interpolate_params(cmd)
116
- self.class.interpolate_params(cmd,@params)
117
- end
157
+ def interpolate_params(cmd)
158
+ self.class.interpolate_params(cmd, @params)
159
+ end
160
+
161
+ def self.interpolate_params(cmd, params)
162
+ return unless cmd
118
163
 
119
- def self.interpolate_params(cmd, params)
120
- return unless cmd
121
- processed = cmd.dup
122
- params.each do |k,v|
123
- processed.gsub!("$#{k}", v.to_s)
164
+ processed = cmd.dup
165
+ params.each do |k, v|
166
+ processed.gsub!("$#{k}", v.to_s)
167
+ end
168
+ processed
124
169
  end
125
- processed
126
170
  end
127
-
128
171
  end