pupistry 0.0.3 → 0.0.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +59 -44
- data/bin/pupistry +67 -19
- data/lib/pupistry/artifact.rb +22 -1
- data/lib/pupistry/bootstrap.rb +72 -5
- metadata +16 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5ff9e16fb23fac8d09a0bf86e4bd88c452f52b29
|
4
|
+
data.tar.gz: f944f6ea67120033772613d9e88186ca9cacdbf8
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: beb20b7264be3c5a105dc6cb5e405944b51126d0a9880f998ec226c058e1869bb05037fa829e962fd487d2281292d355d2bcefbe5bbb2ac2039e8bd016cad278
|
7
|
+
data.tar.gz: 0bc78b2c6f99795c877b75da67068844c89c7fc091b1c371109986d0fce1cc03d32c7ccd2b320ec412d3fe8cd02ad12d97e0719bb714d52f90cea00fb250d10a
|
data/README.md
CHANGED
@@ -109,16 +109,56 @@ off if you have it disabled. :-)
|
|
109
109
|
|
110
110
|
## Bootstrapping nodes
|
111
111
|
|
112
|
-
|
113
|
-
and
|
114
|
-
|
115
|
-
|
112
|
+
New machines need to be bootstrapped in order to install Pupistry, configure it
|
113
|
+
and be able to download configuration. Generally this is a step done differently
|
114
|
+
site-by-site (and you can still do it that way if you want), but if you want a
|
115
|
+
nice easy life, Pupistry can generate you a bootstrap script for your platform.
|
116
|
+
|
117
|
+
$ pupistry bootstrap
|
118
|
+
- rhel-7
|
119
|
+
- ubuntu-14.04
|
120
|
+
|
121
|
+
$ pupistry boostrap --template rhel-7
|
122
|
+
# Bootstrap for Red Hat Enterprise Linux Platform
|
123
|
+
# Compatible with RHEL 7, CentOS 7 and maybe other variations.
|
124
|
+
|
125
|
+
rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
|
126
|
+
rpm -ivh http://download.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
|
127
|
+
|
128
|
+
yum update --assumeyes
|
129
|
+
yum install --assumeyes puppet ruby-devel rubygems
|
130
|
+
yum install --assumeyes gcc zlib-devel libxml2-devel patch
|
131
|
+
|
132
|
+
gem install pupistry
|
133
|
+
mkdir /etc/pupistry
|
134
|
+
cat > /etc/pupistry/settings.yaml << "EOF"
|
135
|
+
general:
|
136
|
+
app_cache: ~/.pupistry/cache
|
137
|
+
s3_bucket: example
|
138
|
+
s3_prefix:
|
139
|
+
gpg_disable: true
|
140
|
+
gpg_signing_key: XYZXYZ
|
141
|
+
agent:
|
142
|
+
puppetcode: /etc/puppet/environents/
|
143
|
+
access_key_id:
|
144
|
+
secret_access_key:
|
145
|
+
region: ap-southeast-2
|
146
|
+
proxy_uri:
|
147
|
+
EOF
|
148
|
+
pupistry apply --verbose
|
116
149
|
|
117
|
-
|
150
|
+
You generally can run this on a new non-Puppetised machine, or paste into the
|
151
|
+
user data field of most cloud providers like AWS or Digital Ocean. If using CFN
|
152
|
+
with AWS, you can make it part of the stack itself.
|
118
153
|
|
154
|
+
These bootstraps aren't mandatory, if you prefer a different approach you can
|
155
|
+
use these as an example and write your own - generally the essential bit is to
|
156
|
+
get puppet installed, get pupistry (and deps to build it's gems) installed and
|
157
|
+
write the config before finally executing your first Pupistry/Puppet run.
|
119
158
|
|
120
|
-
|
121
|
-
|
159
|
+
If using AWS and IAM Roles feature, it is acceptable for access_key_id and
|
160
|
+
secret_access_key to be blank, if not you will need to have these set to an
|
161
|
+
account with read-only access to the configured S3 bucket!
|
122
162
|
|
123
163
|
|
124
164
|
## Running Puppet on target nodes
|
@@ -230,7 +270,7 @@ Pupistry will default to applying the "master" branch if one is not listed, if
|
|
230
270
|
you are doing branch-based environments, you can specifiy when bootstrapping
|
231
271
|
and override on a per-execution basis.
|
232
272
|
|
233
|
-
You'll notice pretty quickly if something is broken when doing `
|
273
|
+
You'll notice pretty quickly if something is broken when doing `pupistry apply`
|
234
274
|
|
235
275
|
Confused? No worried, check out the sample repo that shows a very simple setup.
|
236
276
|
You can copy this and start your own Puppet adventure, just add in your modules
|
@@ -258,11 +298,15 @@ You can fetch the module from:
|
|
258
298
|
https://github.com/jethrocarr/puppet-pupistry
|
259
299
|
|
260
300
|
If you're doing r10k and Puppet masterless from scratch, this is probably
|
261
|
-
something you want to make life easy.
|
301
|
+
something you want to make life easy. With r10k, just add the following to your
|
302
|
+
Puppetfile:
|
262
303
|
|
304
|
+
# Install the Pupistry companion module
|
305
|
+
mod 'jethrocarr/pupistry'
|
263
306
|
|
264
307
|
|
265
|
-
|
308
|
+
|
309
|
+
## 4. Building you first node (Bootstrapping)
|
266
310
|
|
267
311
|
No need for manual configuration of your servers/nodes, you just need to build
|
268
312
|
your first artifact with Pupistry (`pupistry build && pupistry push`) and then
|
@@ -274,41 +318,12 @@ The bootstrap script will:
|
|
274
318
|
2. Download the latest artifact
|
275
319
|
3. Trigger a Puppet run to build your server.
|
276
320
|
|
277
|
-
|
278
|
-
|
279
|
-
|
280
|
-
TODO: Currently being worked on, for now the following is a rough example of
|
281
|
-
what you can do to bootstrap a RHEL/CentOS7 box:
|
282
|
-
|
283
|
-
rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
|
284
|
-
rpm -ivh http://download.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
|
285
|
-
|
286
|
-
yum update --assumeyes
|
287
|
-
yum install --assumeyes puppet ruby-devel rubygems
|
288
|
-
yum install --assumeyes gcc zlib-devel libxml2-devel patch
|
289
|
-
|
290
|
-
gem install pupistry
|
291
|
-
mkdir /etc/pupistry
|
292
|
-
cat > /etc/pupistry/settings.yaml << "EOF"
|
293
|
-
general:
|
294
|
-
app_cache: ~/.pupistry/cache
|
295
|
-
s3_bucket: example
|
296
|
-
s3_prefix:
|
297
|
-
gpg_disable: true
|
298
|
-
gpg_signing_key: XXXXX
|
299
|
-
agent:
|
300
|
-
puppetcode: /etc/puppet/environments
|
301
|
-
access_key_id:
|
302
|
-
secret_access_key:
|
303
|
-
region: ap-southeast-2
|
304
|
-
proxy_uri:
|
305
|
-
EOF
|
306
|
-
pupistry apply --verbose
|
307
|
-
|
308
|
-
It will setup the Puppet repo for RHEL 7, install updates (remember that security
|
309
|
-
thing?) and then install the gem (and deps to build it). Then we generate the
|
310
|
-
minimal configuration file needed, and kick off a Puppet run!
|
321
|
+
These bootstrap scripts can be generated for you. Refer to "Bootstrapping" under
|
322
|
+
"Usage" instructions above for details.
|
311
323
|
|
324
|
+
The bootstrap script goal is to get you from stock OS to running Pupistry and
|
325
|
+
doing your first Puppet run. After that - it's up to you and your Puppet
|
326
|
+
skills to make your node actually do something useful. :-)
|
312
327
|
|
313
328
|
|
314
329
|
# Tutorials
|
data/bin/pupistry
CHANGED
@@ -23,9 +23,11 @@ class CLI < Thor
|
|
23
23
|
## Agent Commands
|
24
24
|
|
25
25
|
desc "apply", "Apply the latest Puppet artifact"
|
26
|
-
method_option :noop,
|
27
|
-
method_option :
|
28
|
-
method_option :
|
26
|
+
method_option :noop, :type => :boolean, :desc => "No changes mode (note: does change checked out artifact, but no Puppet changes)"
|
27
|
+
method_option :force, :type => :boolean, :desc => "Ignore existing versions, re-apply every time"
|
28
|
+
method_option :minimal, :type => :boolean, :desc => "Don't run Puppet unless the artifact has changed"
|
29
|
+
method_option :daemon, :type => :boolean, :desc => "Run as a system daemon"
|
30
|
+
method_option :environment, :type => :string, :desc => "Specifiy which environment to deploy (default: master)"
|
29
31
|
def apply
|
30
32
|
|
31
33
|
# Thor seems to force class options to be defined repeatedly? :-/
|
@@ -52,24 +54,66 @@ class CLI < Thor
|
|
52
54
|
exit 0
|
53
55
|
end
|
54
56
|
|
55
|
-
|
56
|
-
#
|
57
|
-
$logger.info "
|
57
|
+
|
58
|
+
# Fetch artifact versions
|
59
|
+
$logger.info "Checking version of artifact available..."
|
58
60
|
|
59
61
|
artifact = Pupistry::Artifact.new
|
60
62
|
artifact.checksum = artifact.fetch_latest
|
61
|
-
artifact.fetch_artifact
|
62
|
-
artifact.unpack
|
63
|
-
unless artifact.install
|
64
|
-
$logger.fatal "An unexpected error happened when installing the latest artifact, cancelling Puppet run"
|
65
|
-
exit 0
|
66
|
-
end
|
67
63
|
|
68
|
-
|
69
|
-
|
64
|
+
artifact_installed = Pupistry::Artifact.new
|
65
|
+
artifact_installed.checksum = artifact_installed.fetch_installed
|
70
66
|
|
71
67
|
|
72
|
-
#
|
68
|
+
# Download the new artifact if one has changed. If we already have this
|
69
|
+
# version, then we should skip downloading and go straight to running
|
70
|
+
# Puppet - unless the user runs with --force (eg to fix a corrupted
|
71
|
+
# artifact).
|
72
|
+
$logger.debug "Currently on #{artifact_installed.checksum}"
|
73
|
+
|
74
|
+
if artifact.checksum != artifact_installed.checksum or options[:force]
|
75
|
+
if options[:force]
|
76
|
+
$logger.warn "Forcing download of latest artifact regardless of current one."
|
77
|
+
end
|
78
|
+
|
79
|
+
# Install the artifact
|
80
|
+
$logger.info "Downloading latest artifact (#{artifact.checksum})..."
|
81
|
+
|
82
|
+
artifact.fetch_artifact
|
83
|
+
artifact.unpack
|
84
|
+
|
85
|
+
unless artifact.install
|
86
|
+
$logger.fatal "An unexpected error happened when installing the latest artifact, cancelling Puppet run"
|
87
|
+
exit 0
|
88
|
+
end
|
89
|
+
|
90
|
+
# Remove temporary unpacked files
|
91
|
+
artifact.clean_unpack
|
92
|
+
else
|
93
|
+
$logger.info "Already have latest artifact applied."
|
94
|
+
|
95
|
+
# By default we run Puppet even if we have the latest artifact. There's
|
96
|
+
# some grounds for debate about whether this is the right thing - in some
|
97
|
+
# ways it is often a waste of CPU, since if the artifact hasn't changed,
|
98
|
+
# then it's unlikley anything else has changed.
|
99
|
+
#
|
100
|
+
# But that's not always 100% true - Puppet will undo local changes or
|
101
|
+
# upgrade package versions (ensure => latest) if appropiate, so we should
|
102
|
+
# act like the standard command and attempt to apply whatever we can.
|
103
|
+
#
|
104
|
+
# To provide users with options, we provide the --lazy parameter to avoid
|
105
|
+
# running Puppet except when the artifact changes. By default, Puppet
|
106
|
+
# runs every thing to avoid surprise.
|
107
|
+
|
108
|
+
if options[:minimal]
|
109
|
+
$logger.info "Running with minimal effort mode enabled, not running Puppet since artifact version already applied"
|
110
|
+
exit 0
|
111
|
+
end
|
112
|
+
|
113
|
+
end
|
114
|
+
|
115
|
+
|
116
|
+
# Execute Puppet.
|
73
117
|
puppet_cmd = "puppet apply"
|
74
118
|
|
75
119
|
if options[:noop]
|
@@ -242,11 +286,15 @@ class CLI < Thor
|
|
242
286
|
|
243
287
|
if options[:template]
|
244
288
|
$logger.info "Generating bootstrap template #{options[:template]}"
|
245
|
-
else
|
246
|
-
puts "Listing all avaible templates"
|
247
|
-
Pupistry::Bootstrap.templates_list
|
248
289
|
|
249
|
-
|
290
|
+
templates = Pupistry::Bootstrap.new
|
291
|
+
templates.build options[:template]
|
292
|
+
else
|
293
|
+
templates = Pupistry::Bootstrap.new
|
294
|
+
templates.list
|
295
|
+
|
296
|
+
puts "--"
|
297
|
+
puts "Tip: Run `pupistry bootstrap --template example` to generate a specific template"
|
250
298
|
end
|
251
299
|
|
252
300
|
end
|
data/lib/pupistry/artifact.rb
CHANGED
@@ -111,6 +111,27 @@ module Pupistry
|
|
111
111
|
end
|
112
112
|
|
113
113
|
|
114
|
+
def fetch_installed
|
115
|
+
# Fetch the current version that is installed.
|
116
|
+
|
117
|
+
# Make sure the Puppetcode install directory exists
|
118
|
+
unless Dir.exists?($config["agent"]["puppetcode"])
|
119
|
+
$logger.warn "The destination path of #{$config["agent"]["puppetcode"]} does not appear to exist or is not readable"
|
120
|
+
return 0
|
121
|
+
end
|
122
|
+
|
123
|
+
# Look for a manifest file in the directory and read the version from it.
|
124
|
+
if File.exists?($config["agent"]["puppetcode"] + "/manifest.pupistry.yaml")
|
125
|
+
manifest = YAML::load(File.open($config["agent"]["puppetcode"] + "/manifest.pupistry.yaml"))
|
126
|
+
|
127
|
+
return manifest['version']
|
128
|
+
else
|
129
|
+
$logger.warn "No current version installed"
|
130
|
+
return 0
|
131
|
+
end
|
132
|
+
end
|
133
|
+
|
134
|
+
|
114
135
|
def fetch_artifact
|
115
136
|
|
116
137
|
# Figure out which version to fetch (if not explicitly defined)
|
@@ -387,6 +408,7 @@ module Pupistry
|
|
387
408
|
# Clone unpacked contents to the installation directory
|
388
409
|
begin
|
389
410
|
FileUtils.cp_r $config["general"]["app_cache"] + "/artifacts/unpacked.#{@checksum}/puppetcode/.", $config["agent"]["puppetcode"]
|
411
|
+
FileUtils.cp $config["general"]["app_cache"] + "/artifacts/manifest.#{@checksum}.yaml", $config["agent"]["puppetcode"] + "/manifest.pupistry.yaml"
|
390
412
|
return true
|
391
413
|
rescue
|
392
414
|
$logger.fatal "An unexpected error occured when copying the unpacked artifact to #{$config["agent"]["puppetcode"]}"
|
@@ -395,7 +417,6 @@ module Pupistry
|
|
395
417
|
|
396
418
|
end
|
397
419
|
|
398
|
-
|
399
420
|
def clean_install
|
400
421
|
# Cleanup the destination installation directory before we unpack the artifact
|
401
422
|
# into it, otherwise long term we will end up with old deprecated files hanging
|
data/lib/pupistry/bootstrap.rb
CHANGED
@@ -1,4 +1,5 @@
|
|
1
1
|
require 'rubygems'
|
2
|
+
require 'erubis'
|
2
3
|
|
3
4
|
module Pupistry
|
4
5
|
# Pupistry::Bootstrap
|
@@ -6,16 +7,82 @@ module Pupistry
|
|
6
7
|
class Bootstrap
|
7
8
|
attr_accessor :template_dir
|
8
9
|
|
9
|
-
def
|
10
|
-
|
10
|
+
def initialize
|
11
|
+
|
12
|
+
# We need to find where the templates are located - either it should be
|
13
|
+
# in the current working directory, or if we are an installed gem, we
|
14
|
+
# can try the gem's installed path.
|
15
|
+
|
16
|
+
if Dir.exists?("resources/bootstrap/")
|
17
|
+
# Use local PWD version first if possible
|
18
|
+
@template_dir = Dir.pwd
|
19
|
+
else
|
20
|
+
# Check for GEM installed location
|
21
|
+
begin
|
22
|
+
@template_dir = Gem::Specification.find_by_name("pupistry").gem_dir
|
23
|
+
rescue Gem::LoadError
|
24
|
+
$logger.error "Unable to find templates/ directory, doesn't appear we are running from project dir nor as a Gem"
|
25
|
+
return false
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
@template_dir = @template_dir.chomp("/") + "/resources/bootstrap/"
|
30
|
+
|
31
|
+
unless Dir.exists?(@template_dir)
|
32
|
+
$logger.error "Unable to find templates dir at #{@template_dir}, unable to proceed."
|
33
|
+
return false
|
34
|
+
else
|
35
|
+
$logger.debug "Using directory #{@template_dir} for bootstrap templates"
|
36
|
+
end
|
37
|
+
|
11
38
|
end
|
12
39
|
|
13
|
-
|
14
|
-
|
15
|
-
|
40
|
+
|
41
|
+
def list
|
42
|
+
# Simply glob the templates directory and list their names.
|
43
|
+
$logger.debug "Finding all available templates"
|
44
|
+
|
45
|
+
Dir.glob("#{@template_dir}/*.erb").each do |file|
|
46
|
+
puts "- #{File.basename(file, ".erb")}"
|
47
|
+
end
|
16
48
|
end
|
17
49
|
|
18
50
|
|
51
|
+
def build template
|
52
|
+
# Build a template with the configured parameters already to go
|
53
|
+
$logger.debug "Generating a bootstrap script for #{template}"
|
54
|
+
|
55
|
+
unless File.exists?("#{@template_dir}/#{template}.erb")
|
56
|
+
$logger.error "The requested template does not exist, unable to build"
|
57
|
+
return 0
|
58
|
+
end
|
59
|
+
|
60
|
+
# Assume values we care about
|
61
|
+
template_values = {
|
62
|
+
s3_bucket: $config["general"]["s3_bucket"],
|
63
|
+
s3_prefix: $config["general"]["s3_prefix"],
|
64
|
+
gpg_disable: $config["general"]["gpg_disable"],
|
65
|
+
gpg_signing_key: $config["general"]["gpg_signing_key"],
|
66
|
+
puppetcode: $config["agent"]["puppetcode"],
|
67
|
+
access_key_id: $config["agent"]["access_key_id"],
|
68
|
+
secret_access_key: $config["agent"]["secret_access_key"],
|
69
|
+
region: $config["agent"]["region"],
|
70
|
+
proxy_uri: $config["agent"]["proxy_uri"],
|
71
|
+
}
|
72
|
+
|
73
|
+
# Generate template using ERB
|
74
|
+
begin
|
75
|
+
template_contents = Erubis::Eruby.new(File.read("#{@template_dir}/#{template}.erb")).result(template_values)
|
76
|
+
rescue Exception => e
|
77
|
+
$logger.error "An unexpected error occured when trying to generate the bootstrap template"
|
78
|
+
raise e
|
79
|
+
end
|
80
|
+
|
81
|
+
# Output template
|
82
|
+
puts "-- Bootstrap Start --"
|
83
|
+
puts template_contents
|
84
|
+
puts "-- Bootstrap End --"
|
85
|
+
end
|
19
86
|
end
|
20
87
|
end
|
21
88
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pupistry
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.4
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Jethro Carr
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-04-
|
11
|
+
date: 2015-04-14 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: aws-sdk-v1
|
@@ -38,6 +38,20 @@ dependencies:
|
|
38
38
|
- - '>='
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: erubis
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - '>='
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - '>='
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
41
55
|
- !ruby/object:Gem::Dependency
|
42
56
|
name: r10k
|
43
57
|
requirement: !ruby/object:Gem::Requirement
|