packaging 0.88.77

Sign up to get free protection for your applications and to get access to all the features.
Files changed (123) hide show
  1. checksums.yaml +7 -0
  2. data/LICENSE +17 -0
  3. data/README-Solaris.md +117 -0
  4. data/README.md +977 -0
  5. data/lib/packaging.rb +32 -0
  6. data/lib/packaging/archive.rb +126 -0
  7. data/lib/packaging/artifactory.rb +651 -0
  8. data/lib/packaging/artifactory/extensions.rb +94 -0
  9. data/lib/packaging/config.rb +492 -0
  10. data/lib/packaging/config/params.rb +387 -0
  11. data/lib/packaging/config/validations.rb +13 -0
  12. data/lib/packaging/deb.rb +28 -0
  13. data/lib/packaging/deb/repo.rb +264 -0
  14. data/lib/packaging/gem.rb +70 -0
  15. data/lib/packaging/metrics.rb +15 -0
  16. data/lib/packaging/nuget.rb +39 -0
  17. data/lib/packaging/paths.rb +376 -0
  18. data/lib/packaging/platforms.rb +507 -0
  19. data/lib/packaging/repo.rb +155 -0
  20. data/lib/packaging/retrieve.rb +75 -0
  21. data/lib/packaging/rpm.rb +5 -0
  22. data/lib/packaging/rpm/repo.rb +254 -0
  23. data/lib/packaging/sign.rb +8 -0
  24. data/lib/packaging/sign/deb.rb +9 -0
  25. data/lib/packaging/sign/dmg.rb +41 -0
  26. data/lib/packaging/sign/ips.rb +57 -0
  27. data/lib/packaging/sign/msi.rb +124 -0
  28. data/lib/packaging/sign/rpm.rb +115 -0
  29. data/lib/packaging/tar.rb +163 -0
  30. data/lib/packaging/util.rb +146 -0
  31. data/lib/packaging/util/date.rb +20 -0
  32. data/lib/packaging/util/execution.rb +85 -0
  33. data/lib/packaging/util/file.rb +125 -0
  34. data/lib/packaging/util/git.rb +174 -0
  35. data/lib/packaging/util/git_tags.rb +73 -0
  36. data/lib/packaging/util/gpg.rb +66 -0
  37. data/lib/packaging/util/jenkins.rb +95 -0
  38. data/lib/packaging/util/misc.rb +69 -0
  39. data/lib/packaging/util/net.rb +410 -0
  40. data/lib/packaging/util/os.rb +17 -0
  41. data/lib/packaging/util/platform.rb +40 -0
  42. data/lib/packaging/util/rake_utils.rb +112 -0
  43. data/lib/packaging/util/serialization.rb +19 -0
  44. data/lib/packaging/util/ship.rb +300 -0
  45. data/lib/packaging/util/tool.rb +41 -0
  46. data/lib/packaging/util/version.rb +334 -0
  47. data/spec/fixtures/config/ext/build_defaults.yaml +2 -0
  48. data/spec/fixtures/config/ext/project_data.yaml +2 -0
  49. data/spec/fixtures/configs/components/test_file.json +1 -0
  50. data/spec/fixtures/configs/components/test_file_2.json +0 -0
  51. data/spec/fixtures/configs/components/test_file_not_tagged.json +1 -0
  52. data/spec/fixtures/configs/components/test_file_wrong_ext.txt +0 -0
  53. data/spec/fixtures/configs/components/test_file_wrong_ext.wrong +0 -0
  54. data/spec/fixtures/util/pre_tasks.yaml +4 -0
  55. data/spec/lib/packaging/artifactory_spec.rb +221 -0
  56. data/spec/lib/packaging/config_spec.rb +576 -0
  57. data/spec/lib/packaging/deb/repo_spec.rb +157 -0
  58. data/spec/lib/packaging/deb_spec.rb +52 -0
  59. data/spec/lib/packaging/gem_spec.rb +86 -0
  60. data/spec/lib/packaging/paths_spec.rb +418 -0
  61. data/spec/lib/packaging/platforms_spec.rb +178 -0
  62. data/spec/lib/packaging/repo_spec.rb +135 -0
  63. data/spec/lib/packaging/retrieve_spec.rb +100 -0
  64. data/spec/lib/packaging/rpm/repo_spec.rb +133 -0
  65. data/spec/lib/packaging/sign_spec.rb +133 -0
  66. data/spec/lib/packaging/tar_spec.rb +116 -0
  67. data/spec/lib/packaging/util/execution_spec.rb +56 -0
  68. data/spec/lib/packaging/util/file_spec.rb +139 -0
  69. data/spec/lib/packaging/util/git_spec.rb +160 -0
  70. data/spec/lib/packaging/util/git_tag_spec.rb +36 -0
  71. data/spec/lib/packaging/util/gpg_spec.rb +64 -0
  72. data/spec/lib/packaging/util/jenkins_spec.rb +112 -0
  73. data/spec/lib/packaging/util/misc_spec.rb +31 -0
  74. data/spec/lib/packaging/util/net_spec.rb +259 -0
  75. data/spec/lib/packaging/util/os_spec.rb +31 -0
  76. data/spec/lib/packaging/util/rake_utils_spec.rb +70 -0
  77. data/spec/lib/packaging/util/ship_spec.rb +199 -0
  78. data/spec/lib/packaging/util/version_spec.rb +123 -0
  79. data/spec/lib/packaging_spec.rb +19 -0
  80. data/spec/spec_helper.rb +22 -0
  81. data/static_artifacts/PackageInfo.plist +3 -0
  82. data/tasks/00_utils.rake +214 -0
  83. data/tasks/30_metrics.rake +33 -0
  84. data/tasks/apple.rake +268 -0
  85. data/tasks/archive.rake +69 -0
  86. data/tasks/build.rake +12 -0
  87. data/tasks/clean.rake +5 -0
  88. data/tasks/config.rake +35 -0
  89. data/tasks/deb.rake +129 -0
  90. data/tasks/deb_repos.rake +28 -0
  91. data/tasks/deprecated.rake +130 -0
  92. data/tasks/doc.rake +20 -0
  93. data/tasks/education.rake +57 -0
  94. data/tasks/fetch.rake +60 -0
  95. data/tasks/gem.rake +159 -0
  96. data/tasks/jenkins.rake +538 -0
  97. data/tasks/jenkins_dynamic.rake +202 -0
  98. data/tasks/load_extras.rake +21 -0
  99. data/tasks/mock.rake +348 -0
  100. data/tasks/nightly_repos.rake +286 -0
  101. data/tasks/pe_deb.rake +12 -0
  102. data/tasks/pe_rpm.rake +13 -0
  103. data/tasks/pe_ship.rake +226 -0
  104. data/tasks/pe_sign.rake +13 -0
  105. data/tasks/pe_tar.rake +5 -0
  106. data/tasks/retrieve.rake +52 -0
  107. data/tasks/rpm.rake +66 -0
  108. data/tasks/rpm_repos.rake +29 -0
  109. data/tasks/ship.rake +692 -0
  110. data/tasks/sign.rake +154 -0
  111. data/tasks/tag.rake +8 -0
  112. data/tasks/tar.rake +28 -0
  113. data/tasks/update.rake +16 -0
  114. data/tasks/vanagon.rake +35 -0
  115. data/tasks/vendor_gems.rake +117 -0
  116. data/tasks/version.rake +33 -0
  117. data/tasks/z_data_dump.rake +65 -0
  118. data/templates/README +1 -0
  119. data/templates/downstream.xml.erb +47 -0
  120. data/templates/msi.xml.erb +197 -0
  121. data/templates/packaging.xml.erb +346 -0
  122. data/templates/repo.xml.erb +117 -0
  123. metadata +287 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: ae79e9e05a45482476a6cdac7888cdf1aca7bbd6f80ef9bd5502a0f143512c84
4
+ data.tar.gz: e6534c76f98302001bdf1d5c8c2e8f5e343ee6cc05a5c857e77995c4a4ed2711
5
+ SHA512:
6
+ metadata.gz: 9fa1398a9659ff420d3c2b32cf74de533acff812748531ecf1ca1c01fb1b97039a72932a54656f86749720c0986df941f39beaf43c7820656249331328a1fe69
7
+ data.tar.gz: ccc856e8484a179b903857c7a241eb5dd33888bfc4ba975bb5497484c02dc435d88e694aa9c3880eda19f8be4e4d4465d0c301ca1f1965c737e696c2f2f93bf3
data/LICENSE ADDED
@@ -0,0 +1,17 @@
1
+ packaging - Puppet Labs packaging framework
2
+
3
+ Copyright (C) 2011-2013 Puppet Labs Inc
4
+
5
+ Puppet Labs can be contacted at: info@puppetlabs.com
6
+
7
+ Licensed under the Apache License, Version 2.0 (the "License");
8
+ you may not use this file except in compliance with the License.
9
+ You may obtain a copy of the License at
10
+
11
+ http://www.apache.org/licenses/LICENSE-2.0
12
+
13
+ Unless required by applicable law or agreed to in writing, software
14
+ distributed under the License is distributed on an "AS IS" BASIS,
15
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16
+ See the License for the specific language governing permissions and
17
+ limitations under the License.
data/README-Solaris.md ADDED
@@ -0,0 +1,117 @@
1
+ ## Create self signed certificate
2
+
3
+ - Generate the key pair
4
+
5
+ | keytool -genkey -keyalg RSA -alias selfsigned -keystore keystore.jks -storepass password -validity 360 -keysize 2048
6
+
7
+
8
+ - Extract Certificate
9
+
10
+ | keytool -export -rfc -keystore keystore.jks -storepass password -alias selfsigned -file mycert.pem
11
+
12
+ - Extract Cert Key to PEM format
13
+
14
+ | keytool -importkeystore -srcstoretype JKS -srckeystore keystore.jks -deststoretype PKCS12 -destkeystore mykey.der
15
+ | openssl pkcs12 -in mykey.der -nodes -out mykey.pem
16
+
17
+ ## For installation.
18
+
19
+ - Check the publishers
20
+
21
+ | pkg publisher
22
+ solaris origin online http://pkg.oracle.com/solaris/release/
23
+
24
+ - Add puppetlabs.com as a publisher
25
+
26
+ | pkg set-publisher -p http://solaris-11-ips-repo.acctest.dc1.puppetlabs.net puppetlabs.com
27
+
28
+ - Check that we have it correct
29
+
30
+ | pkg publisher
31
+ solaris origin online http://pkg.oracle.com/solaris/release/
32
+ puppetlabs.com origin online http://solaris-11-ips-repo.acctest.dc1.puppetlabs.net/
33
+
34
+ - Verify that puppetlabs.com was set correctly,
35
+
36
+ | pkg publisher puppetlabs.com
37
+ Publisher: puppetlabs.com
38
+ Alias:
39
+ Origin URI: http://solaris-11-ips-repo.acctest.dc1.puppetlabs.net/
40
+ SSL Key: None
41
+ SSL Cert: None
42
+ Client UUID: 37084c4a-fdc6-11e1-832b-8800273bd610
43
+ Catalog Updated: September 13, 2012 08:31:25 PM
44
+ Enabled: Yes
45
+ Signature Policy: verify
46
+
47
+ - Try installing puppet
48
+
49
+ | pkg install puppet
50
+ Creating Plan |
51
+ pkg install: Chain was rooted in an untrusted self-signed certificate.
52
+ The package involved is:pkg://puppetlabs.com/system/management/puppet@3.0.0,5.11-6.192:20120913T212942Z
53
+
54
+ - We have two choices here, first, to ignore the signature policy for both publisher and image
55
+
56
+ | pkg set-publisher --set-property=signature-policy=ignore puppetlabs.com
57
+ | pkg publisher puppetlabs.com
58
+ Publisher: puppetlabs.com
59
+ Alias:
60
+ Origin URI: http://solaris-11-ips-repo.acctest.dc1.puppetlabs.net/
61
+ SSL Key: None
62
+ SSL Cert: None
63
+ Client UUID: 140bb5c8-fe62-11e1-af70-8800273bd610
64
+ Catalog Updated: September 14, 2012 05:11:45 PM
65
+ Enabled: Yes
66
+ Signature Policy: ignore
67
+ | pkg set-property signature-policy ignore
68
+ | pkg install puppet
69
+
70
+ - Or trust the self signed certificate from puppetlabs.com (this certificate needs to be published in our web site)
71
+
72
+ | pkg set-publisher --approve-ca-cert /root/mycert.pem puppetlabs.com
73
+
74
+ - Verify that approve cert went well.
75
+
76
+ | pkg publisher puppetlabs.com
77
+
78
+ Publisher: puppetlabs.com
79
+ Alias:
80
+ Origin URI: http://solaris-11-ips-repo.acctest.dc1.puppetlabs.net/
81
+ SSL Key: None
82
+ SSL Cert: None
83
+ Client UUID: 37084c4a-fdc6-11e1-832b-8800273bd610
84
+ Catalog Updated: September 13, 2012 08:31:25 PM
85
+ Approved CAs: 791b5791a81e9c2eb3fb9f84f4f86f8ea6fcd672
86
+ Enabled: Yes
87
+ Signature Policy: verify
88
+
89
+ | pkg install puppet
90
+
91
+ - Checking about information on the package (See the human readable version string)
92
+
93
+ | pkg info -r puppet
94
+ Name: system/management/puppet
95
+ Summary: Puppet, an automated configuration management tool
96
+ Description: Puppet, an automated configuration management tool
97
+ Category: System/Administration and Configuration
98
+ State: Not installed
99
+ Publisher: puppetlabs.com
100
+ Version: 3.0.0 (3.0.0-rc6)
101
+ Build Release: 5.11
102
+ Branch: 6.170
103
+ Packaging Date: September 14, 2012 06:11:05 PM
104
+ Size: 2.43 MB
105
+ FMRI: pkg://puppetlabs.com/system/management/puppet@3.0.0,5.11-6.170:20120914T181105Z
106
+
107
+ - Check license
108
+
109
+ | pkg info -r --license puppet
110
+ Puppet - Automating Configuration Management.
111
+
112
+ Copyright (C) 2005-2012 Puppet Labs Inc
113
+
114
+
115
+ - Reference
116
+
117
+ http://docs.oracle.com/cd/E19963-01/html/820-6572/managepkgs.html
data/README.md ADDED
@@ -0,0 +1,977 @@
1
+ # Packaging
2
+
3
+ [![Build Status](https://travis-ci.org/puppetlabs/packaging.png?branch=master)](https://travis-ci.org/puppetlabs/packaging)
4
+
5
+ This is a repository for packaging automation for Puppet software.
6
+ The goal is to abstract and automate packaging processes beyond individual
7
+ software projects to a level where this repo can be cloned inside any project
8
+ and used to build Debian and Redhat packages, as well as gems, apple packages
9
+ and tarballs. This repo is currently under heavy development and in a state
10
+ flux, and it should not be considered to have a formal API. However, every
11
+ effort is being made to ensure existing tasks/behavior are not broken as we
12
+ continue to iterate and improve upon it.
13
+
14
+ ## Getting Started
15
+
16
+ As a developer, you can get started working on this by typing:
17
+
18
+ ```
19
+ bundle install
20
+ bundle exec rspec
21
+ ```
22
+
23
+ ## Using the Packaging Repo
24
+
25
+ Several Puppet projects are using the packaging repo. They are:
26
+
27
+ * puppet
28
+ * facter
29
+ * puppet-dashboard
30
+ * hiera
31
+ * puppetdb
32
+ * razor
33
+
34
+ as well as several closed-source projects, including
35
+ * live-management
36
+ * console-auth
37
+ * console
38
+
39
+ Generally speaking, the packaging repo should be compatible with ruby >= 2.0.0
40
+ rake ~> 12.3. To pull the packaging tasks into your source repo,
41
+ do a `bundle install`. This will install the packaging gem and all its
42
+ dependencies. Packaging's rake tasks will be made available by running any rake
43
+ command, e.g. `bundle exec rake -T`. The tasks are generally grouped into two
44
+ categories, `package:` namespaced tasks and `pl:` namespaced tasks.
45
+
46
+ ## `package:` tasks
47
+ `package:` namespaced tasks are general purpose tasks that are set up to use
48
+ the most minimal tool chain possible for creating packages. These tasks will
49
+ create rpms and debs, but any build dependencies will need to be satisifed by
50
+ the building host, and any dynamically generated dependencies may result in
51
+ packages that are only suitable for the OS/version of the build host. However,
52
+ for rolling one's own debs and rpms or for use in environments without many
53
+ OSes/versions, this may work just fine. To build an rpm using the packaging
54
+ repo, do a `bundle exec rake package:rpm`. To build a deb, use `bundle exec rake
55
+ package:deb`.
56
+
57
+ ## `pl:` tasks
58
+ `pl:` namespaced tasks rely on a slighly more complex toolchain for packaging
59
+ inside clean chroot environments for the various operating systems and versions
60
+ that Puppet supports. On the rpm side, this is done with
61
+ [mock](http://fedoraproject.org/wiki/Projects/Mock) and for debs, we use
62
+ pdebuild and [cowbuilder](http://wiki.debian.org/cowbuilder). For the most
63
+ part, these tasks are keyed to puppet infrastructure, and are used by the
64
+ Release Engineering team to create release packages. However, they can
65
+ certainly be modified to suit other environments, and much effort went into
66
+ making tasks as modular and reusable as possible. Several Puppet-specific
67
+ tasks are only available if the file '~/.packaging' is present. This file is
68
+ created by the `pl:fetch` task, which curls two yaml files into 'team' and
69
+ 'project' subdirectories from a [separate build data
70
+ repository](https://github.com/puppetlabs/build-data), which contains
71
+ additional settings/data specific to Puppet release infrastructure. By
72
+ default, the team data file is pulled from the 'dev' branch of the repo, and
73
+ the project data file is pulled from a branch named after the project (e.g. for
74
+ puppet, there is a branch named puppet with a build data file). The goal in
75
+ separating these data and tasks out is to refrain from presenting by
76
+ default yet more Puppet-specific tasks that aren't generally consumable by
77
+ everyone. To build a deb from a local repository using a `pl` task, ssh into a
78
+ builder (e.g., one stood up using the modules detailed below) and clone the
79
+ source repo, e.g. puppet. Then, run `bundle install` and `bundle exec rake pl:deb`
80
+ to create a deb, and `bundle exec rake pl:mock` to make an rpm (on a debian or
81
+ redhat host, respectively).
82
+
83
+ ## `pe:` tasks
84
+ There is also a `pe:` namespace, for the building of Puppet Labs' Puppet
85
+ Enterprise packages that have been converted to using this repo. The `pe:` tasks
86
+ rely heavily on PL internal infrastructure, and are not generally useful outside
87
+ of this environment. To create packages, in the source repository run `bundle
88
+ install`, followed by `bundle exec rake pl:fetch`. These two commands install
89
+ the packaging gem and pull in the additional data needed for PE building (see
90
+ `pl:fetch` notes above). Then, to make a debian package, run `bundle exec rake
91
+ pe:deb`, and to make an rpm, run `bundle exec rake pe:mock`. There are also
92
+ `pe:deb_all` and `pe:mock_all` tasks, which build packages against all shipped
93
+ debian/redhat targets. The `pe:deb_all` task is not generally necessary for
94
+ developer use for building test packages; the `pe:deb` task creates a package
95
+ that will work against virtually all supported PE debian versions. The same is
96
+ generally true for PE internal rpms, but because of variances in build macros
97
+ for rpm, rpms should generally be built with `pe:mock_all`, and then the desired
98
+ version installed, or by building only for a specific target. This is
99
+ accomplished by passing MOCK=<mock> to the rake call, e.g. `bundle exec rake
100
+ pe:mock MOCK=<mock>`. The available mocks are listed in
101
+ `ext/build_defaults.yaml` after `final_mocks:`. For PE, the mocks are formatted
102
+ as `pupent-<peversion>-<distversion>-<arch>`, e.g. `pupent-2.7-el5-i386`. To
103
+ build for a specific target, set `MOCK=<mock>` to the mock that matches the
104
+ target.
105
+
106
+ ## `:remote:` tasks
107
+ There are also sub-namespaces of `:pl` and `:pe` that are
108
+ worth noting. First, the `:remote` namespace. Tasks under `:remote` perform
109
+ builds remotely on internal builders from your local workstation. How they
110
+ work:
111
+
112
+ 0) Run `bundle install` to install the packaging gem and its dependencies.
113
+
114
+ 1) Run `bundle exec pl:fetch` to obtain extra data from the build-data repo. The data
115
+ includes the hostnames of builders to use for packaging.
116
+
117
+ 2) Create a git bundle of the local workspace and tar it up.
118
+
119
+ 3) Create a build parameters file. The params file includes all the information
120
+ about the build, including any values overridden with env vars, and the actual
121
+ task to run, e.g. `bundle exec rake pl:deb`.
122
+
123
+ 4) scp the git bundle and build parameters file to a temporary directory on the
124
+ builder hostname assigned to that particular package build type.
125
+
126
+ 5) ssh into the builder, untar the git bundle, clone it, and run `bundle
127
+ install`.
128
+
129
+ 6) ssh into the builder, cd into the cloned repo, and run `bundle exec rake
130
+ pl:build_from_params PARAMS_FILE=/path/to/previously/sent/file`.
131
+
132
+ 7) Maintain the ssh connection until the build finishes, and rsync the packages
133
+ from the builder to the local workstation.
134
+
135
+ Note that these tasks require ssh access to the builder hosts that are
136
+ specified in the build-data file, and appropriate membership in the build
137
+ groups, e.g. to use mock on the builder, membership in the mock group. This is
138
+ a major hurdle, and is resolved with the `jenkins` tasks below.
139
+
140
+ ## legacy `:jenkins:` workflow tasks
141
+ (Deprecated - see "dynamic jenkins task workflow" below)
142
+ Jenkins tasks are similar to the `:remote:` tasks, but they do not require ssh
143
+ access to the builders. They do require being on the local network - the
144
+ jenkins instance that performs package builds is an internal server only,
145
+ accessible when connected via VPN or on-site. The jenkins tasks enable the
146
+ packaging repo to kick off packaging builds on a remote jenkins slave.
147
+ There are two workflows of jenkins tasks in the packaging repo. The first
148
+ workflow, which is used for creating individual platform packages on jenkins
149
+ (e.g. creating a deb with pl:jenkins:deb) relies on a job that exists on a
150
+ remote jenkins server. The tasks transmit information to the jenkins job, which
151
+ handles the rest. The data passed are the following:
152
+
153
+ 1) $PROJECT\_BUNDLE - a tar.gz of a git-bundle from HEAD of the current
154
+ project, which is cloned on the builder to set up a duplicate of this
155
+ environment
156
+
157
+ 2) $BUILD\_PROPERTIES - a build parameters file, containing all information
158
+ about the build
159
+
160
+ 3) $BUILD\_TYPE - the "type" of build, e.g. rpm, deb, gem, etc The jenkins url
161
+ and job name are obtained via the team build-data file from [the build data
162
+ repository](https://github.com/puppetlabs/build-data)
163
+
164
+ 4) $PROJECT - the project we're building, e.g. facter, puppet. This is used
165
+ later in determining the target for the build artifacts on the distribution
166
+ server
167
+
168
+ 4) $METRICS - a string of data points related to the build which are used for
169
+ data analysis. Contents of this string are items which cannot be obtained from
170
+ within the Jenkins job itself.
171
+
172
+ 5) $DOWNSTREAM\_JOB - The URL of a downstream job that jenkins should post to
173
+ upon success. This is obtained via the DOWNSTREAM\_JOB environment variable.
174
+
175
+
176
+ On the Jenkins end, the job is a parameterized job that accepts five
177
+ parameters. Jenkins has the Parameterized Trigger Plugin, Workspace Cleanup
178
+ Plugin, and Node and Label Parameter Plugin in use for this job. The workspace
179
+ cleanup plugin cleans the workspace before each build. Two are file parameters,
180
+ two string parameters, and a Label parameter provided by the Node and Label
181
+ Parameter Plugin, as described above. When the pl:jenkins:\* task triggers a
182
+ build, it passes values for all of these parameters. The Label parameter is
183
+ associated with the build type. This way we can queue the job on a builder with
184
+ the appropriate capabilities just by assigning a builder the label "deb" or
185
+ "rpm," etc. The job allows parallel execution of jobs - in this way, we can
186
+ queue many package jobs on the jenkins instance, which will farm them out to
187
+ builders that are slaves of that jenkins instance. This also allows us to scale
188
+ building capacity simply by adding builders as slaves to the jenkins instance.
189
+ The actual build itself is accomplished via a shell build task. The contents of
190
+ the task are:
191
+
192
+ ```bash
193
+ #################
194
+
195
+ SHA=$(echo $BUILD_PROPERTIES | cut -d '.' -f1)
196
+
197
+ echo "Build type: $BUILD_TYPE"
198
+
199
+ ### Create a local clone of the git-bundle that was passed
200
+ # The bundle is a tarball, and since this is a project-agnostic
201
+ # job, we don't actually know what's in it, just that it's a
202
+ # git bundle.
203
+
204
+ [ -f "PROJECT_BUNDLE" ] || exit 1
205
+ mkdir project && tar -xzf PROJECT_BUNDLE -C project/
206
+
207
+ cd project
208
+ git clone --recursive $(ls) git_repo
209
+
210
+ cd git_repo
211
+
212
+ ### Install the packaging gem via Bundler
213
+ bundle install && bundle exec rake pl:fetch
214
+
215
+ ### Perform the build
216
+ bundle exec rake pl:load_extras pl:build_from_params PARAMS_FILE=$WORKSPACE/BUILD_PROPERTIES
217
+
218
+ ### Send the results
219
+ bundle exec rake pl:jenkins:ship["artifacts"]
220
+
221
+ ### If a downstream job was passed, trigger it now
222
+ if [ -n "$DOWNSTREAM_JOB" ] ; then
223
+ bundle exec rake pl:jenkins:post["$DOWNSTREAM_JOB"]
224
+ fi
225
+
226
+ #################
227
+ ```
228
+
229
+ To gather metrics related to a Jenkins build, the Groovy Postbuild plugin is used.
230
+ For tasks carried out on the static Jenkins job, the script must be manually added to the job's
231
+ configuration. The script in its entirety can be seen [here.](https://github.com/Whopper92/buildboard#groovyScript)
232
+
233
+ ## dynamic `:jenkins:` task workflow
234
+
235
+ The recommended and far simpler jenkins-based workflow is for initiating the
236
+ "uber_build", or a package build for all of our target platforms.
237
+
238
+ The uber_build is invoked as "pl:jenkins:uber_build" or "pe:jenkins:uber_build"
239
+ depending on if this is a FOSS or PE package.
240
+
241
+ This workflow doesn't actually use a static job on the jenkins-server. Instead
242
+ it _creates_ the jenkins jobs for you, on-demand. Specifically, it creates two
243
+ jenkins-jobs, and can create an optional third.
244
+
245
+ The first job is a matrix job, the cells of which are individual package tasks
246
+ for all of the build targets. This job takes four parameters:
247
+
248
+ 1) $PROJECT\_BUNDLE - a tar.gz of a git-bundle from HEAD of the current
249
+ project, which is cloned on the builder to set up a duplicate of this
250
+ environment
251
+
252
+ 2) $BUILD\_PROPERTIES - a build parameters file, containing all information
253
+ about the build
254
+
255
+ 3) $PROJECT - the project we're building, e.g. facter, puppet. This is used
256
+ later in determining the target for the build artifacts on the distribution
257
+ server
258
+
259
+ 4) $METRICS - a string of data points related to the build which are used for data
260
+ analysis. Contents of this string are items which cannot be obtained from within
261
+ the Jenkins job itself. Note that the Groovy postbuild script needed for metrics
262
+ gathering is dynamically passed to each job.
263
+
264
+ This first job clones the git bundle passed in as a parameter, then installs the
265
+ packaging gem (bundle install) and for every cell in its matrix performs a
266
+ package build for a specific target (e.g. bundle exec rake pl:deb
267
+ COW=base-lucid-i386.cow). Once all cells in the matrix complete (either succeed
268
+ or fail), this job automatically triggers the second of the new jobs as a
269
+ downstream job.
270
+
271
+ To receive an email notification from jenkins about the status of the packaging
272
+ job, pass NOTIFY=<recipient> as an environment variable to the uber_build
273
+ invocation, e.g.:
274
+
275
+ bundle exec rake pl:jenkins:uber_build NOTIFY="foo@puppet.com bar@puppet.com"
276
+
277
+ The second job is an automatic repository creation task for this git repo.
278
+ Specifically, the job copies the git bundle from the packaging job and clones
279
+ it, and uses the git information in the git bundle to clone the packaging repo
280
+ and invoke the repository creation jobs `pl:jenkins:rpm_repos` and
281
+ `pl:jenkins:deb_repos`. The job will always be invoked, but will only actually
282
+ create repos if the upstream packaging job actually succeeded.
283
+
284
+ The third job is only created _if_ the environment variable
285
+ `DOWNSTREAM_JOB=<job_url>` was passed to the initial "pl:jenkins:uber_build"
286
+ invocation. This third job takes the value assigned to `DOWNSTREAM_JOB` and
287
+ creates a proxy jenkins job with a single build step, a curl call to this
288
+ value, presumably a url to a jenkins job to trigger programmatically.
289
+
290
+ An important note about `DOWNSTREAM_JOB`: `DOWNSTREAM_JOB` in the dynamic jenkins
291
+ workflow is _always_ invoked if it is passed in as an environment variable.
292
+ However, it is appended with an additional parameter, `PACKAGE_BUILD_STATUS`,
293
+ which will be the string "success" if package and repo builds succeeded, or
294
+ "failure" if package or repo builds failed. By modifying the actual downstream
295
+ jenkins job to accept a string parameter of `PACKAGE_BUILD_STATUS`, one can
296
+ switch on the success or failure of the packaging job, responding
297
+ appropriately. A second parameter, `PACKAGE_BUILD_URL` is also appended to
298
+ `DOWNSTREAM_JOB`, the value of which is the url of the packaging job itself.
299
+ This is to assist with tracing failures in a multi-jenkins environment. By
300
+ modifying the downstream jenkins job to accept a string parameter of
301
+ `PACKAGE_BUILD_URL`, one can use the value to display the url prominently in
302
+ case of failure, for example.
303
+
304
+ E.g., a job url:
305
+ http://jenkins.example.net/job/downstream/buildWithParameters?FOO=bar
306
+
307
+ in the success case will be transformed into
308
+
309
+ http://jenkins.example.net/job/downstream/buildWithParameters?FOO=bar&PACKAGE_BUILD_STATUS=success&PACKAGE_BUILD_URL=http://jenkins.example.net/job/packaging_job
310
+
311
+ and in the failure case transformed into
312
+
313
+ http://jenkins.example.net/job/downstream/buildWithParameters?FOO=bar&PACKAGE_BUILD_STATUS=failure&PACKAGE_BUILD_URL=http://jenkins.example.net/job/packaging_job
314
+
315
+ Since jenkins will just drop parameters that are not configured in the job,
316
+ accepting PACKAGE_BUILD_STATUS and PACKAGE_BUILD_URL in the downstream job
317
+ isn't mandatory.
318
+
319
+ All 3 jobs are configured by default for removal by jenkins after 3 days, to
320
+ avoid clutter.
321
+
322
+ The goal is to move toward migrating all of the jenkins tasks from the first
323
+ workflow, using a static job that is called many times per package, to this
324
+ second workflow of creating the jobs on demand.
325
+
326
+ ### Polling behavior with `pl:jenkins:uber_build:`
327
+
328
+ It is possible to pass an integer argument to the `uber_build` task which will be
329
+ used by the job as polling interval. If this value is passed in, the task will
330
+ query the dynamic job using the Jenkins API periodically until the build is
331
+ finished. Then, it will query the build to determine the SUCCESS/FAILURE status.
332
+ The job output will look something like:
333
+
334
+ ```bash
335
+ Packaging SUCCESS
336
+ Repo SUCCESS
337
+ ```
338
+
339
+ If for example the Packaging job had failed the output would look like this:
340
+
341
+ ```bash
342
+ Packaging FAILURE
343
+ ```
344
+
345
+ If a build fails then the rake task will terminate with a nonzero exit status
346
+ which can be used during CI or other automated contexts to detect and act on the
347
+ failure. This eliminates the need to pass a `DOWNSTREAM_JOB` variable to the
348
+ uber_build job although it is still possible to do so.
349
+
350
+ ## Task Explanations
351
+ For a listing of all available tasks and their functions, see the [Task
352
+ Dictionary](https://github.com/puppetlabs/packaging#task-dictionary)
353
+ at the end of this README.
354
+
355
+ ## Modules
356
+
357
+ A puppet module,
358
+ [puppetlabs-debbuilder](https://github.com/puppetlabs/puppetlabs-debbuilder),
359
+ has been created to stand up a debian build host compatible with the debian
360
+ side of this packaging repo. The rpm-side module,
361
+ [puppetlabs-rpmbuilder](https://github.com/puppetlabs/puppetlabs-rpmbuilder),
362
+ will set up an rpm builder.
363
+
364
+ ##Setting up projects for the Packaging Repo
365
+
366
+ The packaging repo requires many project-side artifacts inside the ext
367
+ directory at the top level. [facter](https://github.com:puppetlabs/facter) and
368
+ [hiera](https://github.com:puppetlabs/hiera) are good examples.
369
+ It expects the following directory structure in the project
370
+
371
+ * ext/{debian,redhat,osx}
372
+
373
+ each of which contains templated erb files using the instance variables
374
+ specified in the setupvars task. These include a debian changelog, a redhat
375
+ spec file, and an osx preflight and plist.
376
+
377
+ The top level Rakefile or a separate task file in the project should have the following added:
378
+
379
+ ```ruby
380
+ require 'packaging'
381
+ Pkg::Util::RakeUtils.load_packaging_tasks
382
+ ```
383
+
384
+ Also in ext should be two files, build_defaults.yaml and project_data.yaml (optional).
385
+
386
+ This is the sample build_defaults.yaml file from Hiera:
387
+ ```yaml
388
+ ---
389
+ packaging_url: 'git@github.com:puppetlabs/packaging --branch=master'
390
+ packaging_repo: 'packaging'
391
+ default_cow: 'base-squeeze-i386.cow'
392
+ # Which debian distributions to build for. Noarch packages only need one arch of each cow.
393
+ cows: 'base-lucid-amd64.cow base-lucid-i386.cow base-natty-amd64.cow base-natty-i386.cow base-oneiric-amd64.cow base-oneiric-i386.cow base-precise-amd64.cow base-precise-i386.cow base-sid-amd64.cow base-sid-i386.cow base-squeeze-amd64.cow base-squeeze-i386.cow base-testing-amd64.cow base-testing-i386.cow base-wheezy-i386.cow'
394
+ # The pbuilder configuration file to use
395
+ pbuild_conf: '/etc/pbuilderrc'
396
+ # Alternate debian mirrors to build against (must be an array)
397
+ # The __DIST__ string is automatically replaced with codename of the cow being built, so when the squeeze cow is being built the deb_build_mirrors will be:
398
+ # deb http://apt.puppetlabs.com squeeze main dependencies
399
+ # deb http://somethingelse.com/debian squeeze
400
+ # This will happen for each cow during the build.
401
+ deb_build_mirrors:
402
+ - deb http://apt.puppetlabs.com __DIST__ main dependencies
403
+ - deb http://somethingelse.com/debian __DIST__
404
+ # Who is packaging. Turns up in various packaging artifacts
405
+ packager: 'puppetlabs'
406
+ # GPG key ID of the signer
407
+ gpg_key: '4528B6CD9E61EF26'
408
+ # Whether to require tarball signing as a prerequisite of other package building
409
+ sign_tar: false
410
+ # a space separated list of mock configs. These are the rpm distributions to package for. If a noarch package, only one arch of each is needed.
411
+ final_mocks: 'pl-el-5-i386 pl-el-5-x86_64 pl-el-6-i386 pl-el-6-x86_64 pl-fedora-16-i386 pl-fedora-16-x86_64 pl-fedora-17-i386 pl-fedora-17-x86_64'
412
+ # The host that contains the yum repository to ship to
413
+ yum_host: 'yum.puppetlabs.com'
414
+ # The remote path the repository on the yum\_host
415
+ yum_repo_path: '/some/repo/'
416
+ # The host that contains the apt repository to ship to
417
+ apt_host: 'apt.puppetlabs.com'
418
+ # The URL to use for the apt dependencies in cow building
419
+ apt_repo_url: 'http://apt.puppetlabs.com'
420
+ # The path on the remote apt host that debs should ship to
421
+ apt_repo_path: '/opt/repository/incoming'
422
+ # The host that stores the tarballs for downloading
423
+ tar_host: 'downloads.puppetlabs.com'
424
+ # Whether to present the gem and apple tasks
425
+ build_gem: true
426
+ build_dmg: true
427
+ # Whether to execute the rdoc rake tasks prior to composing the tarball
428
+ build_doc: false
429
+ # Whether to kick of a dynamic msi build job along side the uber_build
430
+ # If present, a dynamically generated jenkins job will be kicked off.
431
+ # The automation in puppet_for_the_win is used to build the msi with the
432
+ # following components.
433
+ build_msi:
434
+ puppet_for_the_win:
435
+ ref: 'origin/master'
436
+ repo: 'git://github.com/puppetlabs/puppet_for_the_win.git'
437
+ facter:
438
+ ref: 'refs/tags/2.1.0'
439
+ repo: 'git://github.com/puppetlabs/facter.git'
440
+ hiera:
441
+ ref: 'refs/tags/1.3.4'
442
+ repo: 'git://github.com/puppetlabs/hiera.git'
443
+ sys:
444
+ ref:
445
+ x86: 'origin/1.9.3-x86'
446
+ x64: 'origin/2.0.0-x64'
447
+ repo: 'git://github.com/puppetlabs/puppet-win32-ruby.git'
448
+ # Whether to present the Solaris 11 IPS packaging tasks
449
+ # This requires suitable IPS packaging artifacts in the project in ext/ips
450
+ build_ips: false
451
+ # Whether this project is a PE project or not
452
+ build_pe: false
453
+ # An optional task to execute pre-tarball composition. See the tasks in
454
+ # the 'pretasks' directory
455
+ pre_tar_task: 'package:vendor_gems'
456
+ ```
457
+ This is the sample project_data.yaml file:
458
+ ```yaml
459
+ ---
460
+ project: 'hiera'
461
+ author: 'Puppet'
462
+ email: 'info@puppet.com'
463
+ homepage: 'https://github.com/puppetlabs/hiera'
464
+ summary: 'Light weight hierarchical data store'
465
+ description: 'A pluggable data store for hierarchical data'
466
+ # file containing hard coded version information, if present
467
+ version_file: '/lib/hiera.rb'
468
+ # A string indicating the version strategy for the project (one of 'odd_even' or 'rc_final'), defaults to rc_final
469
+ # odd_even is a final release when the minor version is even, and a development release when it is odd
470
+ # rc_final is a final release when there is no rc at the end of the version string, and a development release when there is
471
+ version_strategy: 'rc_final'
472
+ # Boolean value of whether or not to automatically update the version file before packaging (defaults to false)
473
+ update_version_file: true
474
+ # files and gem\_files are space separated lists
475
+ # files to be packaged into a tarball and released with deb/rpm
476
+ files: '[A-Z]* ext lib bin spec acceptance_tests'
477
+ # space separated list of files to *exclude* from the tarball
478
+ # note that each listing in files, above, is recursively copied into the tarball, so
479
+ # 'tar\_excludes' only needs to include any undesired subdirectories/files of the 'files'
480
+ # list to exclude
481
+ tar_excludes: 'ext/packaging lib/some_excluded_file'
482
+ # Array of templates or globs of templates to evaluate. Note that without this key, the packaging will
483
+ # default to searching for any files in `ext` with the extension '.erb' and evaluate them. When this
484
+ # key is supplied, its values override the defaults, and all desired erb files must be specified with a path or glob.
485
+ templates:
486
+ - ext/redhat/project.spec.erb
487
+ - ext/templates/**/*.erb
488
+ # files to be packaged into a gem
489
+ gem_files: '{bin,lib}/**/* CHANGELOG COPYING README.md LICENSE'
490
+ # To exclude specific files from inclusion in a gem:
491
+ gem_excludes: 'lib/hiera/file_to_exclude.rb lib/hiera/directory_to_exclude'
492
+ # If gem name differs in some way from project name, e.g. only build gem for a client end
493
+ gem_name: hiera_the_gem
494
+ # If gem summary and/or description differs from general summary
495
+ gem_summary: 'A sub-set of the Hiera pluggable data store'
496
+ gem_description: 'A gem of the pluggable data store for hierarchical data'
497
+ gem_require_path: 'lib'
498
+ gem_test_files: 'spec/**/*'
499
+ gem_executables: 'hiera'
500
+ gem_default_executables: 'hiera'
501
+ # To add gem dependencies, indent.
502
+ # This is an example only, hiera doesn't really depend on hiera-puppet/json/facter
503
+ # For no specific version, leave version empty
504
+ gem_runtime_dependencies:
505
+ hiera-puppet: '1.0.0rc'
506
+ hiera-json:
507
+ gem_development_dependencies:
508
+ facter: '>= 1.6.11'
509
+ # To add gem dependencies which only apply to a specific platform, add the key "gem_platform_dependencies".
510
+ # The first key under the gem_platform_dependencies has to be a value that
511
+ # corresponds to a value of RUBY_PLATFORM. The subkeys are the same as the
512
+ # top-level gem dependency keys:
513
+ gem_platform_dependencies:
514
+ x86-mingw32:
515
+ gem_runtime_dependencies:
516
+ win32process: '~> 0.6.5'
517
+ gem_development_dependencies:
518
+ rake: '~> 0.9.0'
519
+ x86_64-darwin:
520
+ gem_runtime_dependencies:
521
+ CFPropertyList: '~> 2.2.4'
522
+ # rdoc options as an array
523
+ gem_rdoc_options:
524
+ - --line-numbers
525
+ - --main
526
+ - Hiera.README
527
+ ```
528
+ For basic mac packaging, add an osx directory in ext containing the following files:
529
+ 1. a preflight.erb template for any pre-flight actions, perhaps removing the old package if present.
530
+ 2. a prototype.plist.erb that is templated into the pkginfo.plist file for the package. This is the one from puppet. Note that these variable names aren't mutable here, but there's no need to worry about their value assignment, it's done in the apple task:
531
+ ```
532
+ <?xml version="1.0" encoding="UTF-8"?>
533
+ <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
534
+ <plist version="1.0">
535
+ <dict>
536
+ <key>CFBundleIdentifier</key>
537
+ <string><%= @title %></string>
538
+ <key>CFBundleShortVersionString</key>
539
+ <string><%= @version %></string>
540
+ <key>IFMajorVersion</key>
541
+ <integer><%= @package_major_version %></integer>
542
+ <key>IFMinorVersion</key>
543
+ <integer><%= @package_minor_version %></integer>
544
+ <key>IFPkgBuildDate</key>
545
+ <date><%= @build_date %></date>
546
+ <key>IFPkgFlagAllowBackRev</key>
547
+ <false/>
548
+ <key>IFPkgFlagAuthorizationAction</key>
549
+ <string>RootAuthorization</string>
550
+ <key>IFPkgFlagDefaultLocation</key>
551
+ <string>/</string>
552
+ <key>IFPkgFlagFollowLinks</key>
553
+ <true/>
554
+ <key>IFPkgFlagInstallFat</key>
555
+ <false/>
556
+ <key>IFPkgFlagIsRequired</key>
557
+ <false/>
558
+ <key>IFPkgFlagOverwritePermissions</key>
559
+ <false/>
560
+ <key>IFPkgFlagRelocatable</key>
561
+ <false/>
562
+ <key>IFPkgFlagRestartAction</key>
563
+ <string><%= @pm_restart %></string>
564
+ <key>IFPkgFlagRootVolumeOnly</key>
565
+ <true/>
566
+ <key>IFPkgFlagUpdateInstalledLanguages</key>
567
+ <false/>
568
+ </dict>
569
+ </plist>
570
+ ```
571
+ A file_mapping.yaml file that specifies a set of files and a set of directories
572
+ from the source to install, with destinations, ownership, and permissions. The
573
+ directories are top level directories in the source to install. The files are
574
+ files somewhere in the source to install. This is the one from puppet 3.x:
575
+ ```yaml
576
+ ---
577
+ directories:
578
+ # this will take the contents of lib, e.g. puppet/lib/\* and place them in
579
+ # /usr/lib/ruby/site\_ruby/1.8
580
+ lib:
581
+ path: 'usr/lib/ruby/site_ruby/1.8'
582
+ owner: 'root'
583
+ group: 'wheel'
584
+ perms: '0644'
585
+ bin:
586
+ path: 'usr/bin'
587
+ owner: 'root'
588
+ group: 'wheel'
589
+ perms: '0755'
590
+ 'man/man8':
591
+ path: 'usr/share/man/man8'
592
+ owner: 'root'
593
+ group: 'wheel'
594
+ perms: '0755'
595
+ files:
596
+ # this will take the file puppet/conf/auth.conf and place it in
597
+ # /private/etc/puppet/, creating the directory if not present
598
+ 'conf/auth.conf':
599
+ path: 'private/etc/puppet'
600
+ owner: 'root'
601
+ group: 'wheel'
602
+ perms: '0644'
603
+ 'man/man5/puppet.conf.5':
604
+ path: 'usr/share/man/man5'
605
+ owner: 'root'
606
+ group: 'wheel'
607
+ perms: '0644'
608
+ '[A-Z]*':
609
+ path: 'usr/share/doc/puppet'
610
+ owner: 'root'
611
+ group: 'wheel'
612
+ perms: '0644'
613
+ ```
614
+
615
+ ## Task Dictionary
616
+
617
+ * **package:apple**
618
+
619
+ Use `PackageMaker` to create a pkg package inside a dmg. Requires 'sudo'
620
+ privileges. `build_dmg: true` must be set in `ext/build_defaults.yaml`.
621
+ Packages are staged in ./pkg/apple. See the Mac packaging section of
622
+ [Setting up projects for the Packaging
623
+ Repo](https://github.com/MosesMendoza/packaging/tree/more_documentation#setting-up-projects-for-the-packaging-repo).
624
+
625
+ * **package:deb**
626
+
627
+ Use `debbuild` to create a deb package and associated debian package
628
+ artifacts from the repository. Requires all build dependencies be satisfied
629
+ locally. Packages are staged in ./pkg/deb.
630
+
631
+ * **package:gem**
632
+ Use the `rubygems/package_task` library to create a rubygem from the
633
+ repository. Requires `build_gem: true` and gem-related parameters be set in
634
+ `ext/build_defaults.yaml` and `ext/project\_data.yaml`. The gem is staged
635
+ in `./pkg`.
636
+
637
+ * **package:ips**
638
+
639
+ Use Solaris 11 pkg* tools to create a IPS package from the project.
640
+ Packages are staged in `./pkg/ips/pkgs`. Requires all `pkg`, `pkgdepend`,
641
+ `pkgsend`, `pkglint`, and `pkgmogrify`. Currently only puppet, facter, and
642
+ hiera have this capability.
643
+
644
+ * **package:rpm**
645
+
646
+ Use `rpmbuild` to create an rpm of the project. This will also make a
647
+ source rpm. Requires all build dependencies by satisfied locally. Packages
648
+ are staged in `./pkg/rpm`.
649
+
650
+ * **package:srpm**
651
+
652
+ Use `rpmbuild` to create a source rpm of the project. Source rpm is staged
653
+ in `./pkg/srpm`.
654
+
655
+ * **package:tar**
656
+
657
+ Create a source tarball of the project. The tarball is staged in `./pkg`.
658
+
659
+ * **package:update**
660
+
661
+ Update the clone of the packaging repo by pulling from origin.
662
+
663
+ * **pl:build_from_params**
664
+
665
+ Invoke a build from a build parameters yaml file. The parameters file
666
+ should be created with `bundle exec rake pl:write_build_params`. The
667
+ settings in the build parameters file will override all values contained in
668
+ `./ext/build_defaults.yaml` and `./ext/project_data.yaml`.
669
+
670
+ * **pl:deb**
671
+
672
+ Use pdebuild with cowbuilder to create a debian package and associated
673
+ source artifacts from the default "cow", currently Debian Squeeze i386.
674
+ Requires that pbuilder/cowbuilder be installed and set up with a Debian
675
+ Squeeze cow. See the
676
+ [puppetlabs-debbuilder](https://github.com/puppetlabs/puppetlabs-debbuilder)
677
+ module for an easy way to set up a host for building with cows. The deb and
678
+ source artifacts are staged in `./pkg/deb/squeeze`.
679
+
680
+ * **pl:deb_all***
681
+
682
+ The same as `bundle exec rake pl:deb`, but a package is built for every cow
683
+ listed in `ext/build_defaults.yaml` on the line `cows:<cows>`. The packages
684
+ and associated source artifacts are staged in `./pkg/deb/$distribution`,
685
+ where $distribution is the Debian/Ubuntu codename of the cow that was used to
686
+ build the package, e.g. "wheezy" or "precise."
687
+
688
+ * **pl:ips**
689
+
690
+ Invoke package:ips, but do so after pl:fetch and pl:load_extras, which load
691
+ signing/certificate information. The resulting packages are signed. IPS
692
+ packages are staged in `./pkg/ips/pkgs`
693
+
694
+ * **pl:jenkins:deb**
695
+
696
+ Perform `pl:deb` by posting a jenkins build, as described above. Once the
697
+ build is complete, run `pl:jenkins:retrieve` to retrieve the built
698
+ packages.
699
+
700
+ * **pl:jenkins:deb_all**
701
+
702
+ Perform what is a effectively a `pl:deb_all` but in a different fashion.
703
+ `pl:deb_all` performs debian cow builds in serial with every cow listed in
704
+ ext/build_defaults.yaml. `pl:jenkins:deb_all` splits the cows listed, and
705
+ posts a separate `pl:jenkins:deb` job for every cow listed to the jenkins
706
+ server, allowing jenkins to parallelize the building of packages for every
707
+ cow. Execute `pl:jenkins:retrieve` to retrieve all packages.
708
+
709
+ * **pl:jenkins:deb_repo_configs**
710
+
711
+ On the distribution server generate a listing of all debian repositories
712
+ that exist for the current SHA/ref of HEAD of the project repository. Then
713
+ generate debian apt client configuration files for every existing
714
+ repository. The distribution server is a web server, so the client
715
+ configurations can be placed on a debian client in /etc/apt/sources.list.d/
716
+ and the client will be able to install the built packages via apt. Requires
717
+ SSH access to the distribution server.
718
+
719
+ * **pl:jenkins:deb_repos**
720
+
721
+ On the distribution server, generate debian apt repositories for every
722
+ distribution containing any packages that are stored that match the current
723
+ SHA/ref of HEAD of the project repository. Requires SSH access to the
724
+ distribution server.
725
+
726
+ * **pl:jenkins:dmg**
727
+
728
+ Perform `package:apple` by posting a jenkins build. Run
729
+ `pl:jenkins:retrieve` to retrieve the built packages.
730
+
731
+ * **pl:jenkins:gem**
732
+
733
+ Perform `package:gem` by posting a jenkins build. Run `pl:jenkins:retrieve`
734
+ to retrieve the built packages.
735
+
736
+ * **pl:jenkins:mock**
737
+
738
+ Perform `pl:mock` by posting a jenkins build. Run `pl:jenkins:retrieve` to
739
+ retrieve the built packages.
740
+
741
+ * **pl:jenkins:mock_all**
742
+
743
+ Perform what is effectively a `pl:mock_all` but in a different fashion.
744
+ `pl:mock_all` performs mock builds in serial with every mock listed in
745
+ ext/build_defaults.yaml. `pl:jenkins:mock_all` splits the mocks listed, and
746
+ posts a separate `pl:jenkins:mock` job for every mock to the jenkins
747
+ server, allowing jenkins to parallelize the building of packages for every
748
+ mock configuration. The mock build root is randomized by the packaging
749
+ repo, avoiding conflicts with existing builds of the same mock
750
+ configuration. To retrieve built packages, call `pl:jenkins:retrieve`.
751
+
752
+ * **pl:jenkins:post[uri]**
753
+
754
+ Post to the jenkins server as specified in the team build_extras.yaml file,
755
+ with the job uri specified.
756
+
757
+ * **pl:jenkins:retrieve[target]**
758
+
759
+ Retrieve packages stored on the distribution server that have been built
760
+ from the current SHA/ref of HEAD of the project repository. Optionally pass
761
+ [target] to override the default, which is to retrieve the contents of the
762
+ "artifacts" subdirectory. Other targets are "repos" and "shipped".
763
+
764
+ * **pl:jenkins:rpm_repo_configs**
765
+
766
+ On the distribution server generate a listing of all yum rpm package
767
+ repositories that exist for the current SHA/ref of HEAD of the project
768
+ repository. Then generate yum client configuration files for every existing
769
+ repository. The distribution server is a web server, so the client
770
+ configurations can be placed on a redhat client in /etc/yum.repos.d/ and
771
+ the client will be able to install the packages via yum install. Requires
772
+ SSH access to the distribution server.
773
+
774
+ * **pl:jenkins:rpm_repos**
775
+
776
+ On the distribution server, generate yum rpm repositories for every
777
+ distribution containing any packages that are stored that match the current
778
+ SHA/ref of HEAD of the project repository. Requires SSH access to the
779
+ distribution server. The yum repos are created in a "repos" subdirectory of
780
+ the standard builds location, e.g.
781
+ /opt/jenkins-builds/$project/${SHA|ref}/repos, using everything currently
782
+ in the "artifacts" subdirectory of the same location.
783
+
784
+ * **pl:jenkins:ship[target]**
785
+
786
+ Take the packages staged in pkg/ and ship them to locations partially
787
+ specified by data in the project build_extras.yaml file. The current
788
+ paradigm is to ship the files to a subdirectory of /opt/jenkins-builds on
789
+ the distribution server. The subdirectory is constructed with the project
790
+ and SHA or ref of HEAD of the project repository. That is, if project HEAD
791
+ is currently at the tag "1.2.3", then the directory that packages will be
792
+ shipped to is /opt/jenkins-builds/$project/1.2.3/. If HEAD is a git SHA,
793
+ then "1.2.3" will instead be that SHA. By default, all artifacts in pkg/
794
+ will be shipped to a "artifacts" subdirectory of the standard target. E.g.
795
+ /opt/jenkins-builds/$project/1.2.3/artifacts. When a final shipping occurs,
796
+ e.g. when shipping signed artifacts into production, a second subdirectory
797
+ is created - "shipped" - and all artifacts that are shipped to production
798
+ are also shipped here. This allows a historical archive of all shipped
799
+ artifacts.
800
+
801
+ * **pl:jenkins:sign_all**
802
+
803
+ Take all packages staged in pkg/ and sign them via the various signing
804
+ tasks. All signing occurs on the distribution server:
805
+ * create a git-bundle of the project and rsync it to the distribution
806
+ * server ssh to the distribution server and clone the git-bundle, and clone
807
+ * the packaging repository rsync the contents of the local pkg/ directory
808
+ * into the pkg/ directory of the remote git project ssh to the distribution
809
+ * server and execute the following rake tasks:
810
+ - pl:sign_tar
811
+ - pl:sign_rpms
812
+ - pl:sign_deb_changes
813
+ * rsync the remote pkg/ directory contents to the local pkg/ directory
814
+
815
+ * **pl:jenkins:tar**
816
+
817
+ Perform `package:tar` by posting a jenkins build. Run `pl:jenkins:retrieve` to
818
+ retrieve the built packages.
819
+
820
+ * **pl:jenkins:uber_build**
821
+
822
+ Create a jenkins job on the fly that performs an aggregate of build tasks.
823
+ These include all the debian builds using `pl:deb COW=<cow>`, rpm builds
824
+ with `pl:mock MOCKS=<mock>`, `package:tar`, `package:apple`, and
825
+ `package:gem` if applicable. See
826
+ [jenkins-tasks](https://github.com/puppetlabs/packaging#jenkins-tasks)
827
+ above for more detail.
828
+
829
+ * **pl:jenkins:uber_ship**
830
+
831
+ An aggregate of retrieval, signing, and shipping tasks. Execute
832
+ `pl:jenkins:retrieve` to retrieve any packages on the distribution server
833
+ that were built from the SHA/ref of HEAD. Then `pl:jenkins:sign_all` to
834
+ sign all packages. Finally, `pl:uber_ship`, `pl:remote:freight`, and
835
+ `pl:remote:update_yum_repo`. `pl:jenkins:uber_build` combined with
836
+ `pl:jenkins:uber_ship` performs the entire build and release process for a
837
+ project.
838
+
839
+ * **pl:mock**
840
+
841
+ Use `mock` to build an rpm package using the default mock distribution,
842
+ Redhat Linux 5, i386. Requires that the `mock` package be installed. See
843
+ the
844
+ [puppetlabs-rpmbuilder](https://github.com/puppetlabs/puppetlabs-rpmbuilder)
845
+ module for an easy way to set up a host for building with mock. Resulting
846
+ rpm is staged in `./pkg/<repo_name>/el/rpm/5/(i386 | SRPMS)`. The
847
+ <repo_name> is determined by whether the package is a final version or not.
848
+ Whether the version is final or not is determined by parsing the
849
+ `git describe` string - anything containing 'rc', 'SNAPSHOT', '-dirty', a
850
+ git sha, or a dash after the version (e.g. 1.0.0-22) is considered non-final.
851
+
852
+ * **pl:mock_all**
853
+
854
+ The same as `bundle exec rake pl:mock`, but a package is built for every
855
+ mock listed in `ext/build_defaults.yaml` on the line `mocks:<mocks>`.
856
+ Packages are staged in `./pkg/<repo_name>/(el | fedora)/$version/(i386 |
857
+ x86_64 | SRPMS)`. The subdirectories are dependent on the mock that is used.
858
+ The task assumes that the mock configurations are the standard Puppet mock
859
+ configurations that are generated by the
860
+ [puppetlabs-rpmbuilder](https://github.com/puppetlabs/puppetlabs-rpmbuilder)
861
+ module.
862
+
863
+ * **pl:print_build_params**
864
+
865
+ Print all build parameters to $stdout as they would be used in a package
866
+ build. This prints data that is loaded from `ext/build_defaults.yaml` and
867
+ `ext/project_data.yaml`, as well as whatever is overridden with environment
868
+ variables. Useful for debugging problems with parameter values.
869
+
870
+ * **pl:print_build_param[param]**
871
+
872
+ Print a specific build parameter to $stdout as it would be used in a package
873
+ build. This prints data that is loaded from `ext/build_defaults.yaml` and
874
+ `ext/project_data.yaml`, as well as whatever is overridden with environment
875
+ variables. Useful for debugging problems with parameter values. param
876
+ should be the name of the parameter as a symbol, e.g. :ref or :version
877
+
878
+ * **pl:remote:freight**
879
+
880
+ Performs an ssh call to the package server that calls a server-side rake
881
+ task. The rake task takes the debian packages that have (presumably) been
882
+ shipped via `pl:ship_debs` and invokes
883
+ [freight](https://github.com/rcrowley/freight) with them, which places them
884
+ in the apt repository.
885
+
886
+ * **pl:remote:update_yum_repo**
887
+
888
+ As with `pl:remote:freight`, this task performs an ssh call to the yum RPM
889
+ package server, and invokes an existing server-side rake task. The task
890
+ iterates through the `el` and `fedora` directories of the yum repository
891
+ and re-creates package server metadata for rpms in the subdirectories.
892
+
893
+ * **pl:ship_debs**
894
+
895
+ Rsync pkg/deb/* to the "incoming" directory on the debian apt package
896
+ repository server. Note: this task does not place the packages into
897
+ production - it is more accurate to consider the packages "staged" on the
898
+ repository server rather than actually shipped. The `pl:remote:freight`
899
+ task takes the packages in the "incoming" directory and actually places
900
+ them in the apt server.
901
+
902
+ * **pl:ship_gem**
903
+
904
+ Takes the built gem in pkg/ and pushes it to rubygems.com. This task assumes
905
+ you have the appropriate rubygems.com access and config to push the gem.
906
+
907
+ * **pl:ship_ips**
908
+
909
+ Takes the IPS packages in pkg/ips/pkgs/ and rsyncs them to a holding
910
+ directory on a package download server. This is not a true IPS server yet,
911
+ but just a basic file server. Eventually the goal is to have a true IPS
912
+ package repository running.
913
+
914
+ * **pl:ship_rpms**
915
+
916
+ Rsyncs the contents of pkg/el and pkg/fedora into the yum repository
917
+ server. While these packages are available immediately for download by
918
+ browsing the yum server directories directly, the yum repodata metadata has
919
+ not been updated, and thus the packages are not available to yum clients.
920
+ The `pl:remote:update_yum_repo` task updates the metadata, after which the
921
+ packages will be available to yum clients.
922
+
923
+ * **pl:sign_rpms**
924
+
925
+ Sign the rpms staged locally under pkg/ with the gpg key user ID (e.g.
926
+ email) specified in ext/build_defaults.yaml as `gpg_name`. This value can
927
+ be overridden by passing GPG_NAME as an environment variable to the rake
928
+ task.
929
+
930
+ * **pl:sign_tar**
931
+
932
+ Use gpg to create a detached signature of the tarball. By default this uses
933
+ the gpg_key value specified in ext/build_defaults.yaml in the project. This
934
+ can be overridden by passing GPG_KEY as an environment variable to the rake
935
+ task.
936
+
937
+ * **pl:tag**
938
+
939
+ Create a signed, annotated git tag of the current repository. Requires TAG
940
+ be passed as an environment variable to the rake task, which is the value
941
+ that will be used as both the tag message and the name of the tag. The gpg
942
+ key that is used for signing is assumed from gpg_key in
943
+ ext/build_defaults.yaml. This can be overridden by passing GPG_KEY as an
944
+ environment variable to the rake task.
945
+
946
+ * **pl:uber_ship**
947
+
948
+ A composite task that performs the following tasks:
949
+ `pl:ship_gem`
950
+ `pl:ship_rpms`
951
+ `pl:ship_debs`
952
+ `pl:ship_dmgs`
953
+ `pl:ship_tar`
954
+ `pl:jenkins:ship`
955
+ This is essentially a "ship all the things" task, but it is important to
956
+ note that it does not update either yum or apt repo metadata on these
957
+ respective servers - this has to be done via `pl:remote:update_yum_repo`
958
+ and `pl:remote:freight`.
959
+
960
+ * **pl:update_ips_repo**
961
+
962
+ Take the packages in pkg/ips/pkg and rsync them to the IPS repository
963
+ server specified in the build_extras.yaml file. Via ssh commands, call
964
+ pkgrecv, pkgrepo, svcadm on the IPS repository server. Via ssh, restart the
965
+ IPS repository service.
966
+
967
+ * **pl:write_build_params**
968
+
969
+ Generate a yaml file with all the build properties that have been loaded
970
+ from build_defaults.yaml, project_data.yaml, (optionally)
971
+ build_extras.yaml(s) via `pl:fetch`, and any environment variables. This
972
+ file can be used by the packaging repo as a single source of truth for
973
+ build data via `pl:build_from_params`. By default it is written to a
974
+ temporary location and its location is printed to $stdout. To override the
975
+ destination, pass OUTPUT_DIR as a environment variable to the task. By
976
+ default, the name of the file will be either the git tag, if HEAD of the
977
+ project repository is a tag, or the git sha of HEAD.